Now, can we lead on tackling its ethical and societal implications?

The news is flooded with examples of AI fails: algorithms that favour male job applicants over women, or image recognition software failing to correctly identify people of colour.

Dr. Foteini Agrafioti, the Head of Borealis AI and one of the country’s strongest voices on ensuring AI is ethical, was announced last week as the co-chair of Canada’s new Advisory Council on AI. She led the latest RBC Disruptors conversation about battling bias in AI with Dr. Elissa Strome, Executive Director, Pan-Canadian AI Strategy at CIFAR, and Dr. Layla El Asri, Research Manager, Microsoft Research Montréal.

Here are their thoughts on what the scientific community, governments and ordinary citizens can do to confront bias in artificial intelligence, and position Canada as a leader in ethical AI.

1. Use Technology to Expose Bias

Bias has long existed in our society – and so it exists in our data. El Asri sees this as an opportunity. Unlike our own unconscious bias, we can at least uncover bias in an algorithm. To do this, companies need to be auditing their AI for bias every step of the way, as the major labs are now doing. El Asri credited Canadian leaders, such as AI pioneer Yoshua Bengio, for developing a will in Canada’s tech community to develop AI in a responsible way.

2. Diversify the Industry

Right now, artificial intelligence is being developed by a very narrow subset of society: mainly highly-educated men who went to the same schools, and now live in the same cities. Only 18% of AI researchers are women, a fact that Strome called “terrible.” Organizations like CIFAR are working to bring more voices into the development of AI, with initiatives such as the AI for Good Summer Lab, a seven-week training program for undergraduate women in AI.

3. Diversify the Data

AI is only as good as the data it’s trained on. “If your data is not representative enough, your model is not going to work,” El Asri said. There needs to be more vigilance in ensuring data is representative — an area where Canada has a homegrown advantage. If you’re working with data collected in a multicultural country like ours, you’re likely working with data that represents different ethnic backgrounds. This kind of data will be essential to building technology that works for everyone, especially when it comes to something like health care.

4. Talk to the Social Scientists

Right now, it’s really just the tech community and policy-makers talking about issues that are going to transform our society. We need to broaden that perspective, building in consultation with social scientists as an integral part of the development process. A recent CIFAR initiative brought together computer scientists and social scientists for a day to discuss the social, legal and ethical implications of AI. “The computer scientists were so eager to get their advice and insights,” Strome said. Similarly, at Microsoft, El Asri noted that their AI and ethics committees are made up of people from different disciplines, including anthropologists and historians.

5. Educate the Public

“There’s a lot of fear and misunderstanding and myths about AI,” Strome said. Over the next few years, it’s going to be critical to bring the public into the AI conversation. People need to be aware of the positive implications, as well as the risks, that AI will have on their lives. The better the next generation understands AI and its societal and ethical implications, the better prepared they’ll be to ask tough questions of their leaders. Agrafioti suggested that Canadian culture is particularly attuned to ensuring fairness, casting a critical eye on technology before implementing it. Our balance of technical expertise and social values is exactly what’s needed to make sure the product that gets to market is ethical.

6. Strong Governance

AI has been advancing much faster than any government can regulate it — so it was big news this week when the OECD adopted a set of AI principles, which set values-based standards for developing AI. Our leaders have an incredibly important role to play in developing policy and regulations around the use of AI, both domestically and internationally. Strome noted that Canada’s solid international reputation could go a long way in urging the world to play catch-up. Last summer, Prime Minister Trudeau and President Macron announced a joint Canada-France initiative on an International Panel on AI to support and guide the responsible adoption of AI, grounded in human rights. The first symposium will be in Paris this fall.

Solving bias in machines will take a human touch — and there’s no country better positioned than Canada to take the reins.

Listen to our conversation on the RBC Disruptors podcast about the potential of artificial intelligence.

Listen on Apple Podcasts, Spotify or Simplecast

Ahead of the next RBC Disruptors event on May 23, “Battling Bias in AI,” our Thought Leadership team is examining the societal and ethical implications of artificial intelligence. In this interview series, John Stackhouse asks Layla El Asri, a Research Manager at Microsoft Research Montréal, about the role of humans in solving AI bias.

John: When did you first become aware and concerned about bias in AI?

Layla: I’ve really been thinking about it just for the last three or four years. After I joined Microsoft, that was about the same time that news stories were breaking about different issues surrounding bias and AI. There was the famous example with Google software, where their algorithm misclassified an African American as a gorilla. That was just one example of bias in the product because the data was not representative enough. It was very striking for me because it was really terrible.

Then there was this example, also with Google software, that if you would type a name would be mostly used within the African American community, then you would get ads about searching for a criminal record. And that’s bias in the system, because of bias in the way humans were using the system.

So those examples were really striking and really showed that things could go wrong if we weren’t more careful with the data that we were using in the models that we were putting out there.

John: As a scientist, how do you think about bias?

Layla: Your model can only be as biased as your data.

The model just tries to optimize its performance. It doesn’t ask, am I being fair? There are ways to put this into the model, but it needs to be put into the model. From a scientific point of view, that’s a matter of changing the objective of the models so that it not only tries to maximize performance, but is incentivized to not amplify bias or to try to reduce bias if possible.

John: So that opportunity to de-bug bias, if I can put it that way, is it as straightforward as you’re laying out? Just a matter of recoding?

Layla: You know, it really depends. It is possible in certain cases to try to kind of re-engineer the models so that it becomes less biased or unbiased. But in certain cases, it is just impossible. If you have data, for instance, that comes from human decisions — you might not even know of the bias that is present in the human decisions in the first place. And it’s a matter of really testing the model to see if it’s biased.

Sometimes in order to make it unbiased, you just need more data, especially when you have a problem with under-representation like you haven’t got data for darker skin in computer vision. There’s nothing you can do except collect data for darker skin and then retrain your models so that it learns about that data.

John: How do you at Microsoft come to grips with these challenges?

Layla: The way it’s been tackled here is very different. There are research groups that are dedicated to researching these questions — fairness, accountability, transparency and ethics. So questions like, what does it mean for a machine learning model to be fair? Fundamental questions that are yet to be answered. That’s at the research level.

And then there is also a committee within Microsoft which is called the AETHER Committee (AI and Ethics in Engineering and Research). This committee serves as kind of a consulting branch for product teams and leadership within Microsoft.

These groups know the technical issues with machine learning models, they know what they can and cannot do. And it’s really important to advise production and leadership about this, so we can all make an educated decision about whether or not it is safe to release machine learning at this stage or not.

Those are the kind of things that have been put in place within Microsoft as safeguards against unsafe use of AI, and ethical use of AI. And auditing has been a really impactful thing to do, too.

John: What kind of people are on these committees?

Layla: It’s actually a good mix of technical people and also people who have more of a sociological background. We have historians, we have anthropologists, sociologists, all working on these questions to try to understand the potential sociology consequences of certain patient learning technology.

John: You were talking earlier about the unintended consequences of machine learning. We have lots of unintended consequences with human learning and human decision-making. Should we be more confident in the ability of science to minimize the unintended consequences on the machine side?

Layla: You know in the future, I want to be optimistic that the answer will be yes.

But one really important flaw that I see right now, and which makes me lean towards no, is that the models are trained on historical data, so they cannot change. Unless you change their learning objectives, or you change the data that they were trained on. And currently they need a lot of data to learn something new.

Human beings, on the other hand, adapt very quickly. So if there is a problem with bias within your organization, you can talk to people and educate them and they will be able to react very quickly. A machine learning model right now will not be able to react very quickly. It will need a lot of data so it can be trained on it.

So I think that you know for the time being, as long as machine learning model cannot really adapt quickly and learn new things quickly, I think they have to work with human beings. And I think we have to find a way to have a good working relationship between machine learning models and human beings, so that we can leverage the amazing adaptation capabilities of human beings and the amazing computational and kind of number-crunching capabilities of machine learning models.

John: Maybe I can wrap up with a question about your own research in dialogue systems. Are voice and text biased? What should we, as consumers or producers of voice and text information, be thinking about?

Layla: If your model understands only certain people and certain voices and doesn’t understand for instance, the elderly or different accents, then you have a biased system because it doesn’t work for everybody. The good thing with human beings is that we kind of work with everybody, we kind of understand all sorts of accidents. Machine learning models might not always.

And even in text, if you only understand really well-formed English and your model doesn’t understand certain idioms or slang that might be used by certain communities, then you have a product that doesn’t really work for everybody and then you have a problem with bias.

You need to be able to understand all the people that you want to serve, really — all the people that you want your product to work for.

John: That’s fantastic. Really great insights. Thank you.

Layla: Great. Thank you.

Listen to our conversation on the RBC Disruptors podcast about the potential of artificial intelligence.

Listen on Apple Podcasts, Spotify or Simplecast

In this interview series, John Stackhouse speaks to Foteini Agrafioti, the Chief Science Officer at RBC and Head of Borealis AI, RBC’s Research Institute in Artificial Intelligence. Their conversation has been edited for length and clarity.

John: Can you give us an example of where you’ve seen bias in AI?

Foteini: One example that has been so interesting to me is gender bias in the Google translation engine. This wouldn’t have been immediately obvious to me beforehand, which shows how far our understanding has come. So, in this case I’m referring to the way we used to translate text from one language to another was by matching word to word, say from English to French.

However, when machine learning became more mainstream, we took a different approach. We fed documents – a lot of long texts – to deep networks in both English and French, and basically asked the machines to learn how the languages corresponded. The performance in translation increased dramatically, beyond what the scientific community anticipated.

What we didn’t anticipate is that bias was built into the documents about how the different genders are depicted. And that was learned by the machine and then perpetuated. So, a typical example now is that if you translate from English to French something with a doctor who is a woman and a nurse who is a man, the machine will mistake the genders because it has learned that more doctors are men and more nurses are women.

John: So how as a scientist do you address this?

It’s a tough one to address. If I could break down the problem here: primarily, the bias already exists in the data. It was baked into the real-world information collected by real people, about real people. In many cases it simply reflects our civic life and all the social structures in place that keep our societies afloat – for better and sadly, for worse. It’s not necessarily that there are bad actors manipulating the levers. The people at Google that built the system are good human beings. This was not intentional in any way on their part. It’s just that this bias was built into the most precious resource that we have.

So, number one is being aware of this pre-existing risk. Don’t assume that data is an objective entity. And I think companies are beginning to recognize that that is the case. It’s the recognition that AI is not only a technology that allows you to transform the way you operate, but it’s one that truly opens up brand new risks. Some of them you can anticipate — but you have to also remember that a lot of them can blindside you. Simply put, we don’t know what we don’t know – yet. So, until we do it’s imperative to have a backup plan for how you’re going to both anticipate and mitigate these risks.

John: One of the interesting emerging areas for bias is biometrics. What should we understand about the risks of bias when it comes to biometrics?

Foteini: Biometric security has always been a very sensitive area. By definition, biometric data have personally identifiable information, which is extremely sensitive if comprised.

What AI brought into the mix is this unprecedented accuracy in human authentication to biometric modalities. So for example, facial recognition. The way biometrics used to work, if you want to recognize somebody consistently you would want them to maintain the same facial expression, not grow a beard, never put makeup on, make sure things are consistent.

But now, with deep learning, all of that vulnerability can be tolerated — even if you grow a beard, change your hairstyle, we can actually still recognize you.

So, now in addition to sensitivity, we have to worry about bias. There’s very systemically an under-representation of so-called ethnic backgrounds in data systems. So the recognition technology is very powerful, but because of inconsistencies in the number of skin tones the models were trained on, it doesn’t work for certain people or the algorithm will systematically pick out particular ethnicity.

Foteini: With regards to the decision in San Francisco, this type of ethical dilemma is really what we should be thinking about, across the public and private sector.

From my purview, this was an excellent, bold move, recognizing that there are risks with the technology and not just jumping on board before taking stock of the potential misuses. It was clear they put public safety first, despite the many useful applications of the technology. So, taking a step back and deciding you know where are you going to draw the line? That was a really, really big deal.

John: But what will we be giving up as citizens and consumers, with these tighter controls, if they are indeed expanded?

Foteini: Security and convenience is usually what’s enabled with biometric systems.

There was a very interesting case of this a few years back here in Ontario with the casinos, where face recognition was being deployed in order to automatically recognize and stop problem gamblers from entering casinos. So people would self-register with the system and the expectation was that if they showed up at a casino in Ontario, they would be stopped from entering by security. Now that list grew very big. The security guards weren’t able to memorize all those faces. So facial recognition came into play as an interesting way of potentially recognizing them and stopping them from entering. It was a huge privacy concern at the time, and I think the privacy commissioner has done an incredible job figuring out how to enable privacy in that context.

John: What risk is there, that other countries — and I’m thinking specifically of China — will move ahead much more quickly scientifically? Because they don’t have these sorts of restrictions or even concerns about individual rights.

Foteini: Well, I want to believe that they do. And I certainly see the scientific community across the world putting pressure on how these systems are being designed, deployed, and adopted broadly. But assuming that that is not the case and you can just do anything you like…it’s not just a head start. It’s leapfrogged many, many, many steps ahead.

The ability to have unconstrained access to data, test systems, deploy them in the real world, get feedback and isolate without any consideration for things that can go wrong? I mean, I would learn a lot more about how to build a better surveillance system if I’m able to deploy a surveillance system and get data and insights out of that.

John: I’m thinking of an analogy of athletes who have to compete with athletes from other countries that use steroids. I don’t know if that’s a fair comparison. But as a scientist, do you feel like you’re losing ground to your Chinese peers?

Foteini: I’m generally trying to stay on the positive side.

One great thing that the academic community has done, especially in the machine learning/computer science space, is they require any new state-of-the-art machine learning system to be benchmarked and tested and proven on a public data set. So we’re all working on the same standard. They are holding people accountable to that.

John: Do you believe that the AI community will be able to smooth out the challenges of both the data collection that you have you referred to, and the initial coding of algorithms, to a reasonable degree that bias will be contained?

Foteini: It’s hard, to be honest. And I don’t want to be pessimistic here about it. But it’s very hard to solve.

And in many cases you may not want to solve for it. Some biases are good.

John: Give us an example.

Foteini: The decisions that we make when we’re driving our vehicles. I know we’re constantly talking about a responsible, self-driving autonomous vehicle. But sometimes it is human bias that saves us from accidents, like a driver’s bias towards driving slower than the speed limit when it’s wet or when they’re rounding a corner.

Or in trading, I can build a system that trades in a consistent way, or I could benefit from the special intuition that an extreme trader has — that’s a place where we may want to keep that bias.

John: Is there a competitive advantage for RBC or indeed for Canada in having this approach to bias?

Foteini: What’s beautiful about Canada is that most of the data that we have, organically, is very diverse. And if you’re building the next generation diagnostic system using MRI technology and you’re using data that was collected in Canada, you’re very likely to be working with data that is very well represented by different ethnic backgrounds. So that’s our home advantage.

The other aspect that is unique here, and part of our Canadian values, is respect for each other and tolerance — and I think we apply a very critical lens when we look at AI technologies and what they could do to our society. I think we have a lower tolerance for risk in imbedding these in our lives. We’re being critical of them and that’s good. That type of pressure is sometimes necessary to get the scientific community and the companies as well to push in the right direction when developing the next technology.

John: This is great Foteini. Thank you.

Foteini: Yes, thank you.

Listen to our conversation on the RBC Disruptors podcast about the potential of artificial intelligence.

Listen on Apple Podcasts, Spotify or Simplecast

Ahead of the next RBC Disruptors event on May 23, “Battling Bias in AI,” our Thought Leadership team is examining the societal and ethical implications of artificial intelligence. In this interview series, John Stackhouse asks the Executive Director of CIFAR’s Pan-Canadian AI Strategy – Elissa Strome about the difference that diversity could make. Their conversation has been edited for length and clarity.

John: AI is reshaping society in many ways, both good and bad. What are you most concerned about?

Elissa: Currently, I am most concerned with the lack of equity diversity and inclusion in AI as it’s being developed around the world.

In Element AI’s Global AI Talent Report, they assessed the demographics of AI researchers around the world, and the quality and quantity of AI research that’s being undertaken by different countries.

There are essentially only five countries in the world that are advancing the science and applications of AI.

John: Who are the five?

Elissa: It’s Canada, the US, the UK, China, and Australia. These are the countries that are heavily involved in the development of AI so it’s very North American-centric and Western European-centric.

China is a huge leader in this space, but there’s a lot of the world that is not as deeply involved and engaged in developing and advancing AI. If we really expect that these technologies are going to have a positive impact on global society, then global society needs to be involved and active in their engagement.

The other area where there’s a real problem around equity, diversity, and inclusion in the development of AI is gender balance. If you look at the number of researchers who were authors at the major international conferences in AI, only 18 percent of those researchers are women.

John: Do you know what that number is in Canada?

Elissa: It’s probably right around that, maybe slightly higher. In the Canada CIFAR AI Chairs program, we currently have 20 percent women named as Chairs. CIFAR is working with partners across the country in a lot of different training programs and learning opportunities, particularly for young women in AI. This is one of the ways that we are addressing the gender gap in Canada.

We work with the Invent the Future program at Simon Fraser University for instance that is focused on engaging high school girls and giving them exposure through a two-week summer camp. It provides them with an understanding of what AI is, and what the future opportunities are for them and their careers.

We also recently announced a new partnership with the OSMO Foundation in Montreal to advance the AI for Good Summer Lab. This is a program that was developed by Doina Precup, who is one of our Canada CIFAR AI Chairs. It is a seven-week training program for undergraduate women in AI. The women enrolled in the lab gain exposure to training and networking opportunities that will serve as a foundation for their future careers.

John: How did we get to this point where four out of five AI scientists are male?

Elissa: It’s a historical problem. In the early 90s, the rates of female computer science students in universities was actually more like 30 percent. Some of the enrollment rates of women in computer science were higher in the 90s than they are today.

I think there were a variety of reasons for why women became less interested, less encouraged, or less mentored. As the number of women leaders in the field started to decline, fewer women enrolled in these training programs.

I think it’s also sort of a cultural thing within computer science. There’s a bit of a nerdy coding guy false image of what it means to be an AI researcher. We have a lot of work to do.

John: What would a Canada AI researcher look like if they’re not a coding nerd?

Elissa: Today, an AI researcher can look like anybody in almost in any discipline. That’s the great thing about AI. It’s a mathematical and computational approach to understanding and leveraging data. And so every discipline of science and research actually has the opportunity to leverage AI, which goes back to my point about the need to increase diversity.

We need to be encouraging students in other disciplines to develop interests and expertise in AI. Not necessarily in coding but thinking about how machine learning can be applied to business questions, or questions in law, humanities, biological sciences, engineering, or physics.

John: Can you help us understand how all of this matters? If I’m thinking of AI recommending something to me on Netflix, why does it matter who is doing the coding?

Elissa: It matters because these algorithms, approaches, and recommendation systems will and already are starting to be applied to many areas of our life right now.

When you’re applying for a mortgage at the bank, when you’re applying for insurance, when you are applying for a job, recommendation systems are already being used without an average person knowing about that or being aware of what the issues are.

Recommendation systems are being designed by people who have a very specific perspective on life and the world, they come from a very homogeneous subset of the population, and their perspective is implicit in the development of the systems.

The other important underlying factor is where the data sets are being drawn from. If the data sets that are used to develop recommendation systems are drawn from samples that already have biases in them, whether they’re gender bias, racial bias, bias against disability, bias against any sort of underrepresented group, then those biases get amplified.

It’s not just that the people who are developing these systems may not represent diverse perspectives, but also the data sets themselves that have problems in their representation.

John: There’s been an explosion of concern about responsible AI including here in Canada. Did science take a wrong turn in the early years or was that just a function of maturing that there was suddenly this concern about responsible AI?

Elissa: I wouldn’t say that science took a wrong turn.

I would say that the technology, science, and the adoption and commercialization of the technology happened very quickly. And because they were both advancing at the same time, it’s just taken a while for the policy side of things to keep in step with the development of the technology.

At this point in the growth and adoption of AI, we as a society must take a very hard look at these technologies, and ask questions about the impact of bias. It’s absolutely critical that we invest a lot of effort and resources in research on the societal implications of AI.

At CIFAR, it’s the fourth pillar of the Pan-Canadian AI Strategy to advance leadership, research, and knowledge around what are the societal implications of AI – social legal ethical economic questions around AI

John: In 2018, we get the Montreal Declaration for Responsible Development of AI – is that sufficient?

Elissa: It’s a great start and it’s absolutely wonderful that there’s a sort of a Canadian pioneering path towards engaging the public in this conversation in this discourse.

And it is one of the few examples worldwide where the public were so deeply engaged as were researchers across many different disciplines, not just computer scientists but also social scientists and humanists as well. But it’s not sufficient.

John: Who should hold science accountable?

Elissa: I think it’s more of a question of holding society accountable. Governments have an incredibly important role to play, both in developing strong domestic policy and regulations around the use and adoption of AI. They also have a role to play in international cooperation, relations, and international policy on this topic, and again Canada is a leader in this space.

Last June, Prime Minister Trudeau and President Macron announced a joint Canada-France initiative on an International Panel on AI, and the first international symposium will be in Paris this fall.

This work will engage G7 countries within the umbrella of the G7 cooperation, but it will be an international collaboration to monitor, observe, and understand best practices around the societal issues related to AI.

John: Are you confident governments can get their heads around this?

Elissa: I am confident that their intentions are good, and I think they understand both the risk and the opportunity. The challenge with governments is the short duration of their terms.

But this is an issue that crosses party lines. This is an issue that affects all Canadians no matter what your political stripes are. It affects people all across the world, and so it’s something that governments, no matter what their sort of ideology is, really have to have to take a leadership role in.

John: What role can Canada play?

Elissa: Canada has a privileged position on the international stage in advancing our responsible use development and adoption of AI.

Much of that is based on our long history of pioneering the science and research, and our track record of research leadership. We also have a strong reputation for our work on the world stage in advancing humanitarian issues around justice, social rights, and freedoms. We are respected internationally both on the science and our democratic values.

The world is looking to Canada right now to take a leadership position, and we are out there and we are doing that.

Listen to our conversation on the RBC Disruptors podcast about the potential of artificial intelligence.

Listen on Apple Podcasts, Spotify or Simplecast

From the route you take home to where you choose to go for lunch, we have our own inherent preferences that predispose us to a particular decision.

And while we assume that our choices are rational, we know that isn’t always the case. Sometimes we can’t explain why we prefer one option over another.

Though that might be acceptable when it comes to choosing between a Coke and a Pepsi, it is exponentially different when we are asking an artificial intelligence (AI) system to make judgements and predictions on something of significant human consequence.

Bias persists in every corner of our society, so it should be no surprise that we see it in AI.

More than 50 years ago, computer scientist Melvin Conway observed that how organizations were structured would have a strong impact on any systems they created. This has become known as Conway’s Law and it holds true for AI. The values of the people developing the systems are not just strongly entrenched, but also concentrated.

In Canada, AI talent and expertise is concentrated in three cities: Edmonton, Toronto, and Montreal. Each city has a leading AI pioneer who people and funding have coalesced around.

Economically, this concentration has led to greater competitiveness and served as a magnet to attract and retain the best AI talent.

And it’s working. At the Canadian Institute for Advanced Research (CIFAR), over half of their 46 AI research chairs were recruited to Canada. But with this concentration also comes challenges that Canada and other global AI hubs are struggling with.

These AI tribes are overwhelmingly homogenous, futurist and author Amy Webb has found. They all attend the same universities, are affluent and highly educated, and are mostly male.

This finding was reinforced by the World Economic Forum’s most recent Global Gender Gap Report, which found only 22% of AI professionals globally are female. In Canada, it is marginally better where 24% are female.

The gender imbalance is worse when you look at who is studying AI. For every woman researching AI, an Element AI study found there were on average nine men.

But the nature of how we develop AI is that it learns what the data shows them and inherits its creators’ unconscious biases.

We see AI’s gender imbalance reflected in the systems being developed. After a visual recognition system was trained on a gender biased data set, which associated the activity of cooking 33% more often with women, the trained model amplified this disparity to 68%.

These tribes are also overwhelmingly white.

Dubbed a “Diversity Disaster” in the AI Now Institute’s report released last month, only 2.5% of Google’s workforce is black, Facebook and Microsoft are each at 4%, while data on LGBTQ participation in AI is non-existent.

As a multicultural society, the lack of diversity in AI should trouble all Canadians.

From setting insurance rates to policing, health care, student admissions, and more, every day this issue becomes more urgent as AI is increasingly integrated into society.

In the next few years, Gartner predicts that 85% of AI projects will deliver erroneous outcomes due to bias in data, algorithms, or the teams responsible for managing them.

We cannot afford to tip toe around the uncomfortable conversations of sexism, racism, and unconscious bias, and how they are making their way into AI.

Suppose a medical company wants to develop an AI system to find the best treatment plan for patients. To do so, they must determine their objective. Is it to minimize the cost of treatment? Or is it to maximize patient outcomes?

If the goal is to minimize costs, the algorithm could decide an effective way to achieve its objective is to recommend less effective treatments based on a cost-benefit analysis of probability of success to cost.

When organizations are setting objectives, whether for personalized medicine or another application, these decisions are often made for business reasons, rather than fairness or discrimination.

Bias can also be introduced through how the data is collected. If the data collected is unrepresentative or reflects existing prejudices, you could intentionally avoid race as an explicit input and still introduce racial bias.

For example, research from United Way Greater Toronto found inequality and race in the Toronto region increasingly divided by postal code. An organization could well intentionally avoid race, and still have a fundamentally flawed output if it used postal code as an input.

Yet, not all of what AI systems output is easily explainable to us. Businesses want systems with the most accurate predictions, but it’s those models that are also the most complex and least understood.

From the songs suggested to us to the ads we see, even the engineers who built these systems cannot fully explain how they arrive at those recommendations. And while that may be okay when it comes to your Spotify playlist recommendations, how comfortable would you be taking medical advice from an AI?

This is the challenge that researchers at Deep Patient faced. Trained on data from 700,000 patients, the AI system was making what turned out to be very good predictions of disease, including early stage liver cancer, but the researchers could not explain how it arrived at these conclusions.

Without explainability, how comfortable would a medical team be in acting on these predictions? Would they be willing to change medication, administer chemotherapy, or go in for surgery?

The issue is not AI, but how we build it.

We must recognize society’s deep social issues captured in human data, and root them out before it becomes encoded. Currently,
bias is often an after-thought – a problem to be solved at the end of the development cycle.

Instead, non-technical sociological experts need to be incorporated throughout the development cycle to work side-by-side with those doing the coding.

We must treat AI systems as we would a new pharmaceutical drug, and ensure it undergoes rigorous testing to understand potential side effects and identify any biases that could impact its output prior to release.

Nor should the evaluation cease after release. AI systems should be continuously monitored and regularly re-evaluated across their lifecycle. A health study on the half-life of data suggests that clinical data has a short window before its usefulness radically declines. After only four months, an AI system could be making decisions using outdated data with serious human consequences.

But despite the fear of AI and its impact on the future of humans, every possible solution to mitigating bias and improving AI requires human intervention.

A symbiotic human-AI relationship is the concept behind human-in-the-loop (HITL) systems. A technical term for a class of systems that are not fully autonomous, HITL is used where there is a low level of confidence, and the cost of errors is high, such as in health care.

Whenever there is low certainty, HITL systems will automatically loop in humans. This is critical for health care where practitioners are interacting with the output, engaging them in the process enhances explainability and provides comfort to act on the resulting output.

What’s clear is that humanity must be at the core of AI development.

Launched in December 2018, the Montreal Declaration for Responsible Development of Artificial Intelligence developed an ethical framework that calls for AI guidelines to ensure it “adheres to our human values and brings true social progress.” It has been signed by 45 organizations and 1,394 citizens.

Earlier this year, the Council of Europe adopted their own declaration on artificial intelligence, which calls on states to ensure that “equality and dignity of all humans as independent moral agents” in an AI-driven world.

In the United States, lawmakers have introduced the Algorithmic Accountability Act, which will require organizations with revenue over $50 million per year, who hold information on at least one million people or devices, or primarily act as data brokers that buy and sell consumer data to audit their systems for bias.

But who will keep the signatories accountable? If it is a role that will be undertaken by government, who provides oversight over how governments use AI? What happens when governments disagree on how AI can or should be deployed?

There are two prevalent world views for AI use. One that is predominantly Silicon Valley-driven is focused on consumerism and value-creation. The other perspective comes from the other side of the Pacific, where the Chinese government views AI as an instrument for social governance.

In the face of these questions and challenges exists an opportunity for Canada to lead the way as a global leader in advancing AI for good. This was one of our 10 recommendations to make Canada AI-ready.

Though Canada was the first country with a national AI strategy, we have fallen behind in capitalizing on its application and commercialization. An ‘AI for Good’ strategy alongside Canada’s AI supercluster, and a highly anticipated national data strategy, could enhance Canada’s competitiveness as a global AI hub and help us to reclaim our position as a driving leader in artificial intelligence.

The question is whether we are willing to ask the hard questions.

AI is holding up a mirror to society and forcing us to confront the polite fictions that we tell ourselves. We can no longer afford to look away.

Listen to our conversation on the RBC Disruptors podcast about the potential of artificial intelligence.

Listen on Apple Podcasts, Spotify or Simplecast

This happens hundreds of times a week, according to Al Lindsay, Amazon’s VP of Alexa Engine Software.

With 80,000 skills and growing, marriage is maybe the one thing Alexa can’t do.

This year, the smart speaker industry will grow to be worth an estimated $7 billion, and demand could potentially surpass that of smartphones.

A Canadian voice tech pioneer, Lindsay joined us at RBC Disruptors, our monthly conversation on innovation and technology, to share his insights into a voice-first world.

Listen on Apple Podcasts, Spotify or Simplecast

Here are 5 takeaways about the future of voice:

1. Voice Tech Is Making Smartphones Look Dumb

A lot of people ask, “Why do I need voice tech? I can do everything I want on my smartphone.” For Lindsay, his ah-ha moment was listening to music: “I’d say, play songs from Sting and two seconds later music would be streaming from the speaker.” No more taping on glass, navigating through apps and waiting. We used to see smartphones as the ultimate convenience. Now, voice tech has usurped them, eliminating the need to pull out your phone — and possibly get even more distracted. Voice could allow us to get back to being humans again.

2. Personality is Everything

In the beginning, Lindsay and his team were focused on making a voice tech assistant that was smart and helpful — those seemed like the obvious selling points. But customers leaned into Alexa as a persona. “It turns out fun and a sense of humour are appreciated as well,” Lindsay said. The company started adding quirks and funny responses, realizing the more natural interacting with voice tech feels, the easier people will adopt using it.

3. Voice Tech Is Disrupting Every Industry

When asked about what industries voice tech is disrupting, Lindsay was definitive: “All of them.” He predicted major changes from health care to hospitality. For hospital patients, smart speakers could handle simple requests like turning on the television or adjusting the bed, saving time and labour. When people go out to dinner, they’ll use voice to order a drink, and to charge the bill to their Visa — “or to the table over there,” he joked.

4. Every Business Needs a Voice Strategy

As we move to a more ambient world, small and large businesses alike will need a voice strategy. Similar to deciding on brand colours and slogans, companies will need to ask: Is our voice soothing? Young and fun? More serious? Sound adds a powerful new dimension to brand identity. It also opens up new opportunities. You don’t need to have speech team to have speech as an interface. Anything that has a computer interface, you can replace or augment with voice.

5.Voice is the Great Equalizer

Every interface before voice had a learning curve — it takes time to learn to type, or navigate an app. Lindsay said the distinction for voice-tech is that if we can truly achieve a natural conversation experience, technology will be immediately accessible to everyone in the world with language skills.

A lifelong entrepreneur, he’s also seen what it takes to succeed in business — and how failing is vital to learning how to get things right.

Today, the company pulls in $370 million annually in revenue and boasts 250 franchises around the world.

“We wouldn’t have gotten there if we didn’t fail,” Scudamore said. “We would never have gotten to where we are today if we didn’t continue to have the courage to make mistakes, and then learn from every one of those.”

One early mistake? “I realized that I was a terrible leader.”

At RBCDisruptors, our monthly conversation on innovation and technology, Scudamore talked about being willing to fail and the leadership lessons he’s learned along the way.

Listen on Apple Podcasts, Spotify or Simplecast

Here are some of his insights:

1. Ask Questions

When Scudamore set his sights on franchising, he went out and talked to 12 people. All 12 of them told him not to do it, that the junk removal business wasn’t franchise-able. He took the opportunity to ask questions, make changes, and tweak the model to make sense as a national business. Entrepreneurs need to have the boldness and the brashness to stand up and ask.

2. Paint a Clear Picture

Focus on what the future looks like, write it down, and share it with everyone. When Scudamore decided to scale 1-800-GOT-JUNK, he told his team the goal was to be in the top 30 metros. The naysayers left. Those who stuck with the company understood the goal, believed in it – and helped him to pull it off.

3. Make Tough Calls

Scudamore says the toughest thing he ever had to do was fire best friend, the company’s Chief Operating Officer. Together, they made rash decisions and almost bankrupted the business. “The right decision is usually a hard one.”

4. Be Out Front

Scudamore didn’t always know how to mentor. “I was hiding in my private office, not spending time with my team,” he said. “I didn’t want to. I was afraid of them, afraid of conflict.” He learned to put himself out there, and realized that leadership isn’t just about growing a business – it’s about helping people to grow.

5. Hire People You’d Want to Have a Beer With

“I learned how to build a team and culture through failure,” he said. Five years into the business, he had 11 employees — and 9 of them were “bad apples.” The workplace became toxic. He called a meeting, apologized – and let everyone go. He rebuilt by finding the right employees, hiring people who wanted to be friends with – who shared his passion for growth, and would represent the company well.

6. Get Out of the Way

Scudamore committed to a bucket list goal on the company’s vision board: be featured on the Oprah Winfrey show. It was clearly there for all to see, and an eager young employee took up the challenge, reaching out to the show and developing relationships until the call came in: there was an opportunity for 1-800-GOT-JUNK to be featured. “I did nothing to make that happen,” Scudamore said. “Tyler got out there and did all the work to pull it together.” You know you’ve gotten leadership right when your team can handle the execution.

Every entrepreneur makes mistakes, but you can’t beat yourself up when things go sideways, Scudamore said. Take a breath, and ask yourself what’s the one positive thing that can happen from this seemingly challenging situation. He finds the list is often longer than you’d think.

“Failure should be your friend,” Scudamore said. “Every single mistake we made we had to make to get to the next level”.

“I like to fix things. I always liked to fix things,” Chen said at the latest RBCDisruptors, our monthly conversation on innovation and how technology is changing the way we work, live and play.

He’s had plenty to fix at BlackBerry, the iconic Canadian tech company he took over as executive chairman and CEO in 2013. One of his first tasks back then was to announce a record $4.4 billion quarterly loss. The company got back in the black after a pivot to software and services, which now drive 90% of its revenue. The numbers are far below the good old days — but they’re climbing.

BlackBerry’s CEO says he’s driven to do what others think isn’t possible. Pulling that off takes a clear head and a sharp understanding of the larger trends around you.

He spoke to our audience about trade tensions between the U.S. and China, the future of mobile devices and how artificial intelligence is changing cybersecurity.

Listen on Apple Podcasts, Spotify or Simplecast

Here are some of his insights:

1. Huawei Will Have to Open Up

In the emerging cold war of technology, 5G is the latest battleground. Chen says China’s Huawei will eventually have to surrender its code for examination if it wants to win access to Western markets. “I see no other way for Huawei to get back in the game, without doing that,” he said. The reality is, Huawei isn’t the only supplier of emerging technologies like 5G routers and modems, and Western governments can move on without them. They can turn to other firms like Ericsson.

2. We May Have Seen Peak Phone

Chen says Apple got the inflation curve wrong, and the $1,000 phone isn’t going to be the next big thing. There’s not enough innovation to justify the eye-popping prices we’ve seen on new models.

3. Cars Are the New Smartphones

Vehicles are increasingly technology devices — and they’re going to need secure software to protect them. Chen sees this as BlackBerry’s big opportunity. Its QNX technology is already being used in 120 million cars. Amid concerns about hackers driving cars into ditches and drones into plane engines, BlackBerry’s goal is to make QNX the standard automotive operating system — what Windows is to PCs.

4. The Cyber Battle is Going From Defense to Offence

If you’re focusing on your most recent hack, you will lose. Security is a cat-and-mouse game that’s top of mind from the armed forces to hospitals to major banks, and both the hackers and defendants are turning to artificial intelligence to get them ahead of the other side. BlackBerry made its offensive play last fall, spending $1.4 billion to acquire Cylance — a cybersecurity company that uses AI to predict future attacks.

5. BlackBerry Doesn’t Want Your Data

Going up against giants like Apple and Google means focusing on your strengths. For BlackBerry, that’s security and privacy. It’s the heritage of the company, and its bet for the future. Unlike its competitors, BlackBerry isn’t trying to manage and monetize data. “We don’t take a look at any of the data. We don’t use it, we don’t care about it. We actually have an algorithm to discard it.”

6. Relish Your Losing Hand

Chen’s approach to business borrows from his love for contract bridge. He likes the game because it’s not about having a lucky hand. You get points based on how well you play the hand you’re dealt. “It’s really about, could you do better with what you’ve got.” That’s the challenge that drew him to BlackBerry. It’s a tougher road, but play it well, and it’s a higher playoff.

Smarter technology doesn’t mean we can sit back and let Alexa take the wheel. It means making our own pivot, and ensuring our society is not only tech-dependent but tech-savvy.
https://www.youtube.com/embed/bGCnjw2_n30

Between Toys “R” Us shuttering all its U.S. stores and tech-savvy toddlers playing with mobile devices, he might not be feeling too jolly about the future of toys. How can an Etch-a-Sketch possibly compete with an iPhone?

Right here in Toronto, we have a monster truck of a toy company who says it can be done.

Spin Master was founded in 1994 by three friends fresh out of university. They had early hits with the Earth Buddy and Devil Sticks; but it’s when they started investing heavily in R&D that their toy company took off. Today it’s one of the top five toy companies in the world, with sales exceeding $1.5 billion last year.

The global toy market is growing at 3–4 percent a year, mainly in Asia. In the U.S., parents spend an average of $350 on toys per child per year. And with R&D investment, toys can be every bit as innovative as our digital devices. Look no further than Spin Master’s Hatchimals — toys that peck their way out of egg when cuddled and rubbed.

I recently sat down with Spin Master Co-Founder and Co-CEO Ronnen Harary for our December session of RBCDisruptors.

Listen on Apple Podcasts, Spotify or Simplecast

Here are 6 lessons for innovators everywhere:

1. Build Your Own Empire

Spin Master created the cartoon phenomenon Paw Patrol by working backwards. About one-in-four toys is a licensed product, associated with a hit movie or show, but the bigger brands were always beating Spin Master for the rights. So instead of competing for rights to a franchise they didn’t own, Spin Master created an empire.

2. Use Tech As a Value-driver

A few years ago, the interactive technology to create a Hatchimal would have put its price point at $250 — way too high for the toy aisle. Today, you can buy one for about $80. Use technology to do things that haven’t been done before, at a price the market likes.

3. Respect Your Talent

Toy inventors are hard to find — there are only a couple hundred worldwide. Those relationships need to be nurtured. The duo who developed Zoomer, a voice-activated dog that responds to its name, have continued to create new toys for the company. Unlike many toy companies, Spin Master encourages inventors to stay involved in the process, and to share in the glory — including onstage when Zoomer won the 2014 Innovative Toy of the Year.

From left to right: Spin Master’s Zoomer, Hatchimal and Bakugan toys.

4. Stand Out From Your Competition

In the 1990s, Spin Master created “Key Charm Cuties” to compete head-to-head with Mattel’s Polly Pocket; but they couldn’t lure customers away from a trusted favourite without a significant point of difference. The lesson? Kids won’t settle for anything less than new and awesome.

5. Always Have Something in the Pipeline

Spin Master had a blockbuster hit with Bakugan, growing from a $450 million company to an $800 million company in just 21 months. But when interest in Bakugan dipped, it led to a major downturn for the company. Kids grow up, fads fade, and you always need to have your next great idea in the pipeline.

6. Don’t Obsess Over Your Failures

Not every toy is going to fly off the shelves. When something fails, no one person should be blamed — just like no one person should take credit when something succeeds. “Don’t ruminate about it,” said Harary. Move on to the next thing.