Margaret Mitchell, researcher and chief ethics scientist at artificial intelligence developer and collaborative platform Hugging Face, is a pioneer in responsible and ethical AI.
One of the most influential narratives around the promise of AI is that, one day, we will be able to build artificial general intelligence (AGI) systems that are at least as capable or intelligent as people. But the concept is ambiguous at best and poses many risks, argues Mitchell. She founded and co-led Google’s responsible AI team, before being ousted in 2021.
In this conversation with the Financial Times’ AI correspondent Melissa Heikkilä, she explains why the focus on humans and how to help them should be central to the development of AI, rather than focusing on the technology.
Melissa Heikkilä: You’ve been a pioneer in AI ethics since 2017, when you founded Google’s responsible AI ethics team. In that time, we’ve gone through several different stages of AI and our understanding of responsible AI. Could you walk me through that?
Margaret Mitchell: With the increased potential that came out of deep learning — this is circa 2012-13 — a bunch of us who were working on machine learning, which is basically now called AI, were really seeing how there was a massive paradigm shift in what we were able to do.
We went from not being able to recognise a bird to being able to tell you all about the bird. A very small set of us started seeing the issues that were emerging based on how the technology worked. For me, it was probably circa 2015 or so where I saw the first glimmers of the future of AI, almost where we are now.
I got really scared and nervous because I saw that we were just going full speed ahead, tunnel vision, and we weren’t noticing that it was making errors that could lead to pretty harmful outcomes. For example, if a system learns that a white person is a person and that a Black person is a Black person, then it’s learnt white is default. And that brings with it all kinds of issues.
One of my “aha” moments was when I saw that my system thought that this massive explosion that ended up hurting 43 people was beautiful because it made purples and pinks in the sky [after] it had learnt sunsets!
So it was so clear to me that we were full speed ahead in the direction of making these systems more and more powerful without recognising the critical relationship between input and output, and the effects that it would have on society.
So I started what’s now called responsible AI or ethical AI. If you’re at the forefront and you see the issue and no one else is paying attention to the issue, you’re like, “I guess I have a responsibility to do something here.” There was a small set of people at the forefront of technology who ended up giving rise to responsible AI.
[Computer scientist and founder of advocacy body the Algorithmic Justice League] Joy Buolamwini was another one who was at the forefront of working on face [recognition]. Then she saw the same kind of bias issues I was noticing in the way that people were being described. For her, it was how faces were being detected.
AI Exchange
This spin-off from our popular Tech Exchange series of dialogues examines the benefits, risks and ethics of using artificial intelligence, by talking to those at the centre of its development.
See the rest of the interviews here
[Around the same time] Timnit Gebru, who I co-led the Google ethical AI team with, was starting to realise this technology that she had been working on could be used to surveil people, to create disproportionate harm for people who are low income.
MH: I remember that era well, because we started to see lots of changes, companies applying ways to mitigate these harms. We saw the first wave of regulation. Now it almost feels like we’ve taken 10 steps back. How does that make you feel?
MM: Within technology, and with society generally, that for every action, there’s a reaction. The interest in ethics and responsible AI and fairness was something that was pleasantly surprising to me, but also what I saw as part of a pendulum swing. We tried to make as much positive impact as we could while the time was ripe for it.
I started working on this stuff when no one was interested in it and pushing back against it as something that’s a waste of time. Then people got excited about it, thanks to the work of my colleagues, to your reporting. [It became] dinner table conversation. People [learnt] about bias and AI. That’s amazing. I never would have thought that we would have made that much of an impact.
[But] now [regulators and industry] are reacting and not caring about [ethics] again. It’s all just part of the pendulum swing.
The pendulum will swing back again. It probably won’t be another massive swing in the other direction until there are some pretty horrible outcomes, just by the way that society tends to move and regulation tends to move, which tends to be reactive.
So, yes, it’s disappointing, but it’s really important for us to keep the steady drumbeat of what the issues are and making it clear, so that it’s possible for the technology to swing back in a way that [takes into account] the societal effects.
MH: Looking at this current generation of technologies, what kind of harm do you expect? What keeps you up at night?
MM: Well, there’s a lot of things that keep me up at night. Not all of them are even related to technology. We’re in a position now where technology is inviting us to give away a lot of private information that then can be used by malicious actors or by government actors to harm us.
As technology has made it possible for people to be more efficient or have greater reach, be more connected, that’s also come with a loss of privacy, a loss of security around the sort of information that we share. So it’s quite possible that the kinds of information we’re putting out and sharing now could be used against us.
This is everything from being stopped at the border from entering the country because you’ve said something negative about someone online, to non-consensual intimate information or deepfake pornography that comes out that’s commonly used to take revenge on women.
With the growth in technological capabilities has come a growth in personal harm to people. I really worry about how that might become more and more intense over the next few years.
MH: One of the contributing factors to this whole AI boom we’re seeing now is this obsession with AGI. You recently co-wrote a paper detailing how the industry shouldn’t see AGI as a guiding principle, or ‘north star’. Why is that?
MM: The issue is that AGI is essentially a narrative. Just the term “intelligence” is not a term that has consensus on what it means, not in biology, not in psychology, certainly not within schooling, within education. It’s a term that’s long been contested and something that is abstract and nebulous, but throughout time, throughout talking about intelligence, has been used to separate haves and have nots, and has been a segregationist force.
But at a higher level, intelligence as a concept is ill-defined and it’s problematic. Just that aspect of AGI is part of why shooting for it is a bit fraught, because it functions to give an air of positivity, of goodness. It provides us [with] a cover of something good, when, in fact, it’s not actually a concrete thing and instead provides a narrative about moving forward whatever technology we want to move forward, as people in positions of power within the tech industry.
Then we can look at the term “general”. General is also a term that within the context of AI doesn’t have a very clear meaning. So you can think about it just in terms of every day. If you have general knowledge about something, what does that mean? It means you know about maths, science, English.
I might say I have general intelligence but that I don’t know anything about medicine, for example. And so if I were a technology that is going to be used everywhere, I think it’s very clear to know, OK, I can help you edit your essay, but I cannot do a surgery. It’s so important to understand that general doesn’t actually mean good at everything ever that we can possibly think of. It means good at some things, as measured on benchmarks for specific topics, based on specific topics.
AGI as a whole is just a super problematic concept that provides an air of objectivity and positivity, when, in fact, it’s opening the door for technologists to just do whatever they want.
MH: Yes, that’s really interesting, because we’re being sold this promise of this super AI. And in a way, a lot of AI researchers, that’s what motivates their work. Is that something you maybe thought about earlier in your career?
MM: One of the fundamental things that I’ve noticed as someone in the tech industry is that we develop technology because we love to do it. We make post hoc explanations about how this is great for society or whatever it is, but fundamentally, it’s just really fun.
This is something I struggle with, as someone who works on operationalising ethics, but really, what I love best is programming. So many times, I’ll see people developing stuff and be like, oh my god, that’s so fun, I wish I could do that. Also, it’s so ethically problematic, I can’t do it.
I think that part of this thing about how we’re pursuing AGI, for a lot of people, that’s a post hoc explanation of just doing what they love, which is advancing technology. They’re saying, there’s this north star, there’s this thing we’re aiming towards, this is some concrete point in time that we’re going to hit, but it’s not. It’s just, we’re advancing technology, given where we are now, without really deeply thinking about where we’re going.
I do think there are some people who have philosophical or maybe religious type beliefs in AGI as a supreme being. I think, for the most part, people in technology are just having fun advancing technology.
MH: When you think about the ultimate goal of AI, what is it?
MM: I come from a background that’s focused on AAC (assistive and augmentative communication). And that’s from language generation research.
I worked on language generation for years and years and years, since 2005. When I was an undergrad, I was first geeking out over it, and I wrote my thesis there on it. And I’ve continued to look at AI through the lens of, how can this assist and augment people? That can be seen as a stark contrast to, how can it replace people?
I always say, it’s important to supplement, not supplant. That comes from this view that the fundamental goal of technology is to help with human wellbeing. It’s to help humans flourish. The AGI thing, the AGI narrative, sidesteps that, and instead actually puts forward technology in place of people.
For me, AI should be grounded and centred on the person and how to best help the person. But for a lot of people, it’s grounded and centred on the technology.
It is quite possible to get swept up in excitement about stuff and then be like, oh, wait, this is terrible for me. It’s like the computer scientist nerd in me still gets really excited about stuff.
But also, that’s why it’s important to engage with people who are more reflective of civil society and have studied social science, are more familiar with social movements and just the impact of technology on people. Because it is easy to lose sight of [it] if you’re just within the tech industry.
MH: In your paper, you laid out reasons why this AGI narrative can be quite harmful, and the different traps it sets out. What would you say the main harms of this narrative are?
MM: One of them that I’m really concerned about is what we call the illusion of consensus, which is the fact that the term AGI is being used in a way that gives the illusion that there is a general understanding of what that term means and a consensus on what we need to do.
We don’t, and there isn’t a traditional understanding. But again, this creates a constant tunnel vision of moving forward with technology, advancing it based on problematic benchmarks that don’t rigorously test application in the real world, and seems to create the illusion that what we’re doing is the right thing and that we know what we’re doing.
There’s also the supercharging bad science trap, which goes to the fact that we don’t really have the scientific method within AI. In chemistry, physics, other sciences outside of computer science, there has probably been a longer history of developing scientific methods, like how do you do significance testing, and what is the hypothesis?
With computer science, it’s much more engineering-focused and much more exploratory. That’s not to say that’s not science, but that is to say that we haven’t, within computer science, understood that when we have a conclusion from our work, that it isn’t necessarily supported by our work.
There’s a tendency to make pretty sweeping, marvellous claims that aren’t actually supported by what the research does. I think that’s a sign of the immaturity of the field.
I imagine that Galileo and Newton didn’t have the scientific method that would be useful to apply to what they were doing. There was still a certain amount of exploring and then getting to a conclusion and going back and forth. But as the science matured, it became very clear what needs to be done in order to support a conclusion.
We don’t have that in computer science. With so much excitement put into what we’re doing, we have a bias to accept the conclusions, even if they’re not supported by the work. That creates a feedback effect or a perpetuation effect, where based on an incorrect conclusion, more science builds on top of it.
MH: Are you seeing any of that in the field now? For example, are large language models (LLMs) the right way to go? Could we be pouring all this money into the wrong thing?
MM: With everything in AI ethics, and to some extent responsible AI, [you need to ask] is it right or wrong for what? What’s the context? Everything is contextual. For language models, they can be useful for language generation in specific types of contexts. I did three theses, undergraduate, masters and PhD, all of them on natural language generation, and a lot of the work was looking towards, how can we assist people who are non-verbal?
People who have non-verbal autism [or] cerebral palsy, is there a way to generate language so that they’re in control? With cerebral palsy, using a button that they can mark with their head to select among a ton of different generated utterances to reflect what they want to say in order to say how their day was.
Or people who are blind, can we create systems that generate language in a way that speaks to what they need to know for what they’re trying to do, like navigating a busy room or knowing how people are reacting to what they’re saying?
Language models can be useful. The current paradigm of large language models is generally not grounded, which means it’s not connected to a concrete reality.
It’s stochastic, it’s probabilistic based on the prompt from the user. I think that utility [for language models] would be even stronger if grounding were a fundamental part of LLM development.
I don’t think that LLMs get us to something that could replace humans for tasks that require a lot of human expertise. I think that we make a mistake when we do that.
MH: What are the real-world implications of blindly buying into this AI narrative?
MM: I’m really concerned about just this growing rift between people who are making money and people who are not making money and people who are losing their jobs. It just seems like a lot of people are losing some of the things that were able to help them make a living.
Their writing, their images, the kinds of things that they’ve created, that’s getting swept up in the advancement of AI technology. People who are developing AI are becoming richer and richer, and the people who have provided the information that AI can be using in order to advance are losing their jobs and losing their income.
It really seems to me that there’s a massive risk, and it’s actually already happening with AI, of creating a huge rift in people in positions of power with money and people who are disempowered and struggling to survive. That gap seems to just be widening more intensely all the time.
MH: So is AGI basically just a scam, or . . .?
MM: Some might say it’s snake oil. Two of my colleagues in this space, Arvind Narayanan and Sayash Kapoor, have put out this book, AI Snake Oil. Although I disagree with some of their conclusions, the basic premise that the public is being sold something that isn’t actually real and can’t actually meet the needs that they’re being told that it can meet, that does seem to be happening, and that is a problem, yes.
This is why we need a more rigorous evaluation approach, better ideas about benchmarking, what it means to know how a system will work, how well it’ll work, in what context, that sort of theme. But as for now, it’s just like vibes, vibes and snake oil, which can get you so far. The placebo effect works relatively well.
MH: Instead of obsessing over AGI, what should the AI sector do instead? How can we create systems that are actually useful and beneficial to all?
MM: The most important thing is to centre the people instead of the technology. So instead of technology first, and then figuring out how it might be applied to people, people first, and then figuring out what technology might be useful for them.
That is a fundamental difference in how technology is being approached. But if we want something like human flourishing or human wellbeing, then we need to centre people from the start.
MH: We’ve talked a lot about the potential risks and harms, but what’s exciting you in AI right now?
MM: I’m cautiously excited about the possibilities with AI agents. I suck at filling out forms. I’m terrible at doing my taxes. I think it might be possible to have AI agents that could do my taxes for me correctly.
I don’t think we’re there yet, because of the grounding problem, because the way technology has been built hasn’t been done in a way that’s grounded. There’s a constant risk of error and hallucination. But I think it might be possible to get to a place where AI agents are grounded enough to provide reasonable information in filling out complex forms.
MH: That is a technology that everyone, I think, could use. I could use that. Bring it on.
MM: I’m not after the singularity. I am after things that will help me do things that I completely suck at.
This transcript has been edited for brevity and clarity