Should We Defer to AI When It’s Smarter than Us?

Or should we value human deliberation even when the results are worse?

Feel like watching or listening instead? Scroll to the bottom to watch.

I’ve been thinking a lot lately about the extent to which we ought to value human deliberation, that is, humans trying to think through/solve problems. On one view, our deliberative capacities are just another tool. We deliberate as a means to an end, to bring about a result we (think we) want, like deliberating about where to go to dinner, whether to have children, and whom to vote for.

On another view, human deliberation is not only of instrumental value but also of intrinsic value. Human deliberation itself is something we ought to care about for its own sake, not just because it brings us desirable results.

I’m going to give you two scenarios that will draw out the tension between these two views. My goal is not only to demonstrate the tension, but to explain how it puts us in ethically difficult waters when we think about how AI may outperform us.

Scenario 1: The Deliberative Doctor and AI

Suppose you have an AI that is very, very good at diagnosing patients. That’s because, among other things, it has all the medical knowledge humans collectively possess. It’s an oncologist, a pediatrician, an endocrinologist, a geneticist, and so on. The set of its medical knowledge is the set of our collective medical knowledge.

It’s very difficult for an individual doctor to determine whether its diagnoses are accurate. (This is documented in The AI Revolution in Healthcare). That’s because the individual doctor has, relative to the wide ranging expertise in medicine, a very narrow expertise. She’s an oncologist but not a geneticist, or a pediatrician but not an ophthalmologist, and so on. Given that, how could the individual doctor possibly know if the diagnosis is accurate? But let us suppose that, prior to release, the AI underwent a tremendous amount of testing and its been found to be better than our best doctors across the board. To be clear, we’re not there yet, but someday we could be.

If I were to take one of my children to this AI and it recommended a particular course of treatment that the human doctor did not understand or agree with and I know the AI is more accurate than the human doctor, I’m very much inclined to go with the AI’s recommendation. The human mind is feeble, after all, and this AI knows more than the doctor, as has been, we’ll suppose, amply demonstrated by the testing it underwent.

In this scenario, I don’t mourn the loss of human doctor deliberation. In healthcare, I just care about the results. I care only about the effectiveness of the tool to bring about the health of my children.

But in the next scenario, things seem different.

Scenario 2: The Deliberative Judge and AI

I want you to consider an AI that determines criminal sentences. The defendant has been found guilty and now it needs to be determined whether they should receive a prison sentence and, if so, for how long.

When human judges do this, they consider a variety of factors, including the severity of the crime and the likelihood that the defendant will engage in that crime again. Sentencing judgments aren’t only backwards looking, or looking at what was done, but also forward looking, or looking at what might be done.

But human judges are bad at sentencing in at least two ways.

First, human judges, just like most people, are very bad at predicting the future. They’re very bad at knowing, of this given individual, the likelihood that he’ll recommit a crime. There are just too many variables and too many unknowns.

Second, human judges can be biased in a great variety of ways. There is the obvious racial bias that can be manifested, and then there are cognitive deficiencies (ranging from general stupidity to common mistakes in reasoning to temporary deficiency due to lack of sleep, and more) as well as various emotions that can cloud their judgment (e.g. suffering from depression, distracted because the judge is going through an acrimonious divorce, resentful of the defendant because the judge has been victim to a similar crime and now has an axe to grind, etc.).

Now I want you to consider an AI that has access to a range of data that makes it better than humans at predicting likelihood of crime. We’re not there yet today, but one day we could be. Further, since AI has no emotions and doesn’t get tired and has no axe to grind, it suffers from none of the bias-inducing causes that human judges do. Of course, cases of biased AI are well documented, but let’s suppose we can get the AI to be less biased than humans which, frankly, is a fairly low bar. The result is that this AI is just better than human judges at sentencing, both in terms of accuracy and bias.

A Lesson to Draw From the Two Scenarios

One thing we could do with this AI is completely replace judges in the sentencing process the way I suggested we might replace the doctor in the previous scenario.  The result would be fewer people getting inappropriate sentences, and this means, among other things, a less racially discriminatory criminal justice system. The impacts would be significant.

But I’m much more hesitant to replace human judges than I am to replace human doctors. The reason is that I think there’s something valuable about the deliberations of the human judge. That deliberation is not only a means to an end, but also intrinsically valuable. What do I think is so valuable about it? I think it’s valuable because that deliberation is part of what it is to treat someone with respect in this context. It’s taking them seriously. The deliberation is a way of according that person’s life a dignity or a weight. It could be contrasted with the judge flipping a coin for sentencing purposes. That flip is, sorry for this, too flippant. It would express too little regard for the humanity of the criminal.

In short, I think that we ought to value the deliberation of a human judge more than the deliberations of a doctor because the latter is purely instrumental while the former is both instrumental and itself instantiates the value of taking other people’s lives and their worth seriously.

Objections to the Lesson

But there are two objections to consider to my resistance to replacing human judges and I think the objections may win the day. (And to be clear, there are other reasons to reject replacing human judges with AI but the ones I have in mind don’t speak to the issue I’m driving at).

The first objection is that I’m romanticizing human judges’ deliberations. Sure, one might say, there are cases of good, impartial, careful judges, but most of them are so riddled with flaws, both cognitive and moral in nature, that their deliberations are too flawed to value in the way I suppose. Their deliberation looks great in the way that the exterior of a gutted brownstone in Brooklyn is beautiful.

The second objection grants that the human judge’s deliberations are valuable but denies that they are valuable enough to stop us from replacing judges. That’s because, very roughly, the value of having a more just criminal justice system and the value of fewer people wronged by inappropriate sentences outweighs the value of having human judges. So sure, this objection says, something valuable is lost when we replace human judges in sentencing, but a lot more is gained by it.

Well, We’ll Always Have Human Deliberation about When AI Deliberation is Appropriate. I think.

As I said, I think these objections probably win the day, though I’m not sure. (I think we need to get into a lot more detail and investigation to conclude that it would be ethically permissible let alone ethically required to replace human judges in sentencing proceedings). I don’t know whether they will always win the day, though. I suspect there are other cases in which the value of human deliberation, or to put it more generally, the value of the human processes by which an outcome is produced, is of great enough value that we shouldn’t replace it with an AI that could produce better results. I might think this about the creative process in fine art, for instance.

What I hope these examples and my thoughts on them bring out is first, that human deliberation is not always merely a means to an end but rather sometimes valuable in itself, and second, that despite its value it may nonetheless be justifiably overridden in a variety of cases that highly impact individual lives and society at large. When the value of our process should win the day and when the outcome is all that matters will have to be determined, I think, on a case-by-case basis. And given the vast array of contexts in which AI may be embedded, that’s a lot of thinking we need to do.

If you like my Substack, you’ll probably like my podcast, Ethical Machines.