We’ve been considering the question whether various creatures other than ourselves can think, feel, and enjoy other mental states, or have mental lives that approximate our own in at least some ways. At different points, we’ve talked about:
In each of these cases, our aim is not to find a way to prove with certainty that they have mental states. It’s unlikely we can even prove with certainty that other adult humans exist, much less that they have the mental states we think they do. Our goal is the more modest one of determining whether there are at least reasonable grounds for thinking that different creatures on this list have mental states.
Also, it’s one thing to figure out what we do have good reasons to think, and another thing to figure out how to answer philosophical challenges from skeptics about how it could be possible to have such reasons. If this were an epistemology class, we would spend time on the second issue, but it’s not. I take it we agree we do have good reasons to think other adult humans have thoughts and feelings, even if we haven’t yet figured out what’s the best philosophical response to give to skeptics who challenge those reasons.
Our aim here is also not to settle whether creatures on the list have exactly all of the same kinds of mental states that we do. Perhaps we don’t have all the same kinds of mental states as each other: maybe I see colors and hear music and taste food somewhat differently than you do. If dolphins and octopuses have mentality, it wouldn’t be surprising if some kinds of experiences or feelings they have are completely alien to human experience, and vice versa. This point is especially important to keep in mind when we think about the mentality of extraterrestrials and machines/AIs.
It may be an interesting question whether there are fundamental differences between human thought and feeling, on the one hand, and whatever mental states machines/AIs will ever be capable of. But that’s not what we want to focus on. Our primary question is rather: Will machines/AIs ever be able to genuinely think or feel at all? Will they ever really be able to have genuine perspectives, opinions, preferences, self-consciousness, a real mental life comparable to our own — even if it may also be different in some ways from our own?
For any disanalogy you think there is between humans and machines — whenever you’re tempted to say “Machines will never be able to X” — ask yourself:
Considerations that seem relevant to whether other creatures have mentality divide into facts about their behavior and facts about their physical makeup. As we move further down our list of candidate creatures, the physical makeup diverges more and more from our own. It’s not clear how important a role this should have in our thinking. Some aspects of our own physical makeup — our eye color, whether we’re right- or left-handed, how well we can walk, what we look like — we take to have little to do with what kinds of thoughts and feelings we have. (Although sometimes it took us unconscionably long to acknowledge it.) What persuaded us that these aspects of our makeup were less relevant to what thoughts and feelings we’re capable of having, than how healthy our brains are? Presumably it was facts about behavior that led us to think that. People with different eye colors, or differently functioning legs, can still behave in ways that make it seem they’re planning, reasoning, reflecting on their decisions and capacities, avoiding some stimuli and seeking out others. So presumably these kinds of behavioral considerations should take the lead when we’re trying to figure out how good our reasons are to attribute mentality to other kinds of creatures — even if their physical makeup is very different from our own.
One notable kind of behavior adult humans engage in is using language. Even if we don’t always understand someone else’s language, we can often tell that they’re using one, and this plays an important role in our willingness to count them as thinking and reasoning in ways akin to ourselves. With animals, this is the kind of intelligent behavior most differing from our own. Chimps, dolphins, some birds and other animals have some kinds of language-like communication. But there are theoretically important differences between what they do and what humans do. With some kinds of extraterrestrials and with machines/AIs there may be fewer differences on this score. (With machines/AIs, though, some will argue that they can consume and produce language but never really understand it.)
On the face of it, the issues we’re considering now are independent of what stand you take on the materialism/dualism debate. Whether you think mentality is a matter of what’s happening physically or what’s happening in a soul, it seems open to you to take any stance about which creatures on our list have mentality and which don’t. If you’re a dualist, then the question whether crows have mentality is a question of whether crows have souls. So too with extraterrestrials and machines/AIs. I don’t know how to figure out whether a crow has a soul, but I don’t know how to figure out whether you do, either. If you can have a soul, then for all I know, perhaps a crow can too, or a machine/AI. When we make human babies in the usual way, on the dualist picture somehow they end up with souls. Perhaps if we make human babies by genetic engineering, they’ll end up with souls too. Perhaps when we program AIs they’ll end up with souls too. I don’t know how it works. I don’t think the dualists do either.
Some interesting issues come up in the Leiber dialogue that aren’t center-stage for our own discussions. One of these is his list of three questions: (a) do the chimp and/or AI have thoughts and feelings; (b) if so, are their thoughts/feelings enough to make them persons — that is, creatures with certain kinds of rights, protections, responsibilities to others; and (c) if they are persons, do they have a right to not have the space station shut down? Our own discussions are focused only on the first of these questions.
Another idea that comes up in that dialogue is the Buddhist idea that nothing is a person, that the division between a self and the rest of reality is an illusion or a matter of perspective. This is an interesting view, that some academic philosophical discussions do engage with seriously. But we won’t be exploring it in our course.
The rest of these notes will focus on the question whether we can ever have good reasons to attribute mentality to machines/AIs.
In the 1950 article Computing Machinery and Intelligence, the philosopher Alan Turing puts forward a test for determining whether machines are genuinely thinking. The test works like this. A human judge carries on remote conversations with a machine and another human, and has to guess which is the machine and which the human. If the machine is often able to fool the human judge into thinking it’s human, then that machine passes the test, and Turing claims we should regard it as genuinely thinking, and having genuine intelligence. Turing calls his test the Imitation Game, but it has come to be known as the Turing Test for machine intelligence.
As Leiber points out in his notes (pp. 72–3), there is some precedent for what Turing proposes in Descartes’s writings from the 1630s. Unlike Turing, though, Descartes was confident that machines would never be able to perform well at this kind of test.
Note that Turing is only claiming that passing his test is enough for being intelligent, or reasonably being counted as intelligent. Turing’s test may be very hard; it may set the bar too high. Perhaps chimps are intelligent, though they can’t pass the test. Perhaps someday there really will be intelligent machines, that also aren’t intelligent enough (or in the right ways) to pass Turing’s Test. Turing acknowledges this; he doesn’t want to say that being able to pass his test is a necessary condition for being intelligent. He’s only saying that the machines which are able to pass his test are intelligent, or should reasonably be so counted.
We discussed in class whether Turing thought, or we should think, that passing the test logically suffices for being intelligent, or whether we should only take it to be a good but defeasible reason for counting a creature as intelligent. And if the latter, what kinds of further evidence would justify changing our mind? We’ll discuss these questions more below.
Turing doesn’t say very much about who’s supposed to be judging these tests. But that’s important, because it’s very easy to fool computer neophytes into thinking that some program is really intelligent, even if the program is in fact totally stupid. One early computer program called ELIZA pretended to be a psychotherapist holding “conversations” with its patients. ELIZA was a very simple program. (You can nowadays get a version of ELIZA even for bargain-level smartphones.) Nobody who understands the program is at all tempted to call it intelligent. What the program does is this. It searches the user’s input for keywords like the word “father.” If it finds a keyword, it issues back some canned response, like “Do you often think about your father?” Here are links to more background and discussion about this program.
As I said, no one who really understands ELIZA wants to claim that this program is intelligent. If we’re ever going to construct a real artificial intelligence, it will take a much more sophisticated approach than was used to make ELIZA.
But when Turing Tests have been set up at public computing exhibitions, and the judges were just people taken off the street, people who sometimes weren’t very familiar with computer programs of this sort, then chatbots using programs with the same underlying structure as ELIZA did sometimes turn out to be able to fool those judges half of the time. (See the links above.)
Hence, if you want to say that passing the Turing Test really is a good test for intelligence, than it’s going to make a difference who’s judging the Turing Test. We should use better judges than just ordinary people off the street, or they should get coaching about what kinds of questions to ask or answers to look for, or they should get lots of time to interact with the contestants.
When I was young I loved Spider-Man comics. Before he gained his superpowers, Peter Parker was a stereotypical weak clumsy nerdy kid. The school jock Flash Thompson bullied him around. After Peter became Spider-Man, he was in fact no longer so weak or clumsy. But he went to great efforts to keep up that appearance, so that no one would figure out his secret identity. Thus when Flash bullied him, he went through the motions, pretended to be hurt. But what his schoolmates were seeing was misleading. The illusion was a fragile house of cards; it was liable to fall apart any moment, and in the comics it sometimes did.
A second case to consider. Suppose you end up somehow turning your mother’s delicate crystal vase into something that looks the same but is in fact nearly indestructible. For some reason you want to keep this a secret. You go through a complex dance trying to give everybody else the impression that the vase is still delicate or fragile. But really it’s not. Its real disposition is to be nearly indestructible. But with effort you might get people not to see that. You might manage to make it still seem fragile, at least for a while.
One thought that comes up with the Turing Test is that even if machines/AIs turn out to do really well at the test, maybe their successes would be unstable in these same kinds of ways. Maybe failure would be just around the corner, just as soon as someone thought up the right question to ask.
I mention this thought just to acknowledge it and be able to refer back to it. I don’t have anything useful to say to address it. Peter/Spider-Man’s schoolmates might have good reasons to think he’s still weak and clumsy, even though he’s not. Your mother might have good reasons to think her vase is still fragile, even though it’s not. We might have good reason to think a machine/AI’s performance on the Turing Test will continue to impress, even though it’s going to break down after the next question. We can’t rule these possibilities out.
Let’s suppose for the sake of argument, though, that this isn’t what’s going to happen. Let’s suppose some machine/AI has so far acted quite flexibly and apparently intelligently, and that its ability to do so is robustly reliable. It’s no more likely to break down after the next question than adult humans are. What should we think in that case? Should we agree with Turing that this would be a good reason — or perhaps even a logically conclusive reason — to count the machine as having real intelligence, thoughts, preferences, and so on?
So some machine/AI turns out to pass Turing’s Test, even when the test is administered by sophisticated, trained and knowledgeable judges. And we suppose it can do this in a way that’s robustly reliable. There’s no trick question we just haven’t figured out yet that’s going to make it break down or go into an infinite loop.
Some theorists think that even if that happens, we still wouldn’t have good reasons to attribute mentality to the machine/AI. I’ll call this the anti-machine camp. The opposing, pro-machine camp thinks we would.
Some people are so pro-machine, they’d say that some currently existing machines/AIs like ChatGPT already have mentality. But let’s keep our focus on imagined future machines/AIs, who are able to do much better on the Turing Test, and be much more reliable and flexible, than anything currently out there.
We’ll talk more about the anti-machine view below. For the moment, let’s sort out different ways of holding the pro-machine view.
One strong form of this view is called behaviorism (or sometimes “logical” or “analytical behaviorism,” to distinguish it from some related movements in scientific psychology). This view says that if a machine can reliably behave intelligently and flexibly in the ways we’re imagining, that’s all there is to really having thoughts, intelligence, feelings, and other mental states. There’s never any more than that going on, even in the case of adult humans. It’d be impossible, it’d make no sense, for machines/AIs to behave as we’re imagining but still lack some extra mentality that humans really have. There is no extra ingredient or phenomenon to be lacking.
A more moderate view would agree with the behaviorist about some mental states, like thinking, reasoning, planning. But for other mental states like pains, feelings, emotions, this view would say there is a gap between how the creature acts and what’s really going on inside its mind. For example, it’d at least be possible for a machine/AI to act angry but not really feel anger. So for such mental states, the machine/AI’s behavior doesn’t constitute or guarantee that the mental states are really there. It may or may not provide reasonable grounds for thinking the mental states are there. That we’d still have to figure out.
Going even more moderate, we could take this stance even about mental states like thinking, reasoning, and planning. In all of these cases, we could say, the machine’s behavior may provide reasonable grounds for thinking it has these mental states. But it doesn’t guarantee it. This is the perspective of most contemporary pro-machine theorists (and may also have been Turing’s own view).
What more would it take for a machine/AI to really have these mental states?
On these views, it turns on the structure of the machine’s internal algorithms or programming. One way a machine might manage to pass the Turing Test is by having a giant lookup table of all the possible inputs and what output to give for each response. Like an ELIZA program on steroids.
If the lookup table were large enough, perhaps such a machine/AI would be able to fool us reliably. We might not be able to find any trick question that would expose its limitations. Even if so, this camp of pro-machine theorists wouldn’t want to count such a machine/AI as really thinking. The program it’s using is too simple and direct.
On the other hand, if the machine’s programming analyzes the meanings of the questions we put to it, and does something like the kind of processing on those meanings that our human brains do, then these theorists would want to say the machine is thinking.
So some kinds of programs count as thinking, and others don’t, and in principle a machine’s external behavior — what outputs it gives to different inputs — might not guarantee it has the one programming rather than the other. This is why I hedged Turing’s proposal earlier, and said if a machine passes the test, that might just make it reasonable to count it as being intelligent. Whether it really is intelligent could turn on issues like what the internal algorithms are, that the Turing Test by itself might not give us access to.
I’ll assume we’re dealing with a pro-machine theorist of this third sort.
One way to get a machine/AI with the right kind of programming might be to build it to run the same kind of “program” as human brains run. Our hardware would be different, but the machine might for all that be processing information in the same abstract ways our brains do.
In the Lycan selection we read earlier, we heard about Henrietta, who has her neurons replaced one-by-one with synthetic digital substitutes. Eventually her brain has no more organic parts left. If the substitutes do the same causal work that the neurons they’re replacing did, then from the outside, we won’t see any difference in Henrietta. She’ll keep walking and talking, processing information and making plans, the same as she always did. Lycan argues that Henrietta herself wouldn’t notice any difference either. When she has just one neuron replaced, none of its neighboring neurons “notice” any difference. And over the process of gradually replacing all her neurons, there doesn’t seem to be any point at which she’d lose her ability to think or feel. Her new brains would keep working the same way they always have.
So why should the difference in what they’re physically made of matter? Shouldn’t any hardware that runs the same “program” as her original brain have the same mental life as the original?
This perspective is taken up in many places in fiction and film — such as the jewel computer in Egan’s story I posted this as optional reading. In more limited ways, the characters in The Matrix…
and Doctorow’s story more optional reading get to acquire certain abilities or memories, or have certain experiences, by loading new “programs” into their brains. All of this speaks to the intuitive force of the idea that our mental lives are driven by what “programs” our brains are running.
The anti-machine theorists think that machines/AIs will never have real thoughts or mental states of their own. They can at best simulate thought and intelligence. All that passing the Turing Test would show is that a machine is a good simulation of a real thinker.
This is the position of the opposing attorney in the Leiber dialogue. He admits that a machine might be “creative” in some sense, such as when it discovers new solutions to math problems, but he argues that the machine never really understands what its doing. Whereas when humans work on problems, they genuinely have insights, and realize what’s going on. Humans genuinely experience their thoughts, the meanings of their sentences, and what’s happening in their environment.
Near the end of the dialogue, the machine/AI they’re arguing about comes on stage itself, and responds to this attorney that it seems to it (the AI) that it also has inner experiences. It asks the attorney, what makes him so sure that other adult humans really have genine thoughts and other mental states. Presumably the most important reasons for thinking so is how they talk and behave. And doesn’t the machine/AI also behave in the same flexible and apparently intelligent ways?
If the attorney thinks he has better reasons for thinking that other humans have real mentality, what are those reasons?
One difference that anti-machine theorists often allege between humans and machines is that the latter can’t make mistakes. (Hofstadter discusses this in his dialogue when they talk about predicting the weather.)
Turing spends some time discussing this allegation in his article. He introduces an important distinction, between what he calls errors of functioning and errors of conclusion (p. 469). Examples of errors of functioning would be mechanical or electrical faults that prevent a machine from doing what it’s designed to do. We can also include “bugs” in the machine’s programming as errors of functioning. These prevent the machine from working as it’s intended to work. Errors of conclusion, on the other hand, would be mistakes like saying “19” in response to the question “What is 8+9?” Now it is true that humans make many errors of this second sort; but Turing points out that there’s no reason why machines shouldn’t also make errors of this second sort. (ChatGPT definitely makes some math and logic mistakes, though I don’t know if it would ever get this kind of calculation wrong.) Whether a machine will make certain errors of conclusion really depends on the nature of its programming. If we program the machine to add in the ways calculators do, and the machine executes its program perfectly, then it will always give the right answer to addition problems. But if we instead program the machine to do math in the ways that humans actually reason mathematically, then it might very well answer some addition problems incorrectly.
You might protest: But won’t some low-level part of the machine still need to be adding and multiplying correctly, in order for the machine to run any program? Yes, but it’s equally true that your low-level neurons need to add and process electrochemical signals properly, for you to be doing any thinking. That doesn’t make you a perfect adder. You don’t know what your neurons are doing. That neural activity might constitute your making an arithmetic mistake. Why can’t the same be true for the machine?
People often assume that if we ever succeed in constructing a real artificial intelligence, it will be much more “rational” and “logical” and “unemotional” than human beings are. I don’t see why that’s so. Why couldn’t the AI be running programming that makes it much less logical, and much more emotional, than human beings?
What tends to happen is — unless we’re thinking about machines in a steampunk story, or something like that — we think of the machines running smoothly and perfectly in the sense of not breaking down, suffering no errors of functioning. So we naturally assume that the machines won’t make any errors of conclusion either. We naturally assume that they will always “do the rational thing.” But that doesn’t really follow. Whether a machine makes mistakes, and whether it “acts rational” or “acts emotional,” will depend on the nature of its programming…
Another objection that Turing discusses in his article has to do with the thought that machines only have fixed patterns of behavior; they can only do what we program them to do.
In a sense this might be true. What the machine does depends on what its program tells it to do. But that doesn’t mean that the machine’s behavior will always be fixed and rigid, in the way the ELIZA program and other simple chatbots’ responses seem fixed and rigid.
Here are some lines of thought pushing back against that inference.
Computers can also be programmed to revise their own programs, to “learn from their mistakes.” For instance, a chess-playing computer can be programmed to thoroughly analyze its opponents’ tactics or strategy whenever it loses a game, and incorporate those winning techniques into its own databases and decision trees. In this way, it can learn how to use those techniques in its own games, and learn how to look out for them in its opponents’ games. This kind of machine would get better and better every time it played chess. It would quickly become able to do things its programmers never anticipated (though of course they’re the ones who programmed it to learn from its mistakes in this way).
Similarly, the “Digients” in Ted Chiang’s story more optional reading have to go through a loving process of being reared and educated, like children. They don’t have their behavior explicitly scripted ahead of time. But then the Digients eventually grow up and make decisions their parents didn’t expect, and aren’t entirely comfortable with.
If it’s possible for machines to learn and evolve in these ways their programmers can’t predict, why should we think that machines have to be “fixed” and “rigid” in a sense that prevents them from being intelligent?
We’ll talk more about machines having all their behavior explicitly scripted ahead of time below, where we look at a difficult passage in which Turing talks about rules that prescribe and guide all your conduct.
Maybe when we encounter “alien lifeforms,” say on Europa, we’ll start out thinking they’re naturally evolved, but then later will discover they were engineered and first constructed milennia ago by some creators now long extinct. But now these artificial “lifeforms” we discovered will have for uncounted years been reprogramming and redesigning themselves.
As mentioned in class, we can also think about “machines” that never had any programmer, even ones who set up their initial framework. The program might just come together through random happenstance. And what if such a machine did turn out to be internally the same as some AI in an Indonsesian Robotics Lab? Would we want to say that the machine that was explicitly programmed in the lab can’t have its own mental life or make its own choices, but the machine that randomly fell together does? Even though the difference between them is just in their histories, not in how they presently work?
One response to that machines-with-no-programmer idea is that regardless of whether there was a person who wrote the program, still this machine would be running a program. It’d still be predetermined how it will respond to every situation it encounters (that doesn’t break it). Even if no programmer or anyone else knows in advance what that response will be, still it’s already settled.
Well, so what? Should that show that a machine can’t have its own opinions, ambitions, feelings? Should it show that it can’t make its own choices? Maybe all the same will turn out to be true for our responses, too. Maybe our behavior is dictated by our genetic and neurophysiological makeup, and the experiences we’ve had in our lives. That doesn’t show that we don’t have our own mental lives. To us, it still feels like we’re free. So why should it threaten to show that machines don’t have their own mentality?
We could get into a discussion of whether machines must have less free will than humans do. It will depend on what views you take about determinism, compatibilism, and so on. These are issues we will discuss in upcoming weeks. For now, I’m just trying to make the point that, even if it would turn out to be true that machines must lack free will, it’s not obvious that should imply they can’t have thoughts and feelings.
One passage in Turing’s article that will be hard to follow reads like this:
It is not possible to produce a set of rules purporting to describe what a man should do in every conceivable set of circumstances… To attempt to provide rules of conduct to cover every eventuality, even those arising from traffic lights [that confusingly show red and green at the same time], appears to be impossible. With all this I agree.
From this it is argued that we cannot be machines. I shall try to reproduce the argument, but I fear I shall hardly do it justice. It seems to run something like this, “If each man had a definite set of rules of conduct by which he regulated his life he would be no better than a machine. But there are no such rules, so men cannot be machines.” The undistributed middle is glaring. (p. 471)
What the heck does that last sentence mean? I can’t expect you to know. I hope when you come across passages like this you will at least be able to work out from context what the author must in general be getting at. I hope it was clear that Turing doesn’t approve of the argument he’s reporting here, and that the passages that come next in his article—where he distinguishes between “rules of conduct” and “laws of behavior”—are meant to be part of a reply to the argument. Some of you may have been industrious enough to google the term “undistributed middle” to try to figure out more specifically what Turing was saying. (If so, great. That disposition will serve you well.)
What you will find is that this is a term from an older logical system. We don’t use the expression so much anymore—in fact I myself needed to look up specifically which fallacy this is. An example of the fallacy of undistributed middle would be the argument “All newts are gross. Harry is gross. So Harry is a newt.” I hope that even without the benefit of any formal training in logic, you’ll be able to see that this is not a good form of argument. (There can be instances of this form whose premises and conclusion are all true, but that doesn’t make this a good form of argument.)
Now I have to scratch my head and speculate a bit to figure out why Turing thought the argument he was discussing displayed this form. He’s grossly exaggerating to say that the presence of this fallacy in the argument he describes is “glaring.”
Here’s my best guess at what Turing is thinking. We begin with the claim:
As we discussed earlier, claims of the form “If R, then M” are always equivalent to “contrapositive” claims of the form “If not-M, then not-R.” (Compare: if Fluffy is a rabbit, then Fluffy is mortal. Equivalent to: if Fluffy is immortal, then Fluffy is not a rabbit.) So 1 is equivalent to:
Note that neither 1 nor 2 establishes that all machines are rule-followers of the sort described. Turing’s opponent and you may think this is also true; but Turing will go on to argue against it. For the moment, just notice that premises 1 and 2 don’t by themselves imply that.
Now Turing is imagining that his opponents continue their argument like this:
Men are not rule-followers of this sort. (…there are no such rules
)
Therefore, men are not (or: they are “better than”) machines.
This argument from 2 and 3 to 4 does display the fallacy of undistributed middle that we described above. Turing’s text doesn’t make this as clear as it might have, though, since it has the beginning premise in form 1 rather than the (equivalent) form 2.
But what fundamentally is Turing thinking his opponents get wrong?
He’s imagining that even if some machines may have definite rules that explicitly script their conduct in every situation they encounter, others may not. The point of the passages that come next in his article are to distinguish between the idea of having such complete and explicit “rules of conduct” and there being low-level “laws of behavior” that settle in advance how the machine (or the human being) will respond to any given stimulus. Turing would agree that there are low-level laws of behavior strictly determining what the machine will do, but there may be such laws for us too. He’d agree that humans don’t choose what to do from complete and explicit rules/scripts telling us how to respond to every situation, but he’d say machines won’t necessarily have that either. Machines and we might both have to figure out what to do, rather than follow some high-level recipe already explicitly scripted out in advance.
I think I understand the distinction Turing is making, but I’m not entirely sure that I do. How about you? Can you make sense of the idea that there may be some low-level laws of behavior (say your genes, and everything that’s happened to you up until this point in your life) that determine how you will act, even though you don’t have rules or a script you consult to guide every choice you make? What more would you say to better explain this distinction? Can you make sense of the idea that some machine might also lack such high-level complete and explicit rules/scripts?
There’s a lot here for us to wrestle with. Hopefully though this will help you better track how the words Turing actually wrote here are supposed to fit into his larger argument.
