Outgrowing the Chinese Room: In Defense of Functionalism

When I commanded Claude to write this essay for me, did it understand me? Putting alignment aside, this question evaluates whether Claude is capable of intrinsic understanding. Can any artificial intelligence (AI) intrinsically understand anything? For John Searle, no digital computer, no AI, would be ever able to truly understand. Consequently, no digital computer can have a mind by virtue of computation alone. In Minds, Brains, and Programs (1980), Searle presents his famous ‘Chinese Room’ thought experiment as an objection to ‘Strong AI’, a consequence of the functionalist theory of mind. My paper is an attempt to object to the Chinese Room, in defense of functionalism. However, this article should not be taken as a direct argument endorsing functionalism, but rather as a humble counter argument to a specific counter argument. My thesis is that the Chinese Room Experiment can lead us to two opposite conclusions, depending on how ‘understanding’ is defined: either both (other) humans and AIs can be capable of understanding, or neither of them can. I will demonstrate why the ability to understand is of the utmost importance in this context, as it establishes that the subject analysed has a mental state, and therefore has a mind. Where necessary, I will explain complex terms and simplify concepts.

The first paragraph introduces functionalism, a theory of mind, and frames it toward our debate. The second paragraph presents the Chinese Room Experiment and John Searle’s arguments before identifying its core weakness: the undefined notion of understanding. The third paragraph argues that the ‘systems objection’ dismantles Searle’s argument with the functional definition of understanding. The fourth paragraph contends that the “Other Minds Reply” shows that an intrinsic definition is a slippery slope towards naive dualism. The fifth paragraph draws the implications of this debate and concludes in defence of functionalism.

Functionalism is a philosophical framework that defines concepts according to their functional roles in terms of inputs and outputs, rather than their material consistency (Putnam, 1967). For example, Forward College is not defined as a specific way of arranging pieces of bricks, woods, or human beings. It is rather its causal effect: taking students and making them smarter. From this perspective, the level of analysis is key, as the subject in question, the system, might contain numerous subsystems that each have their own functions. In the philosophy of mind, functionalism defines mental states in terms of their causal roles within a system, including their relations to inputs, outputs, and other mental states. If it is sunny outside (input), you become happy (intermediate state), so you smile (output) (Schwarz & Clore, 1983). Functionalists make the observation that if mental states cannot be identified through their functional effects, they become unverifiable — and unverifiable entities have no place in a rigorous theory of mind (Ayer, 1936). Yet, a problem persists. Each of us 'feels' that there is more than simple physical phenomena, that we have a consciousness. This 'what it is like to be the self' is not easily explained by causal effects. Isn't there a difference between acting like having a mind and truly having a mind?

It is that distinction that John Searle is trying to make in Minds, Brains, and Programs. Although his main objection is not formulated directly against functionalism, it is a Reductio ad absurdum targeting an implication of it which he named ‘Strong AI’. This term describes the idea that digital computers are theoretically capable of having mental states because, following the functionalist theory, they are theoretically capable of reproducing all cognitive outputs of the human brain. To this end, he formulates the Chinese Room thought experiment to prove that digital computers are and will forever remain incapable of understanding. Before going through this experiment, we have to make a distinction between syntax and semantics in written languages. Syntax describes the ‘form’ of the text. For example, the shape, presence, absence or arrangement of letters and spaces. Semantics is concerned with the ‘content’ of the text, in the sense of its meaning, its symbolic significance (Morris, 1938). Let’s take for instance the word ‘cat’. A syntactic knowledge of this word would be to know that it is the arrangement, in order, of the letters c, a and t. On the other hand, a semantic understanding of it would be to know that it refers to an adorable creature that we ought to cherish. Searle argues that a Strong AI would always only be capable of syntactic knowledge. Imagine that we put a person who knows nothing about Chinese in a closed room. Let’s say Donald J. Trump. There are two holes in this room, one from which he receives papers with Chinese characters, and one from which he gives out other papers with Chinese characters. Trump has with him a rule book, a precise program that explains what Chinese characters to give (output) for all possible sets of Chinese characters he receives (input). Assuming that Trump cannot leave this room, will he ever understand Chinese? Even if Trump becomes so good at applying those rules that no Chinese speaker can actually differentiate the message received from those received from a Chinese person, Trump will never be able to learn Chinese in this room. In fact, the only information he has access to is syntactic rather than semantic information. Trump only knows that he should answer ‘squoggle squoggle’ to ‘squiggle squiggle’ but does not understand either of their meanings. Searle’s thought experiment is an analogy of digital computers that simplifies the complex statistical calculations that a Strong AI can do, which are all based on syntactic data. He argues that if we accept that the person in the room can be indistinguishable from a Chinese speaker and yet never truly understand Chinese, then we must recognise that Strong AI, as good as it can be, cannot understand any languages either. However, Searle’s argument is weakened by an absence of a proper definition of understanding. In his text, he explains that it requires an intrinsic intentionality, but that is not properly defined either. Instead, Searle tries to identify it through a ‘clear case’: you understand when I say ‘hamburger’ that I am talking about a hamburger. It is this absence of a definition that gives the coup de grâce to his argument.

The most famous objection to Searle’s thought experiment is the ‘systems objection’. It is based on a functionalist definition of mental states. It posits that, when testing whether the Chinese Room understands Chinese or not, we shouldn’t test the person inside the room but rather the system as a whole (Churchland and Churchland, 1990). A functional definition of the mental state of ‘understanding Chinese’ would be the measurement of its linguistic output according to its linguistic input. For the case of digital computers, the Turing test was formally proposed by Alan Turing (1950) to evaluate the ability of a computing machine to have indistinguishable cognitive capabilities from those of humans. Evaluators would engage the computer in a conversation and evaluate whether it is a human or an AI. Turing has argued that it is mathematically possible to create a computing machine that would pass the test convincingly. When passing the test, it is not the Central Processing Unit (CPU) that is evaluated, but rather the whole system, including its ability to write text, its memory and so on. Consequently, the person in the room and the rule book are merely the CPU of a whole system that understands Chinese. This ‘systems objection’ was directly addressed by Searle. He asks us to imagine that the subject in the room has now internalized all the capabilities of the whole system, that they can now memorize the full rule book and can directly communicate with the Chinese speakers. In that case, they will be the system and not only a subsystem, but would still not understand Chinese, as it knows only syntax and not semantics. Searle’s rebuttal has its limits. Ned Block (2004) has shown that this example incorporates two separate functional systems in the human brain: the non-Chinese-speaking system and the Chinese-speaking system, with both systems having separate functional understanding. In a sense, Searle is pushing the problem back but is not resolving it. Nevertheless, the question of consciousness remains. We still have the intuition that our states of mind have more than just functions, what Searle calls intrinsic intentionality.

This notion of intrinsic intentionality is the core on which the Chinese Room argument is based, and it is also its most vulnerable point. With it, Searle tries to show that mental states have “meaning in themselves”. He argues that minded entities have self-ascribing semantics that do not depend on the observation of others. Intrinsic intentionality is distinguished from derivative intentionality, whose meaning is merely the projection of the understanding of an external observer with intrinsic intentionality. To understand this point, I would like you to picture your favorite painting in your head. Try to imagine it for a few seconds. For me, it would be “It has come to pass by”, by Sergei Lukin (1958). I can ‘see’ the golden throne with its crimson velvet, and ‘visualize’ the revolutionary soldier standing against the hollow opulence of the fallen Tsar. You might also ‘see’ shapes, colours and people. But is this image that you see really there? Is it physically present in your brain? In the materialist sense of the term, it isn’t. But you still ‘see’ it. This is the same phenomenon as Searle’s intrinsic intentionality: not only can you express your understanding, but you also ‘visualize’ it. This redefinition of understanding gives rise to another problem. If the presence of mental states depends on an intrinsic experience not measurable by external evaluators, how can we say that other humans have mental states? As it is not possible to access the inner experience of a third person entity (Thomas Nagel, 1974), we can neither prove that a human nor that a Strong AI has a mind. Again, Searle predicted this "Other Minds Reply” and tried to find an answer to it. He claimed that, as each one of us can individually know that they have mental states, we can infer that similar biological entities have mental states as well. The problem with this argument is that it presupposes that having mental states is enabled by specific ‘biological stuff’ that we share with other people, such as DNA. However, we only have one point (our mind) as a measure, and therefore can neither know the ‘range’ of similarity, nor what is the nature of the ‘thing’ that is required for mental states. I have about 0.05% of genetic difference from my brother and 0.1% from other humans. Cats and I are 80% genetically alike, and this number is in the range of 20-30% for me and trees. How can one justify that conscious experience stops at 0.1% of genetic differences or starts with 80% genetic similarities?
In Searle's terms, it is equally (un)justified to say that only I and other humans have minds as to say that only I and my brother do. Moreover, how can one prove that it is “biological stuff” that enables mental states and not something of a completely different nature, such as the weight of the entities in question? One that weighs 67 kg could claim that all things between 66 kg and 68 kg are conscious and that the rest is not. Inevitably, redefining understanding in terms of intrinsic intentionality leads to no other justification for the claim that other human beings are also conscious than a faith-based naive dualism. Only a framework grounded in verifiable, functional elements can escape this trap.

Funnily enough, this essay can be addressed to all entities fitting the functional definition of mind and claiming to be conscious, whether human, AI, or other. The core of the Chinese Room thought experiment is the analysis of other entities, third persons. In this regard, some have described it as dishonest (Geoffrey Hinton, in Jaimungal, 2025). I disagree, and believe that it is elegant and highly intuitive. However, as intuitive as it can be, it remains philosophically inaccurate. As I have shown in this essay, choosing a functionalist or an intrinsic definition of understanding leads us to mutually exclusive conclusions. It is a constructive dilemma (Constructive dilemma: (P → Q) ∧ (R → S), P ∨ R; therefore, Q ∨ S).
Functionalism is paradoxically strengthened by the Chinese Room, because this thought experiment has proven that only a functionalist definition would permit admitting that other humans have minds. It has shown that admitting it would also require recognising that digital computers with artificial intelligence can also have minds. Just as some argue that animals, as sentient beings, should have rights, philosophy of ethics must determine whether a minded digital computer deserves some form of justice. As Dario Amodei, CEO of Anthropic, has said, "If it quacks like a duck and it walks like a duck, maybe it’s a duck.” (Council on Foreign Relations, 2025).

Read the original submission: