But these were just random reports and anecdotes; no systematic proof of these abilities existed. The weight of that proof fell on the shoulders of a chimpanzee named Nim Chimpsky (a not-so-subtle dig). “Project Nim”—as this chimpanzee’s language-acquisition project eventually became called—was aimed at producing the systematic evidence of simian language use that others had not. And indeed, the project’s researchers thought they were succeeding: Nim seemed to use language in a way that humans did, combining signs in order to share an experience about things with no instrumental goals in sight. Put another way, Nim appeared to be having “water bird” moments like Washoe all the time, and these human-like interactions were recorded as proof.
However, in preparing the results of this work for publication, Herbert Terrace (a psychologist involved with the project) reviewed the video tapes of the experiments and noticed something that no one had before. In those “water bird” interactions, the trainer who was acting as a conversation partner was subtly (and probably unconsciously) signing the desired sign just a split-second before Nim. It turns out Nim wasn’t initiating a signed conversation about an object to share with his trainers; he was imitating signs that his trainers had unconsciously initiated. The reason Nim made those signs was that he had been trained to do it—during his training, according to behaviourist principles, he had learned that imitating human signs correctly would result in a reward. Nim, it turned out, wasn’t using language in a declarative, human way; he was merely imitating it to get rewarded.
It was concluded on this evidence that Koko, Washoe, Nim, and other non-human primates like them were responding to instrumental demands of their conversation partners—not engaging in communication per se. Their imitations made it seem to the conversation partner that they were having a signed conversation about something. But in reality they were merely producing the results of their conditioning. It turns out there were no water birds; there were only bananas.
LLMs are trained to produce language using behaviourist principles—just like the non-human primates who attained an impressive level of nonverbal linguistic ability—in order to imitate human language use in “conversation” with their users. I could simply end here, noting that no matter how impressive these LLMs become, since they acquire language through imitation and reinforcement, they will, in principle, only ever use language in response to invisible reinforcement mechanisms that have been employed in the design or training of the LLMs. What’s more, any appearance of “general intelligence” in LLMs is exactly that—an appearance, one might even call it an illusion—just as the supposed human-like language use among non-human primates was nothing but simian simulacra.
Yet there is a substantial difference between non-human machines and non-human animals. While a chimpanzee typing at random with an infinite amount of time may happen to produce the complete works of Shakespeare, no one is tempted to use a chimpanzee as a means of generating an email to a friend, or the text for a presentation, or the content of an HR memo for employees; but many people have used LLMs to produce those things. How might such widespread use affect us in the long run? To find out, we first need to understand communication’s role in establishing reality and building trust in relationships, and to do that we must return once more to the twentieth century.
In a study first conducted in the 1970s, E. Tory Higgins and William Rholes demonstrated what they called the “saying-is-believing effect.” In this study, participants were given neutral information about a person—in the most recent versions of this study, the person’s name is Michael—and then they were asked to play a game with a communication partner. In this game, they were told they needed to write a short description of Michael for their conversation partner to see whether that partner could correctly identify Michael. They were told that the conversation partner knows Michael—and also told that this person doesn’t particularly like him. (Half were told that the person did like Michael, but since the effects were the same, let’s just focus on the negative for simplicity’s sake.) Participants then wrote a short description of Michael using the information they were given about him, except they tended to shift that information to make it negative, so that their conversation partner would know whom they were talking about.
For example, maybe the participant was told, “Once Michael makes up his mind to do something, it’s as good as done, regardless of how difficult it is. Only rarely does Michael change his mind even when it might well be better if he did.” The material is evaluatively ambiguous—is Michael persistent or stubborn? In this study, participants generally tuned their description to the audience, so if told that their communication partner doesn’t like Michael, the communication emphasized how stubborn he is.
After writing this communication, participants gave it to the experimenters, who claimed to pass it to their partner. After some time passed, the experimenter came back to tell them that their communication partner (who, as the reader is likely suspecting, didn’t really exist) correctly identified Michael as the target of the description. Then the experimenter asked the participants to recall the information they were given about Michael. Intriguingly, they didn’t recall the original, ambiguous information; they recalled instead the negative information they produced in their communication—Michael is stubborn, full stop. Perhaps even more intriguingly, if they were told by the experimenter that the communication partner failed to identify Michael, the reverse happened: they recalled the original, ambiguous behavioural information, not the negative version from their communication. Participants, in other words, recalled the information about Michael that they judged most likely to be true: If their negative communication was confirmed as accurate through recognition by someone who supposedly knew Michael, then it must be the truth. If not, then the best information they had about Michael was the ambiguous information they got at the outset of the experiment.
Or maybe this result was just another behaviourist artifact, where the social verification acted as a reinforcement for the communication, rather than being about truth per se. Maybe the positive or negative feedback is just telling participants which version of a communication is preferred, regardless of where the truth lies (like feedback given to an LLM). One way to test this is to alter why the participants were communicating. If the saying-is-believing effect is just a matter of reinforcement, then the audience’s social verification should reinforce the communication regardless of the motivation behind that communication. If humans are just like LLMs, then positive verification following communication should influence memory regardless of the intent of that communication.
To test this, in a series of experiments in the early twenty-first century, researchers tried versions of the saying-is-believing paradigm where the reason for communicating was instrumentalized. In one version, rather than trying to get the audience to correctly guess that the person they’re talking about is Michael, participants were told that if they wrote a message congruent with the audience’s perception of Michael, they would get a monetary reward. In another version, they were asked to create a version of the communication in which Michael’s traits were wildly and hilariously exaggerated in a manner congruent with the audience’s point of view in order to entertain themselves. In other words, in these alternative versions of the experiment, participants were asked to subordinate their motivation for accuracy to something else. Instead of using language to communicate, they were asked to use it to entertain themselves or to obtain a material reward.
In both cases, the effect failed. Was this because there was inadequate behavioural reinforcement? Quite the opposite. Participants received social verification, just like in the original version, and in these versions they also got money or entertainment to boot. Instead, the reason participants didn’t recall their communication was that they knew it wasn’t true. Working to create language that bears the burden of truth is a unique and indispensable component of human social life.
The results from these experiments led the researchers to propose the theory of shared reality. Building on all of the work surrounding how linguistic communication works in humans, they proposed that we use language intersubjectively in order to understand the way the world actually is: A child might call a swan a “water bird” and then look at her mother for confirmation—not for any instrumental reward like bananas, but because she really wants to know the right word to use to refer to the swan. We are sharing animals, and we most prominently share through language—this is why humans are so positively weird when it comes to acquiring language, why this particular use of it is so unique to us, and, ultimately, why we so desperately need to jealously guard this aspect of our social lives from being replaced by machines.
LLMs, since they are designed and trained using behaviourist principles that rely on reinforcement, have an almost invisible instrumental component to their communication. They are aiming at what they have been trained to produce, and they produce it very well: language compiled in a way that is satisfying to the user. The text is completely empty of any attempt to approximate reality, and it certainly is devoid of any motivation to communicate.
All use of text produced by LLMs will necessarily carry with it this instrumental limitation, because the very act of using LLMs recasts language from its uniquely human purpose of discovering the truth about things and instead subordinates it to the purposes underlying LLMs. The most obvious case is the college student who chooses not to do the class readings and provides a ChatGPT-generated paper summarizing those readings to get a passable grade. The student doesn’t care about the readings or whether the paper reflects his knowledge—in short, the student doesn’t care about how language can describe something true; he only cares about how language can be useful. His words become specious—outwardly pleasing but having no relation to reality.
Now imagine some future iteration of LLMs that writes emails, designs presentations, constructs text messages, perhaps even imitates our voice and makes phone calls on our behalf. Perhaps in each of these instances we really want to communicate with our partner, friend, or audience. But the LLM pops up in the corner as we begin to write and promises to make it “better”—write the email in a manner that is more influential, make the text message more entertaining, improve the presentation to land the promotion—and do so while increasing efficiency, letting us use the time for some other purpose. Instead of communicating—using language to deliver our best approximation of reality as we understand it in order to share it and to learn from our communication partners—our text or presentation or email will now carry with it an instrumental purpose to please or entertain or obtain some material end. As LLMs integrate with more text-generating applications and their use becomes more and more frequent, our textual exchange will become less and less human, and more and more like a non-human animal (or, worse, a non-human machine) shaped by behaviourist principles. After over sixty-five years, Skinner will have the last laugh.