AightBits

There’s a lot of excitement around the recent paper, “Large Language Models Pass the Turing Test (Jones, C. R. & Bergen, B. K.), and there should be. But what does passing the Turing Test really mean?

AGI (Artificial General Intelligence) and ASI (Artificial Super Intelligence) are the latest buzz (and arguably the next wave of the AI hype cycle). But what is it? Are we close? A recent paper from UC San Diego—“Large Language Models Pass the Turing Test”—might lead some people to believe so. It’s gotten a lot of attention for claiming that GPT-4.5, when persona-prompted, effectively passed a more rigorous version of the test, modeled on Turing’s original three-party setup.

That’s a big claim, and an interesting result. But it’s not the first AI system to “pass” a Turing Test—just the first to do so under this particular experimental design, with statistically significant results. It’s also important to understand that this was a short-form interaction (five-minute text chats), not an open-ended interview. And more than anything, it tested how well the model could imitate a human socially—not whether it actually understands anything.

There are a lot of misunderstandings around the Turing Test as described in Alan Turing’s 1950 landmark paper. This test does not evaluate a machine’s ability to truly think or reason, or determine if a machine is self-aware or conscious. Rather, it simply evaluates whether a machine can exhibit behavior indistinguishable from a human in a text-only conversation. It’s a practical benchmark for humanlike mimicry, not an actual test of understanding or cognition.

Deceiving humans through machine imitation is also not new: in 1966, ELIZA fooled many despite its rudimentary design. As people became more familiar with computers and more critical, the test became harder in practice. PARRY (1972) and Eugene Goostman (2014) also “passed” by mimicking humans with specific limitations (e.g., mental illness, or a non-native child). These weren’t AI models like we see today but rule-based programs written specifically to exploit conversational expectations.

The Turing Test is also a moving target. There’s no single definition or strict criterion. ChatGPT and other LLMs fooled many people early on. But now, even the most advanced models can often be identified by those who are observant and experienced. That said, the UCSD study—using a stricter, three-party setup closer to Turing’s original formulation—showed that GPT-4.5, when prompted to adopt a specific humanlike persona, was judged to be human more often than the actual human participants. This doesn’t mean it’s intelligent in a general or conscious sense, only that it successfully mimicked human conversation well enough to fool people in a short-form, text-message-style exchange.

It’s important to note this nuance: passing the Turing Test often relies more on human perception—projection, assumptions, and bias—than on the underlying cognitive abilities of the model. Or as the recent paper put it: “The Turing test is… a measure of substitutability: whether a system can stand in for a real person without an interlocutor noticing.”

So now we understand that passing the Turing Test doesn’t imply real intelligence, emotion, self-awareness, or consciousness. But what does? That’s a more complex question.

References

• Alan Turing. (1950). Computing Machinery and Intelligence. Mind, 59(236), 433–460.
• Weizenbaum, J. (1966). ELIZA – A Computer Program for the Study of Natural Language Communication Between Man and Machine. Communications of the ACM, 9(1), 36–45.
• Colby, K. M., Weber, S., & Hilf, F. D. (1971). Artificial Paranoia: A Computer Simulation of Paranoid Processes. American Journal of Psychiatry, 128(1), 64–69.
• Loebner, H. (2014). Eugene Goostman: Chatbot That “Passed” Turing Test in 2014.
• Jones, C. R., & Bergen, B. K. (2025). Large Language Models Pass the Turing Test. arXiv preprint arXiv:2503.23674.