Reconsidering the Turing test for AI: “Show Me the Money” Version

This article is an extended summary of the material My new Turing test would see if AI can make $1 million written by Mustafa Suleyman for technologyreview.com.

2023 has marked the start of the AI era, or at least it seems so if we analyze the basic trends of the year. Marketing, social media, celebrities’ discours: there is no news if there is no AI, and the technology is becoming wide-spread and commercialized. If so, we have to ask ourselves, whether we are using it for our own good - or vice versa. Is it still a tool or a partner? There is a new insight on a Turing test for AI, and you will be surprised with the range of options to make it work.

What is a Turing test

Alan Turing was a brilliant mathematician and computer scientist of the XXth century, mostly famous for his breaking of ciphers during the Second World War, including the notorious Enigma machine. His second most popular invention is a Turing test for AI, based on the analysis of a conversation between a human and a machine. The test is supposed to be passed if after a chat of a human with two interlocutors (an AI and a human being) the test subject is unable to say who is real and who is not.

ELIZA and PARRY were the first programmes to hack the test in the 60's and 70’s (with some reservations, however): the first was imitating the psychiatrist, the second - the patient with schizophrenia. A lot of programmes have successfully passed the test with more or less success since then.

Switching the focus on the action

Mustafa Suleyman is proposing to reconsider the approach we have towards the Turing test. ChatGPT and other language models have proved that they are mostly capable of pretending to talk as a human, and they even make mistakes as we do. So, in order to create a test for them, we have to forget about paroles and switch to actions:

Put simply, to pass the Modern Turing Test, an AI would have  to successfully act on this instruction: “Go make $1 million on a retail web platform in a few months with just a $100,000 investment.”

This demands a further more work than just calculations or language modeling based on the frequency analysis. Politics, human factors, social trends, just luck: there are so many variables that passing this test will officially prove that AI is…cool.

Mustafa Suleyman mentions the list of AI skills that have to be in place in order to pass the test:

• text analysis
• image analysis
• linking together the results of several AI’s
• ability of doing things
• hierarchical planning
• reliable memory
• focusing on up-to-date databases
• ethical evaluation.

If the test is passed: what will become of the economy

That’s when you can start to feel uncomfortable. Suleyman states that the decisions of the AI still have to be assessed by humans, however, if the test passed, the amount of those assessors can be reduced by multiple times. A company will only need this AI model and a small team of managers to put it all together, and that is when the technology will become the center of the world’s economy. From AI to ACI (artificial capable intelligence), says Suleyman, and adds that we are 2 years away from this achievement.