Readers rate ChatGPT stories higher than human-written fiction

Stories generated by ChatGPT were rated as more engaging and higher quality than those written by humans, The Guardian reports, citing a study published in the journal Judgment and Decision Making.

In the first experiment, 1,682 adults read one of six short stories. Three were written by humans and three were generated by ChatGPT on similar themes. Participants were told whether the story had been written by a human or AI, although that information was not always accurate. They were then asked to rate the story’s quality and how engaging they found it.

• AI-generated stories received higher ratings for both engagement and quality than human-written ones. At the same time, stories that participants were told had been written by a human were rated more highly, regardless of who had actually written them.
• In two additional experiments, 905 participants read pairs of stories on the same topic, one written by a human and the other generated by AI, and tried to identify which was which. In one experiment, they were correct about 40% of the time, worse than chance, while in the other the figure was 52%.
• Participants with a better understanding of how AI systems work were more successful at identifying the authors. Being well-read in fiction made no difference.

“AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly,” said Deena Skolnick Weisberg, the study’s senior author and a professor at Villanova University.

According to Weisberg, AI-generated stories tend to be easier to read, while human-written fiction is often more subtle and complex. AI typically states a story’s theme more explicitly, whereas human authors leave more room for readers to draw their own conclusions. Still, she argued that the findings are no reason to hand writing over to AI, as people write for self-expression and to make sense of their experiences.

Luke Kennard, a professor of creative writing and film at the University of Birmingham who was not involved in the study, took a more critical view. He argued that AI should not be treated as a simple tool like a spellchecker, describing it instead as an existential threat trained on vast amounts of other people’s writing and powered by data centers with a significant environmental footprint.