An AI classifier for detecting text generated by artificial intelligence

OpenAI presents a classifier designed to differentiate between text written by AI and text written by humans.

OpenAI has developed a classifier that can differentiate between texts written by humans and those generated by various AI providers. Although it is impossible to accurately identify all AI-generated texts, a good classifier can help combat false claims that AI text was written by a human, such as in spreading misinformation, academic dishonesty, or posing AI chatbots as humans.

It must be noted that the classifier is not foolproof. In English text challenge set, it correctly identifies 26% of AI-generated texts as "likely AI-written" (true positives), but mislabels human-written texts as AI-generated 9% of the time (false positives). The accuracy of the classifier generally improves with longer inputs. Compared to their previous classifier, this new version is significantly more effective in identifying texts from more recent AI systems.

OpenAI is making this classifier publicly available for feedback on its usefulness. They continue to work on detecting AI-generated texts and aim to share more advanced methods in the future.

The classifier is a language model that has been fine-tuned using a dataset of text pairs written by both humans and AI on the same topic. The source of the human-written text was from various places including the pre-training data and human demonstrations from InstructGPT prompt submissions. The texts were separated into prompts and responses, and responses were generated using different language models trained by OpenAI and other organizations. To maintain a low false positive rate, the confidence threshold was adjusted in the web app, only marking text as likely AI-written if the classifier is highly confident.