Teachers Are Absolutely Loving The Student Who Made A Tool That Shows If Your Essay Was Written By AI
GPTZero can tell if an essay about Hamlet was written using a bot.Katie Notopoulos
BuzzFeed News Reporter
Posted on January 5, 2023 at 4:18 pm
Baona / Getty Images/iStockphoto
High school English students who were hoping to use artificial intelligence to write their homework have a new enemy: Edward Tian, a 22-year-old senior at Princeton University, who created a website that can detect if a piece of writing has been created using the AI tool ChatGPT.
Meanwhile, instructors everywhere are rejoicing. “So many teachers have reached out to me,” said Tian, whose recent tweet about his tool, GPTZero, went viral. “From Switzerland, France, all over the world.”
The latest version of ChatGPT, called GPT3, was released to the public in late November. The tool is able to produce amazingly coherent writing, which has endless possibilities, ranging from wonderful things (like allowing a pool installer with dyslexia to communicate effectively with his customers over email) to more nefarious uses.
Educators fear that high school or college students might use this technology to, say, write their homework assignment summarizing the main events of the Battle of Tippecanoe. Their concerns are not misplaced: A reporter at the Wall Street Journal recently succeeded in using ChatGPT to help her pass the AP English test.
According to Chalkbeat, New York City public schools just blocked access to the ChatGPT website on school devices and internet networks. The anxiety is even coming from inside the house: A top AI conference has banned the submission of academic papers written entirely by AI (although it will allow some tools to “polish” writers’ work).
“AI is here to stay,” said Tian, a computer science major and journalism minor who coded the tool over a few days during winter break. “AI-generated writing is going to just get better and better. I’m excited about this future, but we have to do it responsibly.”
He isn’t against using AI tools for writing, but he sees this as a precarious moment. “I want people to use ChatGPT,” he said. “And it's only going to be normalized, but it has to have safeguards.”
Edward Tian, the 22-year-old student who created GPTZero
GPTZero works by analyzing a piece of text and determining if there is a high or low indication that a bot wrote it. It looks for two hallmarks: “perplexity” and “burstiness.” “Perplexity” is how likely each word is to be suggested by a bot; a human would be more random. “Burstiness” measures the spikes in how perplex each sentence is. A bot will likely have a similar degree of perplexity sentence to sentence, but a human is going to write with spikes — maybe one long, complex sentence followed by a shorter one. Like this.
To test out Tian’s creation, I fed it a short essay written by ChatGPT using a prompt that a would-be high school cheater might try: Describe the main theme of Hamlet. (“The main theme of Shakespeare's play ‘Hamlet’ is the struggle of the main character, Hamlet, to come to terms with the fact that his uncle has murdered his father and taken the throne…” blah blah and so on.)
GPTZero gave the essay a perplexity score of 10 and a burstiness score of 19 (these are pretty low scores, Tian explained, meaning the writer was more likely to be a bot). It correctly detected this was likely written by AI.
For comparison, I entered the first half of this article, which I wrote myself, into the tool. Perplexity: 39; burstiness: 387. (Ironically, it determined the sentence with highest perplexity was “I want people to use ChatGPT,” he said.) Ultimately, GPTZero deemed the essay likely to be human. Correct!
However, the exact success rate of GPTZero is unclear. At least one Twitter user said that it failed to catch a few of their AI-written samples. Elsewhere on the platform, the reaction has been mixed: Adults are praising the effort, and others, mostly teens, are calling Tian a “narc.”
Tian told the Daily Beast in an interview that after his tweet about it, his DMs were blowing up from venture capital interest. For now, though, he plans on keeping his creation free and accessible. “I want to support freshman English teachers everywhere,” he said.
Source:https://www.buzzfeednews.com/article/katienotopoulos/ai-writing-detection-tool-homework-students
numbers....
