How This College Plagiarism Case Shows Catching AI Cheaters Is Tricky

A recent college controversy shows how generative artificial intelligence (AI) makes distinguishing between computer-generated writing and human prose harder. 

Texas A&M University is investigating after an instructor's email accusing students of using AI on their final assignments went viral. It's part of a growing effort to crack down on using AI to cheat on schoolwork. 

"AI-generated text has become increasingly sophisticated, making it difficult to distinguish from human-written content," Anthony Clemons, who researches AI in education at Northern Illinois University, told Lifewire in an email interview. "Existing detection tools have varying degrees of accuracy, but none are infallible​."

AI-Generated or Not?

In the recent suspected cheating case, an instructor in the agricultural sciences and natural resources department reportedly told students they would not receive credit in a course after he found they'd used AI chatbot software to write their final assignments. He said that he tested each paper twice and that the bot claimed to have written every final assignment.

However, it turned out that the professor was wrong, at least in some cases. Not all of the students used software to write their work. 

The case is an example of why figuring out whether people use AI to cheat is no simple task. Existing detection tools, such as the GPT-2 Output Detector, Writer AI Content Detector, Content at Scale, GPTZero, and Giant Language Model Test Room, have varying degrees of accuracy, but none are infallible​, Clemons said.

AI does not use plagiarism software databases to write essays which means a significant portion of what plagiarism software would use to look for examples and instances of students cheating is not used when AI creates a student essay, Andrew Selepak, a social media professor at the University of Florida, said in an email. 

"AI combs the Internet to create an essay for a student using the unlimited content of the Internet," he added. "Previous examples of the work do not exist, and therefore, the plagiarism software is unable to match a student's submitted work to previous work."

Suppose the suspected use of AI case involves identifying excerpts from a text which originate word for word in another source. In that case, detecting it is relatively simple, provided one has a large enough bank of source texts, Tim Boucher, an expert in detecting copyright infringement of text posts, said in an email. AI-generated text is a different problem because no single source is referenced as an exact copy or as copies of excerpts. 

"There is instead a massive corpus of texts which have been ingested as training data into a large language model," he added. "The model then analyzes statistical relationships between tokens (words and parts of words) and uses this to derive patterns, with which it creates entirely new texts that are not exact copies and are not paraphrasing of singular original texts."

The Battle to Detect AI-Generated Text

Detecting plagiarism is harder in disciplines where writing isn't taught, Shaun T. Schafer, the co-chair of the Generative Artificial Intelligence Taskforce at MSU Denver, said in an email. 

"I don't worry about an English instructor catching plagiarism because they are seeing writing work from a student throughout a semester," he said. "The challenge is in a discipline like marketing or history or anywhere that requires a great deal of writing but does not teach writing. It is much tougher there to catch plagiarism."

Instead of relying on new and better software to detect plagiarism, Selepak suggests going old school. 

"A better solution is to eliminate technology from the equation," Selepak said. "Rather than having students write papers and exams at home and submit them digitally, is to return to in-class, hand-written essays and exams like schools and universities used for hundreds of years before technology became so prevalent in the classroom. Pencils and pens don't come with AI."

And Boucher is optimistic that even if schools can't detect all work produced with AI, human-powered writing will ultimately prevail. 

"AI writing tools will augment it at every step of the way, from improving spelling and grammar to style and tone suggestions, all the way through to generating complete texts," Boucher added.
"Far from being a deterrent to human creativity, it will be an accelerant, enabling authors and artists of all kinds to generate more content at a greater scale than ever before."