With the rise of AI-driven writing assistants, there has been a growing concern among academic institutes regarding the use of duplicate work by students.
As we all know, AI tools like ChatGPT can generate thousands of words within seconds, and while AI detection tools are evolving with time, none proves to be 100% effective against AI-generated content.
Today, we are discussing Turnitin, an AI content detection tool built for educators that claims to distinguish between human-written and AI content. The question is: Can Turnitin detect ChatGPT? Let’s investigate.
While ChatGPT is a groundbreaking AI invention, it somehow has challenged institutes to uphold academic integrity, as more and more students are now using the platform to complete their assignments and research papers. Let’s see if Turnitin has done enough to maintain the originality of the content.
Can Turnitin Detect ChatGPT Content?
Turnitin, which has done remarkable work in similarity detection over the years, has now introduced an AI detection tool that claims to identify texts from ChatGPT.
While Turnitin doesn’t oppose the integration of Artificial Intelligence in learning and development, it stands for academic integrity. It ensures that the technology isn’t used uncontrollably, which could impact the learning process.
After collaborating with educators and students worldwide, the platform has developed AI detection tools to scan through student’s submitted work. The detection tool gives insights into how much of the content is original and how much is coming from Artificial Intelligence.
It’s important to notice that Turnitin doesn’t conclusively determine academic misconduct, even though the submitted text appears to be generated by AI. Instead, it provides educators with the data to help them make appropriate judgments on student’s submissions.
However, the platform also claims that there is a 1% chance of wrongly identifying human content as AI-generated, so educators must apply their professional judgment to determine the final assessment of the work.
Is Turnitin Helpful to Educators?
Turnitin was established in 1998 as a similarity detection platform, allowing institutes and educators to combat plagiarism. The company sells its licenses to high schools and universities, which can use its workflow to automate the submission and plagiarism detection process.
The platform maintains a huge library of academic data, including previous submissions, research papers, essays, and other educational content. The tool compares the submission against the database and the content from other relevant online sources to identify plagiarism.
Based on the assessment, an originality report is generated, which students can review to gain insights into any plagiarism issues and enhance their writing.
Educators and institutes can ask students to submit their assignments directly through Turnitin, or they can manually upload each student’s work themselves.
Since 2014, the platform has made huge developments, nearly matching the advancements in Artificial Intelligence. Currently, it’s the most accurate plagiarism detection tool, which can also identify AI-generated content with remarkable precision.
How Does Turnitin Detect AI-Generated Content?
Turnitin uses something called a Similarity Report as a measurement of plagiarism in any given content. As I mentioned above, students can directly submit their work on Turnitin, or educators can do it manually.
When an assignment is tested for plagiarism, Turnitin employs a suite of tools to compare it with a huge database of academic publications, previously submitted papers, research reports, websites, online educational materials, etc.
Turnitin has also incorporated AI detection tools, which can further scan the document to find the possible involvement of AI. Turnitin detection algorithms have been designed to recognize texts generated by ChatGPT’s popular language models, including GPT-33 and GPT-3.5.
Over the past few months, the platform has continuously improved its algorithms and detection parameters to keep up with the fast-evolving ChatGPT language models.
To accomplish this, the tool breaks down the submission into multiple sections of words, depending on the length of the document. Each segment is thoroughly inspected for AI content.
Turnitin uses its AI language model, which is heavily trained with academic-based texts and AI-generated content. It allows the model to identify patterns and features that distinguish human-generated content from AI-generated one.
Based on the assessment, each segment is given a score of 0 or 1; 0 indicates no AI text, and 1 indicates AI text. After the whole document is scanned, an average score is generated, which refers to the percentage of AI-generated content within the submission.
What is Turnitin AI Writing Indicator?
The AI writing indicator is the percentage of AI involvement in the given text. The AI writing indicator is displayed on the left-side panel when you open the Similarity Report.
As you review a document, this indicator will show one of the three results:
1. Blue with Percentage:
This result indicates that the tool has processed the document successfully. The percentage indicates the portion of text that the Turnitin detection tool has determined to be AI-generated.
However, please remember that this percentage may not necessarily be the evaluation of the entire submission. Turnitin’s evaluation specifically pertains to text categorized as long-form, which may not include shorter text elements in this assessment.
If the score shows an asterisk (*) next to it, it means that it’s not reliable and must not be used as factual proof to take any action against the student.
You can access the detailed AI writing report to examine the result before making your final judgment.
2. Gray with No Percentage:
If the indicator appears gray without any percentage, the detection tool cannot process the document. In this case, the AI writing report will not be generated due to the incorrect format and length of the document.
3. Red Error:
The red indicates that the Turnitin detection tool failed to access the document and will not generate an AI writing report. This happens mainly due to technical difficulties.
How Accurate is Turnitin AI Detection?
Being a relatively new platform, some questions exist regarding Turnitin AI detection accuracy. The creators claim it to be approximately 98% accurate. However, no conclusive evidence exists that the tool will perform with such reliability every time.
As we all know, language models are constantly improving, adding more and more content to their databases. While. Turnitin has done a staggering job keeping up with GPT-3 and GPT-3.5 models; somehow, it remains challenging to match the development of an organization like Open AI.
Turnitin AI detection false positives can go up to 10%, which means the tool can wrongly identify human-authored content as AI-generated. Students often raised concerns regarding false AI detection within their submissions.
Nevertheless, Turnitin scored the highest in our assessment among all the tools we tested in the past. Although, for a better assessment, we would like to put it to the test with a broader range of content, not just academic material.
The Turnitin AI detection tool identified 85-90% of AI-generated texts with remarkable precision in our testing. We also tried it with a blend of AI and human content, and it successfully managed to outline the AI content with an accurate score.
While no AI detection tool can claim 100% reliability and accuracy, Turnitin outperforms most of them. The creators have promised that they will keep improving Turnitin to be as accurate as they can be.
Can Turnitin Detect GPT 4?
Since OpenAI has released its latest GPT-4 language model within ChatGPT, many wonder whether Turnitin can detect it.
Yes, the detection tool can identify GPT-4 content, but the accuracy is not great. We tested it with an essay generated by GPT-4, and the AI writing indicator returned a score of 67%, failing to identify more than a third of GPT-4 AI content.
We don’t have any official confirmation from the creators regarding GPT-4 detection capabilities. However, the company has made its stance clear that with the release of GPT-4, it will continue its development and testing to keep pace with the upcoming developments and language models from OpenAI.
In some countries, Turnitin has started testing its detection algorithms with the GPT-4 model. It’s just a matter of time before this feature officially becomes a part of the tool.
Frequently Asked Questions:
Yes. Turnitin can detect AI writing, including the content generated by GPT-3 and GPT-3.5 language models from OpenAI.
Yes. Even if you paraphrase your content, Turnitin can identify AI-generated content based on distinguished patterns and features commonly found in AI-based content.
No. Turnitin is not free. Its subscription costs $3/student/year. This means an educational institute with 100 students will pay $300 annually.
The best is to stick to original content and avoid copying from previous papers, websites, or books. You can use AI text generation platforms as a resource, but directly copying from them can land you in trouble.
Final Word:
So yes, Turnitin can detect ChatGPT text; however, with the language models constantly evolving, it will be interesting to see how long it can keep up. The creators are gearing up for the upcoming developments and testing the tool with the latest language models to make it future-proof.
The only disadvantage is that the tool is unreliable when tested against the content from Google Bard, Bing Chat, etc. While the company seems to be focused on countering ChatGPT only, we are hopeful it will also start looking in other directions.
I am an engineer by education, tech writer by profession, and a motor sport fan by heart.
