Sujita Kumar Kar, Teena Bansal, Sumit Modi, Amit Singh
{"title":"How Sensitive Are the Free AI-detector Tools in Detecting AI-generated Texts? A Comparison of Popular AI-detector Tools","authors":"Sujita Kumar Kar, Teena Bansal, Sumit Modi, Amit Singh","doi":"10.1177/02537176241247934","DOIUrl":null,"url":null,"abstract":"Background:Recently, Artificial intelligence (AI) has significantly influenced academic writing. We aimed to investigate the sensitivity of the free versions of popular AI-detection software programs in detecting AI-generated text.Methods:We searched for AI-content-detection software on Google and selected the first 10 free versions that allowed a minimum of 500 words for text analysis. Then, we gave ChatGPT 3.5 version a command to generate a scientific article on the “Role of Electroconvulsive Therapy (ECT) in Treatment-resistant Depression” under 500 words. After generating the primary text, we rephrased it using three different software tools. We then used AI-detection software to analyse the original and paraphrase texts.Results:10 AI-detector tools were tested on their ability to detect AI-generated text. The sensitivity ranged from 0% to 100%. 5 out of 10 tools detected AI-generated content with 100% accuracy. For paraphrased texts, Sapling and Undetectable AI detected all three software-generated contents with 100% accuracy. Meanwhile, Copyleaks, QuillBot, and Wordtune identified content generated by two software programs with 100% accuracy.Conclusion:The integration of AI technology in academic writing is becoming more prevalent. Nonetheless, relying solely on AI-generated content can diminish the author’s credibility, leading most academic journals to suggest limiting its use. AI-content-detection software programs have been developed to detect AI-generated or AI-assisted texts. Currently, some of the platforms are equally sensitive. However, future upgrades may enhance their ability to detect AI-generated text more accurately.","PeriodicalId":13476,"journal":{"name":"Indian Journal of Psychological Medicine","volume":null,"pages":null},"PeriodicalIF":1.9000,"publicationDate":"2024-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Indian Journal of Psychological Medicine","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1177/02537176241247934","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"PSYCHIATRY","Score":null,"Total":0}
引用次数: 0
Abstract
Background:Recently, Artificial intelligence (AI) has significantly influenced academic writing. We aimed to investigate the sensitivity of the free versions of popular AI-detection software programs in detecting AI-generated text.Methods:We searched for AI-content-detection software on Google and selected the first 10 free versions that allowed a minimum of 500 words for text analysis. Then, we gave ChatGPT 3.5 version a command to generate a scientific article on the “Role of Electroconvulsive Therapy (ECT) in Treatment-resistant Depression” under 500 words. After generating the primary text, we rephrased it using three different software tools. We then used AI-detection software to analyse the original and paraphrase texts.Results:10 AI-detector tools were tested on their ability to detect AI-generated text. The sensitivity ranged from 0% to 100%. 5 out of 10 tools detected AI-generated content with 100% accuracy. For paraphrased texts, Sapling and Undetectable AI detected all three software-generated contents with 100% accuracy. Meanwhile, Copyleaks, QuillBot, and Wordtune identified content generated by two software programs with 100% accuracy.Conclusion:The integration of AI technology in academic writing is becoming more prevalent. Nonetheless, relying solely on AI-generated content can diminish the author’s credibility, leading most academic journals to suggest limiting its use. AI-content-detection software programs have been developed to detect AI-generated or AI-assisted texts. Currently, some of the platforms are equally sensitive. However, future upgrades may enhance their ability to detect AI-generated text more accurately.
期刊介绍:
The Indian Journal of Psychological Medicine (ISSN 0253-7176) was started in 1978 as the official publication of the Indian Psychiatric Society South Zonal Branch. The journal allows free access (Open Access) and is published Bimonthly. The Journal includes but is not limited to review articles, original research, opinions, and letters. The Editor and publisher accept no legal responsibility for any opinions, omissions or errors by the authors, nor do they approve of any product advertised within the journal.