Pub Date : 2024-06-24DOI: 10.1109/mis.2024.3396371
Daniel E. O’Leary
This article compares numeric assessments generated by ChatGPT and Claude along four dimensions of novelty, feasibility, impact, and disruption, to study their ability to rate ideas. We find that those chatbots make numeric assessments that are consistent with the expected relationships between those dimensions, for example, novelty is negatively correlated with feasibility. We also find that the two chatbots make statistically significantly different numeric assessments of the same idea information. We suggest that this type of analysis can also be used to provide a type of validation of underlying chatbot capabilities. In addition, we suggest that, as part of their chatbot requirements analysis, enterprises use this approach to ensure that the chatbot appropriately “understands” concepts, in which they are directly interested.
本文比较了 ChatGPT 和 Claude 从新颖性、可行性、影响力和破坏性四个维度生成的数字评估，以研究它们对想法进行评级的能力。我们发现，这些聊天机器人做出的数字评估符合这些维度之间的预期关系，例如，新颖性与可行性呈负相关。我们还发现，两个聊天机器人对相同创意信息的数字评估在统计学上存在显著差异。我们认为这种分析也可以用来验证聊天机器人的基本能力。此外，我们建议，作为聊天机器人需求分析的一部分，企业可以使用这种方法来确保聊天机器人能够恰当地 "理解 "他们直接感兴趣的概念。
{"title":"A Comparison of Numeric Assessments of Ideas From Two Large Language Models: With Implications for Validating and Choosing LLMs","authors":"Daniel E. O’Leary","doi":"10.1109/mis.2024.3396371","DOIUrl":"https://doi.org/10.1109/mis.2024.3396371","url":null,"abstract":"This article compares numeric assessments generated by ChatGPT and Claude along four dimensions of novelty, feasibility, impact, and disruption, to study their ability to rate ideas. We find that those chatbots make numeric assessments that are consistent with the expected relationships between those dimensions, for example, novelty is negatively correlated with feasibility. We also find that the two chatbots make statistically significantly different numeric assessments of the same idea information. We suggest that this type of analysis can also be used to provide a type of validation of underlying chatbot capabilities. In addition, we suggest that, as part of their chatbot requirements analysis, enterprises use this approach to ensure that the chatbot appropriately “understands” concepts, in which they are directly interested.","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"30 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2024-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141524008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-17DOI: 10.1109/mis.2024.3408290
Trong-Nghia Nguyen, Soo-Hyung Kim, Bo-Gun Kho, Hyung-Jeong Yang
{"title":"Multi-Gradient Siamese Temporal Model for the Prediction of Clinical Events in Rapid Response Systems","authors":"Trong-Nghia Nguyen, Soo-Hyung Kim, Bo-Gun Kho, Hyung-Jeong Yang","doi":"10.1109/mis.2024.3408290","DOIUrl":"https://doi.org/10.1109/mis.2024.3408290","url":null,"abstract":"","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"1 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141937278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-10DOI: 10.1109/mis.2024.3399053
Xu Bo, Gao Bin, Li Yunhu
{"title":"Improved Small Object Detection Algorithm Based on YOLOv5","authors":"Xu Bo, Gao Bin, Li Yunhu","doi":"10.1109/mis.2024.3399053","DOIUrl":"https://doi.org/10.1109/mis.2024.3399053","url":null,"abstract":"","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"11 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2024-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140933700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：