{"title":"人机协作:揭示智能决策支持系统中用户熟练程度和人工智能代理能力的影响","authors":"","doi":"10.1016/j.ergon.2024.103629","DOIUrl":null,"url":null,"abstract":"<div><p>Artificial intelligence (AI) agents are integral components of modern intelligent decision support systems (IDSS), providing their capability to assist in decision-making processes. Understanding the influence of AI involvement on human responses within these systems is crucial for fostering appropriate reliance on AI advice and ensuring a comfortable user experience. This study delves into the impact of AI agents on decision change, correct rate change, confidence rate change and pleasure rating among users. It explores the main and interactive effects of user proficiency and AI agent capability on user responses. A block-factorial experiment involving 45 participants was conducted. The results indicated that high-capability AI agents were associated with substantial decision change, correct rate change and confidence rate change. Participants with higher proficiency, both subjectively and objectively, exhibited reduced reliance on AI agents for task execution. Notably, the involvement of AI agents led to a slight decline in correct rates for participants of high competence (i.e., participants' independent correct rate exceeds AI agent capability), while still enhancing their confidence levels. Additionally, agents lacking human-like embodiment tended to enhance participants’ pleasure ratings. Individual differences in gender, collaboration experience with AI agent, and stereotype on AI agent capability also shaped the decision changes and pleasure ratings of participants. These findings underscore the presence of complementary competence interaction between user and AI agent within IDSS, and can be applied in designing capability of advisory AI agents with consideration of user proficiency and individual characteristics.</p></div><div><h3>Relevance to industry</h3><p>The findings can be used as guidelines for designing AI agents and specifying their application strategies to integrate the assistance of AI agents effectively in intelligent decision support systems.</p></div>","PeriodicalId":50317,"journal":{"name":"International Journal of Industrial Ergonomics","volume":null,"pages":null},"PeriodicalIF":2.5000,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Human-AI collaboration: Unraveling the effects of user proficiency and AI agent capability in intelligent decision support systems\",\"authors\":\"\",\"doi\":\"10.1016/j.ergon.2024.103629\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Artificial intelligence (AI) agents are integral components of modern intelligent decision support systems (IDSS), providing their capability to assist in decision-making processes. Understanding the influence of AI involvement on human responses within these systems is crucial for fostering appropriate reliance on AI advice and ensuring a comfortable user experience. This study delves into the impact of AI agents on decision change, correct rate change, confidence rate change and pleasure rating among users. It explores the main and interactive effects of user proficiency and AI agent capability on user responses. A block-factorial experiment involving 45 participants was conducted. The results indicated that high-capability AI agents were associated with substantial decision change, correct rate change and confidence rate change. Participants with higher proficiency, both subjectively and objectively, exhibited reduced reliance on AI agents for task execution. Notably, the involvement of AI agents led to a slight decline in correct rates for participants of high competence (i.e., participants' independent correct rate exceeds AI agent capability), while still enhancing their confidence levels. Additionally, agents lacking human-like embodiment tended to enhance participants’ pleasure ratings. Individual differences in gender, collaboration experience with AI agent, and stereotype on AI agent capability also shaped the decision changes and pleasure ratings of participants. These findings underscore the presence of complementary competence interaction between user and AI agent within IDSS, and can be applied in designing capability of advisory AI agents with consideration of user proficiency and individual characteristics.</p></div><div><h3>Relevance to industry</h3><p>The findings can be used as guidelines for designing AI agents and specifying their application strategies to integrate the assistance of AI agents effectively in intelligent decision support systems.</p></div>\",\"PeriodicalId\":50317,\"journal\":{\"name\":\"International Journal of Industrial Ergonomics\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.5000,\"publicationDate\":\"2024-08-12\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Industrial Ergonomics\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0169814124000854\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, INDUSTRIAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Industrial Ergonomics","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0169814124000854","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
Human-AI collaboration: Unraveling the effects of user proficiency and AI agent capability in intelligent decision support systems
Artificial intelligence (AI) agents are integral components of modern intelligent decision support systems (IDSS), providing their capability to assist in decision-making processes. Understanding the influence of AI involvement on human responses within these systems is crucial for fostering appropriate reliance on AI advice and ensuring a comfortable user experience. This study delves into the impact of AI agents on decision change, correct rate change, confidence rate change and pleasure rating among users. It explores the main and interactive effects of user proficiency and AI agent capability on user responses. A block-factorial experiment involving 45 participants was conducted. The results indicated that high-capability AI agents were associated with substantial decision change, correct rate change and confidence rate change. Participants with higher proficiency, both subjectively and objectively, exhibited reduced reliance on AI agents for task execution. Notably, the involvement of AI agents led to a slight decline in correct rates for participants of high competence (i.e., participants' independent correct rate exceeds AI agent capability), while still enhancing their confidence levels. Additionally, agents lacking human-like embodiment tended to enhance participants’ pleasure ratings. Individual differences in gender, collaboration experience with AI agent, and stereotype on AI agent capability also shaped the decision changes and pleasure ratings of participants. These findings underscore the presence of complementary competence interaction between user and AI agent within IDSS, and can be applied in designing capability of advisory AI agents with consideration of user proficiency and individual characteristics.
Relevance to industry
The findings can be used as guidelines for designing AI agents and specifying their application strategies to integrate the assistance of AI agents effectively in intelligent decision support systems.
期刊介绍:
The journal publishes original contributions that add to our understanding of the role of humans in today systems and the interactions thereof with various system components. The journal typically covers the following areas: industrial and occupational ergonomics, design of systems, tools and equipment, human performance measurement and modeling, human productivity, humans in technologically complex systems, and safety. The focus of the articles includes basic theoretical advances, applications, case studies, new methodologies and procedures; and empirical studies.