Pub Date : 2023-12-08DOI: 10.1109/mis.2023.3328527
{"title":"IEEE Computer Society Career Center","authors":"","doi":"10.1109/mis.2023.3328527","DOIUrl":"https://doi.org/10.1109/mis.2023.3328527","url":null,"abstract":"","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"37 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1109/mis.2023.3334639
Roberto Confalonieri, Jose Maria Alonso-Moral
The assessment of explanations by humans presents a significant challenge within the context of explainable and trustworthy artificial intelligence. This is attributed not only to the absence of universal metrics and standardized evaluation methods but also to the complexities tied to devising user studies that assess the perceived human comprehensibility of these explanations. To address this gap, we introduce a survey-based methodology for guiding the human evaluation of explanations. This approach amalgamates leading practices from existing literature and is implemented as an operational framework. This framework assists researchers throughout the evaluation process, encompassing hypothesis formulation, online user study implementation and deployment, and analysis and interpretation of collected data. The application of this framework is exemplified through two practical user studies.
{"title":"An Operational Framework for Guiding Human Evaluation in Explainable and Trustworthy Artificial Intelligence","authors":"Roberto Confalonieri, Jose Maria Alonso-Moral","doi":"10.1109/mis.2023.3334639","DOIUrl":"https://doi.org/10.1109/mis.2023.3334639","url":null,"abstract":"The assessment of explanations by humans presents a significant challenge within the context of explainable and trustworthy artificial intelligence. This is attributed not only to the absence of universal metrics and standardized evaluation methods but also to the complexities tied to devising user studies that assess the perceived human comprehensibility of these explanations. To address this gap, we introduce a survey-based methodology for guiding the human evaluation of explanations. This approach amalgamates leading practices from existing literature and is implemented as an operational framework. This framework assists researchers throughout the evaluation process, encompassing hypothesis formulation, online user study implementation and deployment, and analysis and interpretation of collected data. The application of this framework is exemplified through two practical user studies.","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"11 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140002424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-01DOI: 10.1109/mis.2023.3328519
{"title":"IEEE Computer Society D & I Fund: Drive Diversity & Inclusion in Computing","authors":"","doi":"10.1109/mis.2023.3328519","DOIUrl":"https://doi.org/10.1109/mis.2023.3328519","url":null,"abstract":"","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"45 1","pages":""},"PeriodicalIF":6.4,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139298215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-01DOI: 10.1109/MIS.2023.3291567
Zikang Wang, Linjing Li, Jinlin Li, Pengfei Zhao, D. Zeng
Multihop reasoning is essential in knowledge graph (KG) research and applications. Current methods rely on specific KG entities, while human cognition operates at a more abstract level. This article proposes a category-aware rule-based (CRule) approach for symbolic multihop reasoning. Specifically, given a KG, CRule first categorizes entities and constructs a category-aware KG; it then uses rules retrieved from the categorized KG to perform multihop reasoning on the original KG. Experiments on five datasets show that CRule is simple, is effective, and combines the advantages of symbolic and neural network methods. It overcomes symbolic reasoning’s complexity limitations, can perform reasoning on KGs of more than 300,000 edges, and can be three times more efficient than neural network models.
{"title":"CRule: Category-Aware Symbolic Multihop Reasoning on Knowledge Graphs","authors":"Zikang Wang, Linjing Li, Jinlin Li, Pengfei Zhao, D. Zeng","doi":"10.1109/MIS.2023.3291567","DOIUrl":"https://doi.org/10.1109/MIS.2023.3291567","url":null,"abstract":"Multihop reasoning is essential in knowledge graph (KG) research and applications. Current methods rely on specific KG entities, while human cognition operates at a more abstract level. This article proposes a category-aware rule-based (CRule) approach for symbolic multihop reasoning. Specifically, given a KG, CRule first categorizes entities and constructs a category-aware KG; it then uses rules retrieved from the categorized KG to perform multihop reasoning on the original KG. Experiments on five datasets show that CRule is simple, is effective, and combines the advantages of symbolic and neural network methods. It overcomes symbolic reasoning’s complexity limitations, can perform reasoning on KGs of more than 300,000 edges, and can be three times more efficient than neural network models.","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"38 1","pages":"56-64"},"PeriodicalIF":6.4,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45862641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fact verification is a very challenging task that requires retrieving multiple evidence sentences from a reliable corpus to authenticate claims. Many claims require the simultaneous integration and reasoning of several pieces of evidence for verification. Existing models exhibit limitations in two aspects: 1) during the sentence selection stage, they only consider the interaction between the claim and the evidence, disregarding the intersentence information, and 2) most fusion strategies employed in current research, such as addition, concatenation, or simple neural networks, fail to capture the relationships and logical information among the evidence. To alleviate these problems, we propose select and fact verification modeling (SFVM). Our model utilizes a multihead self-attention mechanism combined with a gating mechanism to facilitate sentence interaction and enhance sentence embeddings. Then, we utilize factorization machines to effectively express the compressed alignment vectors, which are then used to expand the representations of the base evidence. To distinguish the importance of features, we use the evidence fusion network to determine the importance of various feature interactions. Results from experiments on the two public datasets showed that SFVM can utilize richer information between the claim and the evidence for fact verification and achieve competitive performance on the FEVER dataset.
{"title":"Effectively Modeling Sentence Interactions With Factorization Machines for Fact Verification","authors":"Zhen-Heng Chen, Fuzhen Zhuang, Lejian Liao, Meihuizi Jia, Jiaqi Li, Heyan Huang","doi":"10.1109/MIS.2023.3301170","DOIUrl":"https://doi.org/10.1109/MIS.2023.3301170","url":null,"abstract":"Fact verification is a very challenging task that requires retrieving multiple evidence sentences from a reliable corpus to authenticate claims. Many claims require the simultaneous integration and reasoning of several pieces of evidence for verification. Existing models exhibit limitations in two aspects: 1) during the sentence selection stage, they only consider the interaction between the claim and the evidence, disregarding the intersentence information, and 2) most fusion strategies employed in current research, such as addition, concatenation, or simple neural networks, fail to capture the relationships and logical information among the evidence. To alleviate these problems, we propose select and fact verification modeling (SFVM). Our model utilizes a multihead self-attention mechanism combined with a gating mechanism to facilitate sentence interaction and enhance sentence embeddings. Then, we utilize factorization machines to effectively express the compressed alignment vectors, which are then used to expand the representations of the base evidence. To distinguish the importance of features, we use the evidence fusion network to determine the importance of various feature interactions. Results from experiments on the two public datasets showed that SFVM can utilize richer information between the claim and the evidence for fact verification and achieve competitive performance on the FEVER dataset.","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"38 1","pages":"18-27"},"PeriodicalIF":6.4,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44360647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-01DOI: 10.1109/MIS.2023.3265868
Xiao Liu, Shuyang Liu, Bo An, Yang Gao, Shangdong Yang, Wenbin Li
Interpretable policy distillation aims to imitate a deep reinforcement learning (DRL) policy into a self-explainable model. However, the distilled policy usually does not generalize well to complex tasks. To investigate this phenomenon, we examine the experience pools of DRL tasks and find that these interactive experience distributions are heavy tailed. However, this critical issue is largely ignored by existing approaches, and, thus, they do not fully unitize the less frequent but very critical experience points. To address this issue, we propose characterizing decision boundaries via the minimum experience retention to deal with the heavy-tailed experience distributions. Our method identifies critical experience points that are close to the model’s decision boundaries, and such experience points are more critical because they portray the prerequisite of a model to take an action. As a result, our method distills the DRL policy to a self-explainable structure without a neural structure and ambiguous intermediate parameters. Through experiments on six games, we show that our method outperforms the state-of-the-art baselines in cumulative rewards, stability, and faithfulness.
{"title":"Effective Interpretable Policy Distillation via Critical Experience Point Identification","authors":"Xiao Liu, Shuyang Liu, Bo An, Yang Gao, Shangdong Yang, Wenbin Li","doi":"10.1109/MIS.2023.3265868","DOIUrl":"https://doi.org/10.1109/MIS.2023.3265868","url":null,"abstract":"Interpretable policy distillation aims to imitate a deep reinforcement learning (DRL) policy into a self-explainable model. However, the distilled policy usually does not generalize well to complex tasks. To investigate this phenomenon, we examine the experience pools of DRL tasks and find that these interactive experience distributions are heavy tailed. However, this critical issue is largely ignored by existing approaches, and, thus, they do not fully unitize the less frequent but very critical experience points. To address this issue, we propose characterizing decision boundaries via the minimum experience retention to deal with the heavy-tailed experience distributions. Our method identifies critical experience points that are close to the model’s decision boundaries, and such experience points are more critical because they portray the prerequisite of a model to take an action. As a result, our method distills the DRL policy to a self-explainable structure without a neural structure and ambiguous intermediate parameters. Through experiments on six games, we show that our method outperforms the state-of-the-art baselines in cumulative rewards, stability, and faithfulness.","PeriodicalId":13160,"journal":{"name":"IEEE Intelligent Systems","volume":"38 1","pages":"28-36"},"PeriodicalIF":6.4,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41930464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：