Walter R. Boot , Cheryl L. Dunn , Bachman P. Fulmer , Gregory J. Gerard , Severin V. Grabski
{"title":"同义词概念模型验证的眼动追踪实验","authors":"Walter R. Boot , Cheryl L. Dunn , Bachman P. Fulmer , Gregory J. Gerard , Severin V. Grabski","doi":"10.1016/j.accinf.2022.100578","DOIUrl":null,"url":null,"abstract":"<div><p>A key advantage of conceptual models is that their quality can be evaluated and validated before beginning the costlier stages of information system development. Few research studies investigate the validation process for such models, particularly regarding multiplicities, even though multiplicity mistakes can be very costly. We investigated the validation of conceptual model multiplicities, varying how closely natural language statements of business rules match the models that purport to represent those rules. Participants in an eye tracking experiment completed validation tasks in which they viewed a statement and an accompanying UML class diagram in which a specified multiplicity was consistent with the statement (valid) or inconsistent with the statement (invalid). We varied whether the focal multiplicity was a minimum or a maximum and varied the class diagram’s semantics and order compared to that of the statement. Logistic regression was used to analyze the relationship between accuracy and the experimental manipulations and controls. The results show that the odds of accuracy in validating class diagrams that used synonyms instead of the exact statement terminology were only 0.46 times the odds of accuracy when the class diagram and statement words matched, showing a costly effect of synonymy. Interestingly, independent of the three levels of relative semantics, the odds of accuracy were 0.48 times when class diagrams were consistent with business rules as they were when class diagrams were inconsistent with business rules. To gain insight into cognition under correct task performance, we conducted additional linear regression analysis on various eye tracking metrics for only the accurate responses. Again, synonymy was observed to be costly, with a cognitive burden of increased integrative transitions between statement and model in the range of 39 to 66%.</p></div>","PeriodicalId":47170,"journal":{"name":"International Journal of Accounting Information Systems","volume":null,"pages":null},"PeriodicalIF":4.1000,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"An eye tracking experiment investigating synonymy in conceptual model validation\",\"authors\":\"Walter R. Boot , Cheryl L. Dunn , Bachman P. Fulmer , Gregory J. Gerard , Severin V. Grabski\",\"doi\":\"10.1016/j.accinf.2022.100578\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>A key advantage of conceptual models is that their quality can be evaluated and validated before beginning the costlier stages of information system development. Few research studies investigate the validation process for such models, particularly regarding multiplicities, even though multiplicity mistakes can be very costly. We investigated the validation of conceptual model multiplicities, varying how closely natural language statements of business rules match the models that purport to represent those rules. Participants in an eye tracking experiment completed validation tasks in which they viewed a statement and an accompanying UML class diagram in which a specified multiplicity was consistent with the statement (valid) or inconsistent with the statement (invalid). We varied whether the focal multiplicity was a minimum or a maximum and varied the class diagram’s semantics and order compared to that of the statement. Logistic regression was used to analyze the relationship between accuracy and the experimental manipulations and controls. The results show that the odds of accuracy in validating class diagrams that used synonyms instead of the exact statement terminology were only 0.46 times the odds of accuracy when the class diagram and statement words matched, showing a costly effect of synonymy. Interestingly, independent of the three levels of relative semantics, the odds of accuracy were 0.48 times when class diagrams were consistent with business rules as they were when class diagrams were inconsistent with business rules. To gain insight into cognition under correct task performance, we conducted additional linear regression analysis on various eye tracking metrics for only the accurate responses. Again, synonymy was observed to be costly, with a cognitive burden of increased integrative transitions between statement and model in the range of 39 to 66%.</p></div>\",\"PeriodicalId\":47170,\"journal\":{\"name\":\"International Journal of Accounting Information Systems\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.1000,\"publicationDate\":\"2022-12-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Accounting Information Systems\",\"FirstCategoryId\":\"91\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1467089522000306\",\"RegionNum\":3,\"RegionCategory\":\"管理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"BUSINESS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Accounting Information Systems","FirstCategoryId":"91","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1467089522000306","RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"BUSINESS","Score":null,"Total":0}
An eye tracking experiment investigating synonymy in conceptual model validation
A key advantage of conceptual models is that their quality can be evaluated and validated before beginning the costlier stages of information system development. Few research studies investigate the validation process for such models, particularly regarding multiplicities, even though multiplicity mistakes can be very costly. We investigated the validation of conceptual model multiplicities, varying how closely natural language statements of business rules match the models that purport to represent those rules. Participants in an eye tracking experiment completed validation tasks in which they viewed a statement and an accompanying UML class diagram in which a specified multiplicity was consistent with the statement (valid) or inconsistent with the statement (invalid). We varied whether the focal multiplicity was a minimum or a maximum and varied the class diagram’s semantics and order compared to that of the statement. Logistic regression was used to analyze the relationship between accuracy and the experimental manipulations and controls. The results show that the odds of accuracy in validating class diagrams that used synonyms instead of the exact statement terminology were only 0.46 times the odds of accuracy when the class diagram and statement words matched, showing a costly effect of synonymy. Interestingly, independent of the three levels of relative semantics, the odds of accuracy were 0.48 times when class diagrams were consistent with business rules as they were when class diagrams were inconsistent with business rules. To gain insight into cognition under correct task performance, we conducted additional linear regression analysis on various eye tracking metrics for only the accurate responses. Again, synonymy was observed to be costly, with a cognitive burden of increased integrative transitions between statement and model in the range of 39 to 66%.
期刊介绍:
The International Journal of Accounting Information Systems will publish thoughtful, well developed articles that examine the rapidly evolving relationship between accounting and information technology. Articles may range from empirical to analytical, from practice-based to the development of new techniques, but must be related to problems facing the integration of accounting and information technology. The journal will address (but will not limit itself to) the following specific issues: control and auditability of information systems; management of information technology; artificial intelligence research in accounting; development issues in accounting and information systems; human factors issues related to information technology; development of theories related to information technology; methodological issues in information technology research; information systems validation; human–computer interaction research in accounting information systems. The journal welcomes and encourages articles from both practitioners and academicians.