Development of a Concept Inventory on Open and Transparent Research Practices

IF 3.1 3区 心理学 Q1 PSYCHOLOGY, MULTIDISCIPLINARY Collabra-Psychology Pub Date : 2023-01-01 DOI:10.1525/collabra.75226
D. Markant, Alexia Galati
{"title":"Development of a Concept Inventory on Open and Transparent Research Practices","authors":"D. Markant, Alexia Galati","doi":"10.1525/collabra.75226","DOIUrl":null,"url":null,"abstract":"Over the past decade psychology researchers have begun adopting practices that promote openness and transparency. While these practices are increasingly reflected in undergraduate psychology curricula, pedagogical research has not systematically examined whether instruction on open science practices improves students’ conceptual understanding of research methods. We developed the Open Science Concept Inventory (OSCI) to evaluate the impact of integrating open science practices into research methods courses. First, we created a set of hypothetical dilemmas related to a range of open science concepts and elicited open-ended responses from undergraduates (N = 64, Study 1). Based on the responses, we created a 40-item multiple-choice questionnaire, which we administered to a new group of participants (N = 262, Study 2) and used item response theory to select 33 items for the final OSCI. Finally, in two implementation rounds across two semesters (Study 3, total N = 37), we evaluated students’ learning gains with the OSCI in a pre-test/post-test design. The implementation rounds involved new materials on open science for a psychology research methods course, including video lectures that situated questionable research practices in the current norms of science and introduced emerging solutions. After excluding extremely fast survey responders, an exploratory analysis showed learning gains among students who expended appropriate effort when completing the OSCI. By systematically evaluating a tool that is easily integrated into existing curricula, we aim to facilitate the adoption of open science practices in undergraduate instruction and the assessment of students’ conceptual foundations for conducting robust and transparent research.","PeriodicalId":45791,"journal":{"name":"Collabra-Psychology","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Collabra-Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1525/collabra.75226","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

Over the past decade psychology researchers have begun adopting practices that promote openness and transparency. While these practices are increasingly reflected in undergraduate psychology curricula, pedagogical research has not systematically examined whether instruction on open science practices improves students’ conceptual understanding of research methods. We developed the Open Science Concept Inventory (OSCI) to evaluate the impact of integrating open science practices into research methods courses. First, we created a set of hypothetical dilemmas related to a range of open science concepts and elicited open-ended responses from undergraduates (N = 64, Study 1). Based on the responses, we created a 40-item multiple-choice questionnaire, which we administered to a new group of participants (N = 262, Study 2) and used item response theory to select 33 items for the final OSCI. Finally, in two implementation rounds across two semesters (Study 3, total N = 37), we evaluated students’ learning gains with the OSCI in a pre-test/post-test design. The implementation rounds involved new materials on open science for a psychology research methods course, including video lectures that situated questionable research practices in the current norms of science and introduced emerging solutions. After excluding extremely fast survey responders, an exploratory analysis showed learning gains among students who expended appropriate effort when completing the OSCI. By systematically evaluating a tool that is easily integrated into existing curricula, we aim to facilitate the adoption of open science practices in undergraduate instruction and the assessment of students’ conceptual foundations for conducting robust and transparent research.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
开放和透明研究实践概念清单的开发
在过去的十年里,心理学研究人员已经开始采用促进开放和透明的做法。虽然这些实践越来越多地反映在本科心理学课程中,但教学研究尚未系统地考察开放科学实践的教学是否能提高学生对研究方法的概念理解。我们开发了开放科学概念清单(OSCI)来评估将开放科学实践整合到研究方法课程中的影响。首先,我们创建了一组与一系列开放科学概念相关的假设困境,并从本科生(N = 64,研究1)中引出开放式回答。基于这些回答,我们创建了一份40项选择题问卷,并将其分发给一组新的参与者(N = 262,研究2),并使用项目反应理论选择33个项目用于最终的OSCI。最后,在两个学期的两轮实施中(研究3,总N = 37),我们在测试前/测试后设计中评估了OSCI学生的学习收益。实施轮次涉及心理学研究方法课程的开放科学新材料,包括将有问题的研究实践置于当前科学规范中并介绍新兴解决方案的视频讲座。在排除极快的调查应答者后,探索性分析显示,在完成OSCI时付出适当努力的学生获得了学习收益。通过系统地评估一个容易整合到现有课程中的工具,我们的目标是促进在本科教学中采用开放科学实践,并评估学生进行稳健和透明研究的概念基础。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Collabra-Psychology
Collabra-Psychology PSYCHOLOGY, MULTIDISCIPLINARY-
CiteScore
3.60
自引率
4.00%
发文量
47
审稿时长
16 weeks
期刊介绍: Collabra: Psychology has 7 sections representing the broad field of psychology, and a highlighted focus area of “Methodology and Research Practice.” Are: Cognitive Psychology Social Psychology Personality Psychology Clinical Psychology Developmental Psychology Organizational Behavior Methodology and Research Practice.
期刊最新文献
Characterizing Human Habits in the Lab. Breaking Ban: Belgium’s Ineffective Gambling Law Regulation of Video Game Loot Boxes Revisiting the Differential Centrality of Experiential and Material Purchases to the Self: Replication and Extension of Carter and Gilovich (2012) Cyberloafing: Investigating the Importance and Implications of New and Known Predictors Psychometric Properties of the Chilean Version of the Oviedo Grit Scale
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1