首页 > 最新文献

Organizational Research Methods最新文献

英文 中文
A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research 强迫选择非认知评估中随机反应行为的混合模型及其在组织研究中的应用
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-06-27 DOI: 10.1177/10944281231181642
Siwei Peng, K. Man, B. Veldkamp, Yan Cai, Dongbo Tu
For various reasons, respondents to forced-choice assessments (typically used for noncognitive psychological constructs) may respond randomly to individual items due to indecision or globally due to disengagement. Thus, random responding is a complex source of measurement bias and threatens the reliability of forced-choice assessments, which are essential in high-stakes organizational testing scenarios, such as hiring decisions. The traditional measurement models rely heavily on nonrandom, construct-relevant responses to yield accurate parameter estimates. When survey data contain many random responses, fitting traditional models may deliver biased results, which could attenuate measurement reliability. This study presents a new forced-choice measure-based mixture item response theory model (called M-TCIR) for simultaneously modeling normal and random responses (distinguishing completely and incompletely random). The feasibility of the M-TCIR was investigated via two Monte Carlo simulation studies. In addition, one empirical dataset was analyzed to illustrate the applicability of the M-TCIR in practice. The results revealed that most model parameters were adequately recovered, and the M-TCIR was a viable alternative to model both aberrant and normal responses with high efficiency.
由于各种原因,被迫选择评估(通常用于非认知心理结构)的受访者可能会因犹豫不决而对个别项目做出随机反应,或因脱离而对全局做出反应。因此,随机回答是衡量偏差的复杂来源,并威胁到强制选择评估的可靠性,而强制选择评估在高风险的组织测试场景中至关重要,例如招聘决策。传统的测量模型在很大程度上依赖于非随机的、构造相关的响应来产生准确的参数估计。当调查数据包含许多随机响应时,拟合传统模型可能会产生有偏差的结果,这可能会削弱测量的可靠性。本研究提出了一种新的基于强迫选择测度的混合项目反应理论模型(称为M-TCIR),用于同时建模正常和随机反应(区分完全随机和不完全随机)。通过两次蒙特卡罗模拟研究,研究了M-TCIR的可行性。此外,还分析了一个经验数据集,以说明M-TCIR在实践中的适用性。结果表明,大多数模型参数都得到了充分恢复,M-TCIR是高效模拟异常和正常反应的可行替代方案。
{"title":"A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research","authors":"Siwei Peng, K. Man, B. Veldkamp, Yan Cai, Dongbo Tu","doi":"10.1177/10944281231181642","DOIUrl":"https://doi.org/10.1177/10944281231181642","url":null,"abstract":"For various reasons, respondents to forced-choice assessments (typically used for noncognitive psychological constructs) may respond randomly to individual items due to indecision or globally due to disengagement. Thus, random responding is a complex source of measurement bias and threatens the reliability of forced-choice assessments, which are essential in high-stakes organizational testing scenarios, such as hiring decisions. The traditional measurement models rely heavily on nonrandom, construct-relevant responses to yield accurate parameter estimates. When survey data contain many random responses, fitting traditional models may deliver biased results, which could attenuate measurement reliability. This study presents a new forced-choice measure-based mixture item response theory model (called M-TCIR) for simultaneously modeling normal and random responses (distinguishing completely and incompletely random). The feasibility of the M-TCIR was investigated via two Monte Carlo simulation studies. In addition, one empirical dataset was analyzed to illustrate the applicability of the M-TCIR in practice. The results revealed that most model parameters were adequately recovered, and the M-TCIR was a viable alternative to model both aberrant and normal responses with high efficiency.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41961861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Demographic Inference in the Digital Age: Using Neural Networks to Assess Gender and Ethnicity at Scale 数字时代的人口推断:使用神经网络大规模评估性别和种族
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-06-14 DOI: 10.1177/10944281231175904
Amal Chekili, Ivan Hernandez
Gender and ethnicity are increasingly studied topics within I-O psychology, helpful for understanding the composition of collectives, experiences of marginalized group members, and differences in outcomes between demographics and capturing diversity at higher levels. However, the absence of explicit, structured, demographic information online makes applying these research questions to Big Data sources challenging. We highlight how deep neural networks can be used to infer demographics based on people's names, which are commonly found online (e.g., social media profiles, employee pages, and membership rosters), using broad international data to train and evaluate the effectiveness of these models and find that validity coefficients meet minimum reliability thresholds at the individual level ( rgender  =  .91, rethnicity  =  .80) highlighting their ability to contextualize and facilitate Big Data research. Using empirical data extracted from databases, websites, and mobile apps, we highlight how these models can be applied to large organizational data sets by presenting illustrative demonstrations of research questions that incorporate the information provided by the model. To promote broader usage, we offer an online application to infer demographics from names without requiring advanced programming knowledge.
性别和种族是io心理学中越来越多的研究主题,有助于理解集体的组成,边缘化群体成员的经历,以及人口统计学结果的差异,并在更高层次上捕捉多样性。然而,由于缺乏明确的、结构化的、在线的人口统计信息,使得将这些研究问题应用于大数据源具有挑战性。我们强调如何使用深度神经网络来根据人们的姓名推断人口统计数据,这些数据通常在网上发现(例如,社交媒体简介,员工页面和会员名单),使用广泛的国际数据来训练和评估这些模型的有效性,并发现有效性系数满足个人层面的最小可靠性阈值(rgender =)。91,种族= .80),突出了他们在背景化和促进大数据研究方面的能力。利用从数据库、网站和移动应用程序中提取的经验数据,我们通过展示包含模型提供的信息的研究问题的说明性演示,强调了这些模型如何应用于大型组织数据集。为了促进更广泛的使用,我们提供了一个在线应用程序,可以从名字中推断人口统计数据,而不需要高级编程知识。
{"title":"Demographic Inference in the Digital Age: Using Neural Networks to Assess Gender and Ethnicity at Scale","authors":"Amal Chekili, Ivan Hernandez","doi":"10.1177/10944281231175904","DOIUrl":"https://doi.org/10.1177/10944281231175904","url":null,"abstract":"Gender and ethnicity are increasingly studied topics within I-O psychology, helpful for understanding the composition of collectives, experiences of marginalized group members, and differences in outcomes between demographics and capturing diversity at higher levels. However, the absence of explicit, structured, demographic information online makes applying these research questions to Big Data sources challenging. We highlight how deep neural networks can be used to infer demographics based on people's names, which are commonly found online (e.g., social media profiles, employee pages, and membership rosters), using broad international data to train and evaluate the effectiveness of these models and find that validity coefficients meet minimum reliability thresholds at the individual level ( rgender  =  .91, rethnicity  =  .80) highlighting their ability to contextualize and facilitate Big Data research. Using empirical data extracted from databases, websites, and mobile apps, we highlight how these models can be applied to large organizational data sets by presenting illustrative demonstrations of research questions that incorporate the information provided by the model. To promote broader usage, we offer an online application to infer demographics from names without requiring advanced programming knowledge.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46027719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature 元分析效应大小的异质性:对文献现状的评估
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-05-19 DOI: 10.1177/10944281231169942
S. Kepes, Wenhao Wang, J. Cortina
Heterogeneity refers to the variability in effect sizes across different samples and is one of the major criteria to judge the importance and advancement of a scientific area. To determine how studies in the organizational sciences address heterogeneity, we conduct two studies. In study 1, we examine how meta-analytic studies conduct heterogeneity assessments and report and interpret the obtained results. To do so, we coded heterogeneity-related information from meta-analytic studies published in five leading journals. We found that most meta-analytic studies report several heterogeneity statistics. At the same time, however, there tends to be a lack of detail and thoroughness in the interpretation of these statistics. In study 2, we review how primary studies report heterogeneity-related results and conclusions from meta-analyses. We found that the quality of the reporting of heterogeneity-related information in primary studies tends to be poor and unrelated to the detail and thoroughness with which meta-analytic studies report and interpret the statistics. Based on our findings, we discuss implications for practice and provide recommendations for how heterogeneity assessments should be conducted and communicated in future research.
异质性是指不同样本间效应大小的可变性,是判断一个科学领域重要性和先进性的主要标准之一。为了确定组织科学研究如何处理异质性,我们进行了两项研究。在研究1中,我们研究了元分析研究如何进行异质性评估,并报告和解释所获得的结果。为此,我们对发表在五种主要期刊上的荟萃分析研究中的异质性相关信息进行了编码。我们发现大多数荟萃分析研究报告了一些异质性统计数据。然而,与此同时,对这些统计数字的解释往往缺乏细节和彻底性。在研究2中,我们回顾了原始研究如何报告异质性相关的结果和荟萃分析的结论。我们发现,在初级研究中,报告异质性相关信息的质量往往较差,与元分析研究报告和解释统计数据的细节和彻底性无关。基于我们的研究结果,我们讨论了对实践的影响,并就异质性评估应如何在未来的研究中进行和交流提供了建议。
{"title":"Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature","authors":"S. Kepes, Wenhao Wang, J. Cortina","doi":"10.1177/10944281231169942","DOIUrl":"https://doi.org/10.1177/10944281231169942","url":null,"abstract":"Heterogeneity refers to the variability in effect sizes across different samples and is one of the major criteria to judge the importance and advancement of a scientific area. To determine how studies in the organizational sciences address heterogeneity, we conduct two studies. In study 1, we examine how meta-analytic studies conduct heterogeneity assessments and report and interpret the obtained results. To do so, we coded heterogeneity-related information from meta-analytic studies published in five leading journals. We found that most meta-analytic studies report several heterogeneity statistics. At the same time, however, there tends to be a lack of detail and thoroughness in the interpretation of these statistics. In study 2, we review how primary studies report heterogeneity-related results and conclusions from meta-analyses. We found that the quality of the reporting of heterogeneity-related information in primary studies tends to be poor and unrelated to the detail and thoroughness with which meta-analytic studies report and interpret the statistics. Based on our findings, we discuss implications for practice and provide recommendations for how heterogeneity assessments should be conducted and communicated in future research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43622205","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Assessing Common-Metric Effect Sizes to Refine Mediation Models 评估常用度量效应大小以改进中介模型
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-05-08 DOI: 10.1177/10944281231169943
Juan I. Sanchez, Chen Wang, A. Ponnapalli, Hock-Peng Sin, Le Xu, M. Lapeira, Mohan Song
Mediation analysis tests X → M → Y processes in which an independent variable ( X) exerts an indirect effect on a dependent variable ( Y) through its influence on an intervening or mediator variable ( M). A preponderance of mediation studies, however, focuses on determining solely whether mediation effects are statistically significant, instead of focusing on what the results tell us about potential theoretical refinements in the mediation model. We argue in favor of employing a set of three standardized effect sizes based on variance proportions that allow researchers to compare their results with those of other mediation studies employing similar combinations of X, M, and Y variables. These standardized effect sizes constitute a set of common metrics signaling potential gaps in a mediation model, and as such provide useful insights for the theoretical refinement of mediation models in organizational research. We illustrate the utility of comparing these common-metric effect sizes using the examples of abusive and transformational leadership effects on employee outcomes as transmitted by social exchange quality.
中介分析测试X→ M→ Y过程,其中自变量(X)通过对干预变量或中介变量(M)的影响对因变量(Y)施加间接影响。然而,大多数中介研究只关注于确定中介效果是否具有统计学意义,而不是关注结果告诉我们中介模型中潜在的理论改进。我们主张使用一组基于方差比例的三种标准化效应大小,使研究人员能够将他们的结果与使用X、M和Y变量类似组合的其他中介研究的结果进行比较。这些标准化的效应大小构成了一组共同的指标,表明中介模型中存在潜在的差距,因此为组织研究中中介模型的理论完善提供了有用的见解。我们通过社会交换质量传递的滥用和转型领导对员工结果的影响的例子,说明了比较这些常见度量效应大小的效用。
{"title":"Assessing Common-Metric Effect Sizes to Refine Mediation Models","authors":"Juan I. Sanchez, Chen Wang, A. Ponnapalli, Hock-Peng Sin, Le Xu, M. Lapeira, Mohan Song","doi":"10.1177/10944281231169943","DOIUrl":"https://doi.org/10.1177/10944281231169943","url":null,"abstract":"Mediation analysis tests X → M → Y processes in which an independent variable ( X) exerts an indirect effect on a dependent variable ( Y) through its influence on an intervening or mediator variable ( M). A preponderance of mediation studies, however, focuses on determining solely whether mediation effects are statistically significant, instead of focusing on what the results tell us about potential theoretical refinements in the mediation model. We argue in favor of employing a set of three standardized effect sizes based on variance proportions that allow researchers to compare their results with those of other mediation studies employing similar combinations of X, M, and Y variables. These standardized effect sizes constitute a set of common metrics signaling potential gaps in a mediation model, and as such provide useful insights for the theoretical refinement of mediation models in organizational research. We illustrate the utility of comparing these common-metric effect sizes using the examples of abusive and transformational leadership effects on employee outcomes as transmitted by social exchange quality.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48021064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Out of Shape: The Implications of (Extremely) Nonnormal Dependent Variables 变形:(极度)非正常因变量的含义
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-05-07 DOI: 10.1177/10944281231167839
S. Trevis Certo, Kristen Raney, Latifa Albader, John R. Busenbark
Organizational researchers have increasingly noted the problems associated with nonnormal dependent variable distributions. Most of this scholarship focuses on variables with positive values and lo...
组织研究人员越来越注意到与非正态因变量分布相关的问题。这些学术研究大多集中在具有正值和低值的变量上。
{"title":"Out of Shape: The Implications of (Extremely) Nonnormal Dependent Variables","authors":"S. Trevis Certo, Kristen Raney, Latifa Albader, John R. Busenbark","doi":"10.1177/10944281231167839","DOIUrl":"https://doi.org/10.1177/10944281231167839","url":null,"abstract":"Organizational researchers have increasingly noted the problems associated with nonnormal dependent variable distributions. Most of this scholarship focuses on variables with positive values and lo...","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50165318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Team Composition Revisited: Expanding the Team Member Attribute Alignment Approach to Consider Patterns of More Than Two Attributes 重新审视团队组成:扩展团队成员属性对齐方法,以考虑两个以上属性的模式
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-05-03 DOI: 10.1177/10944281231166656
Kyle J. Emich, M. McCourt, Li Lu, Amanda J. Ferguson, R. Peterson
The attribute alignment approach to team composition allows researchers to assess variation in team member attributes, which occurs simultaneously within and across individual team members. This approach facilitates the development of theory testing the proposition that individual members are themselves complex systems comprised of multiple attributes and that the configuration of those attributes affects team-level processes and outcomes. Here, we expand this approach, originally developed for two attributes, by describing three ways researchers may capture the alignment of three or more team member attributes: (a) a geometric approach, (b) a physical approach accentuating ideal alignment, and (c) an algebraic approach accentuating the direction (as opposed to magnitude) of alignment. We also provide examples of the research questions each could answer and compare the methods empirically using a synthetic dataset assessing 100 teams of three to seven members across four attributes. Then, we provide a practical guide to selecting an appropriate method when considering team-member attribute patterns by answering several common questions regarding applying attribute alignment. Finally, we provide code ( https://github.com/kjem514/Attribute-Alignment-Code ) and apply this approach to a field data set in our appendices.
团队组成的属性比对方法使研究人员能够评估团队成员属性的变化,这种变化同时发生在单个团队成员内部和之间。这种方法有助于理论的发展,测试个人成员本身就是由多个属性组成的复杂系统,这些属性的配置会影响团队级别的过程和结果。在这里,我们扩展了这种最初针对两个属性开发的方法,通过描述研究人员可以捕捉三种或更多团队成员属性对齐的三种方式:(a)几何方法,(b)强调理想对齐的物理方法,以及(c)强调对齐方向(而不是大小)的代数方法。我们还提供了每个人都可以回答的研究问题的例子,并使用合成数据集对四个属性的100个由三到七名成员组成的团队进行了实证比较。然后,我们通过回答关于应用属性对齐的几个常见问题,提供了一个在考虑团队成员属性模式时选择适当方法的实用指南。最后,我们提供代码(https://github.com/kjem514/Attribute-Alignment-Code),并将此方法应用于我们附录中的现场数据集。
{"title":"Team Composition Revisited: Expanding the Team Member Attribute Alignment Approach to Consider Patterns of More Than Two Attributes","authors":"Kyle J. Emich, M. McCourt, Li Lu, Amanda J. Ferguson, R. Peterson","doi":"10.1177/10944281231166656","DOIUrl":"https://doi.org/10.1177/10944281231166656","url":null,"abstract":"The attribute alignment approach to team composition allows researchers to assess variation in team member attributes, which occurs simultaneously within and across individual team members. This approach facilitates the development of theory testing the proposition that individual members are themselves complex systems comprised of multiple attributes and that the configuration of those attributes affects team-level processes and outcomes. Here, we expand this approach, originally developed for two attributes, by describing three ways researchers may capture the alignment of three or more team member attributes: (a) a geometric approach, (b) a physical approach accentuating ideal alignment, and (c) an algebraic approach accentuating the direction (as opposed to magnitude) of alignment. We also provide examples of the research questions each could answer and compare the methods empirically using a synthetic dataset assessing 100 teams of three to seven members across four attributes. Then, we provide a practical guide to selecting an appropriate method when considering team-member attribute patterns by answering several common questions regarding applying attribute alignment. Finally, we provide code ( https://github.com/kjem514/Attribute-Alignment-Code ) and apply this approach to a field data set in our appendices.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43887552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Macro-iterativity: A Qualitative Multi-arc Design for Studying Complex Issues and Big Questions 宏观迭代性:研究复杂问题和大问题的定性多弧设计
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-04-17 DOI: 10.1177/10944281231166649
Christina Hoon, Alina M. Baluch
The impact and relevance of our discipline's research is determined by its ability to engage the big questions of the grand challenges we face today. Our central argument is that we need innovative methods that engage large-scope phenomena, not least because these phenomena benefit from going beyond individual study design. We introduce the concept of macro-iterativity which involves multiple iterations that move between, and link across, a set of research cycles. We offer a multi-arc research design that comprises the discovery arc and extension arc and three extension logics through which scholars can combine these arcs of inquiry in a coherent way. Based on this research design, we develop a roadmap that guides scholars through the four steps of how to engage in multi-arc research along with the main techniques and outputs. We argue that a multi-arc design supports the move toward more generative theorizing that is required for researching problems dealing with the complex issues and big questions of our time.
我们学科研究的影响力和相关性取决于它处理我们今天面临的重大挑战中的重大问题的能力。我们的核心论点是,我们需要创新的方法来处理大范围的现象,尤其是因为这些现象受益于超越个人学习设计。我们引入了宏观迭代性的概念,它涉及在一组研究周期之间移动和链接的多次迭代。我们提供了一个多弧研究设计,包括发现弧和扩展弧,以及三个扩展逻辑,通过这些逻辑,学者可以以连贯的方式将这些研究弧结合起来。基于这一研究设计,我们制定了一个路线图,指导学者完成如何进行多弧研究的四个步骤以及主要技术和产出。我们认为,多弧设计支持向更具生成性的理论化迈进,这是研究处理我们这个时代的复杂问题和重大问题所必需的。
{"title":"Macro-iterativity: A Qualitative Multi-arc Design for Studying Complex Issues and Big Questions","authors":"Christina Hoon, Alina M. Baluch","doi":"10.1177/10944281231166649","DOIUrl":"https://doi.org/10.1177/10944281231166649","url":null,"abstract":"The impact and relevance of our discipline's research is determined by its ability to engage the big questions of the grand challenges we face today. Our central argument is that we need innovative methods that engage large-scope phenomena, not least because these phenomena benefit from going beyond individual study design. We introduce the concept of macro-iterativity which involves multiple iterations that move between, and link across, a set of research cycles. We offer a multi-arc research design that comprises the discovery arc and extension arc and three extension logics through which scholars can combine these arcs of inquiry in a coherent way. Based on this research design, we develop a roadmap that guides scholars through the four steps of how to engage in multi-arc research along with the main techniques and outputs. We argue that a multi-arc design supports the move toward more generative theorizing that is required for researching problems dealing with the complex issues and big questions of our time.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41372087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Publication Notice 公告
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-03-31 DOI: 10.1177/10944281231155392
{"title":"Publication Notice","authors":"","doi":"10.1177/10944281231155392","DOIUrl":"https://doi.org/10.1177/10944281231155392","url":null,"abstract":"","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42379322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
“Transforming” Personality Scale Development: Illustrating the Potential of State-of-the-Art Natural Language Processing “转换”人格量表的发展:说明最先进的自然语言处理的潜力
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-03-06 DOI: 10.1177/10944281231155771
Shea Fyffe, Philseok Lee, Seth A. Kaplan
Natural language processing (NLP) techniques are becoming increasingly popular in industrial and organizational psychology. One promising area for NLP-based applications is scale development; yet, while many possibilities exist, so far these applications have been restricted—mainly focusing on automated item generation. The current research expands this potential by illustrating an NLP-based approach to content analysis, which manually categorizes scale items by their measured constructs. In NLP, content analysis is performed as a text classification task whereby a model is trained to automatically assign scale items to the construct that they measure. Here, we present an approach to text classification—using state-of-the-art transformer models—that builds upon past approaches. We begin by introducing transformer models and their advantages over alternative methods. Next, we illustrate how to train a transformer to content analyze Big Five personality items. Then, we compare the models trained to human raters, finding that transformer models outperform human raters and several alternative models. Finally, we present practical considerations, limitations, and future research directions.
自然语言处理(NLP)技术在工业心理学和组织心理学中越来越受欢迎。基于nlp的应用程序的一个有前途的领域是规模开发;然而,尽管存在许多可能性,但到目前为止,这些应用程序还受到限制——主要集中在自动生成项目上。目前的研究通过说明基于nlp的内容分析方法扩展了这一潜力,该方法通过测量的结构手动对量表项目进行分类。在NLP中,内容分析作为文本分类任务执行,其中模型被训练以自动将刻度项分配给它们测量的构造。在这里,我们提出了一种基于过去方法的文本分类方法——使用最先进的变压器模型。我们首先介绍变压器模型及其相对于替代方法的优势。接下来,我们将说明如何训练一个转换器来分析五大人格项目。然后,我们将训练的模型与人类评级器进行比较,发现变压器模型优于人类评级器和几个替代模型。最后,提出了现实考虑、局限性和未来的研究方向。
{"title":"“Transforming” Personality Scale Development: Illustrating the Potential of State-of-the-Art Natural Language Processing","authors":"Shea Fyffe, Philseok Lee, Seth A. Kaplan","doi":"10.1177/10944281231155771","DOIUrl":"https://doi.org/10.1177/10944281231155771","url":null,"abstract":"Natural language processing (NLP) techniques are becoming increasingly popular in industrial and organizational psychology. One promising area for NLP-based applications is scale development; yet, while many possibilities exist, so far these applications have been restricted—mainly focusing on automated item generation. The current research expands this potential by illustrating an NLP-based approach to content analysis, which manually categorizes scale items by their measured constructs. In NLP, content analysis is performed as a text classification task whereby a model is trained to automatically assign scale items to the construct that they measure. Here, we present an approach to text classification—using state-of-the-art transformer models—that builds upon past approaches. We begin by introducing transformer models and their advantages over alternative methods. Next, we illustrate how to train a transformer to content analyze Big Five personality items. Then, we compare the models trained to human raters, finding that transformer models outperform human raters and several alternative models. Finally, we present practical considerations, limitations, and future research directions.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47097386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Supervised Construct Scoring to Reduce Personality Assessment Length: A Field Study and Introduction to the Short 10 监督构式计分法减少人格评估长度:一项实地研究及简短的介绍
IF 9.5 2区 管理学 Q1 Business, Management and Accounting Pub Date : 2023-01-03 DOI: 10.1177/10944281221145694
Andrew B. Speer, James Perrotta, R. Jacobs
Personality assessments help identify qualified job applicants when making hiring decisions and are used broadly in the organizational sciences. However, many existing personality measures are quite lengthy, and companies and researchers frequently seek ways to shorten personality scales. The current research investigated the effectiveness of a new scale-shortening method called supervised construct scoring (SCS), testing the efficacy of this method across two applied samples. Using a combination of machine learning with content validity considerations, we show that multidimensional personality scales can be significantly shortened while maintaining reliability and validity, and especially when compared to traditional shortening methods. In Study 1, we shortened a 100-item personality assessment of DeYoung et al.'s 10 facets, producing a scale 26% the original length. SCS scores exhibited strong evidence of reliability, convergence with full scale scores, and criterion-related validity. This measure, labeled the Short 10, is made freely available. In Study 2, we applied SCS to shorten an operational police personality assessment. By using SCS, we reduced test length to 25% of the original length while maintaining similar levels of reliability and criterion-related validity when predicting job performance ratings.
性格评估有助于在做出招聘决定时确定合格的求职者,并在组织科学中得到广泛应用。然而,许多现有的人格测试都相当长,公司和研究人员经常寻求缩短人格量表的方法。目前的研究调查了一种名为监督结构评分(SCS)的新型量表缩短方法的有效性,测试了该方法在两个应用样本中的有效性。结合机器学习和内容效度的考虑,我们表明多维人格量表可以在保持信度和效度的同时显着缩短,特别是与传统的缩短方法相比。在研究1中,我们缩短了DeYoung等人的10个方面的100项人格评估,产生了原始长度的26%。SCS评分表现出强有力的可靠性、与全量表评分的收敛性和标准相关的效度。这个指标被称为Short 10,是免费提供的。在研究2中,我们应用SCS来缩短一个行动警察的人格评估。通过使用SCS,我们将测试长度减少到原始长度的25%,同时在预测工作绩效评级时保持相似的信度和标准相关效度水平。
{"title":"Supervised Construct Scoring to Reduce Personality Assessment Length: A Field Study and Introduction to the Short 10","authors":"Andrew B. Speer, James Perrotta, R. Jacobs","doi":"10.1177/10944281221145694","DOIUrl":"https://doi.org/10.1177/10944281221145694","url":null,"abstract":"Personality assessments help identify qualified job applicants when making hiring decisions and are used broadly in the organizational sciences. However, many existing personality measures are quite lengthy, and companies and researchers frequently seek ways to shorten personality scales. The current research investigated the effectiveness of a new scale-shortening method called supervised construct scoring (SCS), testing the efficacy of this method across two applied samples. Using a combination of machine learning with content validity considerations, we show that multidimensional personality scales can be significantly shortened while maintaining reliability and validity, and especially when compared to traditional shortening methods. In Study 1, we shortened a 100-item personality assessment of DeYoung et al.'s 10 facets, producing a scale 26% the original length. SCS scores exhibited strong evidence of reliability, convergence with full scale scores, and criterion-related validity. This measure, labeled the Short 10, is made freely available. In Study 2, we applied SCS to shorten an operational police personality assessment. By using SCS, we reduced test length to 25% of the original length while maintaining similar levels of reliability and criterion-related validity when predicting job performance ratings.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":null,"pages":null},"PeriodicalIF":9.5,"publicationDate":"2023-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48250283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Organizational Research Methods
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1