Pub Date : 2023-09-28DOI: 10.1177/10944281231202740
Ze Zhu, John A. Aitken, Reeshad S. Dalal, Seth A. Kaplan
Organizational researchers are now making widespread use of ecological momentary assessments but have not yet taken the logical next step to ecological momentary interventions, also called Just-in-Time Adaptive Interventions (JITAIs). JITAIs have the potential to test within-person causal theories and maximize practical benefits to participants through two developmental phases: The microrandomized trial and the randomized controlled trial, respectively. In the microrandomized trial design, within-person randomization and experimental manipulation maximize internal validity at the within-person level. In the randomized controlled trial design, interventions are delivered in a timely and ecological manner while avoiding unnecessary and ill-timed interventions that potentially increase participant fatigue and noncompliance. Despite these potential advantages, the development and implementation of JITAIs require consideration of many conceptual and methodological factors. Given the benefits of JITAIs, but also the various considerations involved in using them, this review introduces organizational behavior and human resources researchers to JITAIs, provides guidelines for JITAI design, development, and evaluation, and describes the extensive potential of JITAIs in organizational behavior and human resources research.
{"title":"The Promise of Just-in-Time Adaptive Interventions for Organizational Scholarship and Practice: Conceptual Development and Research Agenda","authors":"Ze Zhu, John A. Aitken, Reeshad S. Dalal, Seth A. Kaplan","doi":"10.1177/10944281231202740","DOIUrl":"https://doi.org/10.1177/10944281231202740","url":null,"abstract":"Organizational researchers are now making widespread use of ecological momentary assessments but have not yet taken the logical next step to ecological momentary interventions, also called Just-in-Time Adaptive Interventions (JITAIs). JITAIs have the potential to test within-person causal theories and maximize practical benefits to participants through two developmental phases: The microrandomized trial and the randomized controlled trial, respectively. In the microrandomized trial design, within-person randomization and experimental manipulation maximize internal validity at the within-person level. In the randomized controlled trial design, interventions are delivered in a timely and ecological manner while avoiding unnecessary and ill-timed interventions that potentially increase participant fatigue and noncompliance. Despite these potential advantages, the development and implementation of JITAIs require consideration of many conceptual and methodological factors. Given the benefits of JITAIs, but also the various considerations involved in using them, this review introduces organizational behavior and human resources researchers to JITAIs, provides guidelines for JITAI design, development, and evaluation, and describes the extensive potential of JITAIs in organizational behavior and human resources research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135344361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-28DOI: 10.1177/10944281231195788
Lisa Schurer Lambert, Tine Köhler
{"title":"Celebrating 25 Years of ORM","authors":"Lisa Schurer Lambert, Tine Köhler","doi":"10.1177/10944281231195788","DOIUrl":"https://doi.org/10.1177/10944281231195788","url":null,"abstract":"","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135385379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-11DOI: 10.1177/10944281231195704
Jason Kiley, Aaron McKenny, Jeremy Short, Anne Smith
{"title":"Call for Papers for a Feature Topic: Having A Way with Words: Innovations and Improvements in Text Analysis Methods","authors":"Jason Kiley, Aaron McKenny, Jeremy Short, Anne Smith","doi":"10.1177/10944281231195704","DOIUrl":"https://doi.org/10.1177/10944281231195704","url":null,"abstract":"","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"2015 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136024452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-30DOI: 10.1177/10944281231195298
C. Kam, S. Cheung
Using constrained factor mixture models (FMM) for careless response identification is still in its infancy. Existing models have overly restrictive statistical assumptions that do not identify all types of careless respondents. The current paper presents a novel constrained FMM model with more reasonable assumptions that capture both longstring and random careless respondents. We provide a comprehensive comparison of the statistical assumptions between the proposed model and two previous constrained models. The proposed model was evaluated using both real data ( N = 1,455) and statistical simulation. The results showed that the model had a superior fit, stronger convergent validity with other indicators of careless responding, more accurate parameter recovery and more accurate identification of careless respondents when compared to its predecessors. The proposed model does not require additional data collection effort, and thus researchers can routinely use it to control careless responses. We provide user-friendly syntax with detailed explanations online to facilitate its use.
{"title":"A Constrained Factor Mixture Model for Detecting Careless Responses that is Simple to Implement","authors":"C. Kam, S. Cheung","doi":"10.1177/10944281231195298","DOIUrl":"https://doi.org/10.1177/10944281231195298","url":null,"abstract":"Using constrained factor mixture models (FMM) for careless response identification is still in its infancy. Existing models have overly restrictive statistical assumptions that do not identify all types of careless respondents. The current paper presents a novel constrained FMM model with more reasonable assumptions that capture both longstring and random careless respondents. We provide a comprehensive comparison of the statistical assumptions between the proposed model and two previous constrained models. The proposed model was evaluated using both real data ( N = 1,455) and statistical simulation. The results showed that the model had a superior fit, stronger convergent validity with other indicators of careless responding, more accurate parameter recovery and more accurate identification of careless respondents when compared to its predecessors. The proposed model does not require additional data collection effort, and thus researchers can routinely use it to control careless responses. We provide user-friendly syntax with detailed explanations online to facilitate its use.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47298575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-07-01DOI: 10.1177/10944281211058469
Federica Bianchi, Alessandro Lomi
Relational event models expand the analytical possibilities of existing statistical models for interorganizational networks by: (i) making efficient use of information contained in the sequential ordering of observed events connecting sending and receiving units; (ii) accounting for the intensity of the relation between exchange partners, and (iii) distinguishing between short- and long-term network effects. We introduce a recently developed relational event model (REM) for the analysis of continuously observed interorganizational exchange relations. The combination of efficient sampling algorithms and sender-based stratification makes the models that we present particularly useful for the analysis of very large samples of relational event data generated by interaction among heterogeneous actors. We demonstrate the empirical value of event-oriented network models in two different settings for interorganizational exchange relations-that is, high-frequency overnight transactions among European banks and patient-sharing relations within a community of Italian hospitals. We focus on patterns of direct and generalized reciprocity while accounting for more complex forms of dependence present in the data. Empirical results suggest that distinguishing between degree- and intensity-based network effects, and between short- and long-term effects is crucial to our understanding of the dynamics of interorganizational dependence and exchange relations. We discuss the general implications of these results for the analysis of social interaction data routinely collected in organizational research to examine the evolutionary dynamics of social networks within and between organizations.
{"title":"From Ties to Events in the Analysis of Interorganizational Exchange Relations.","authors":"Federica Bianchi, Alessandro Lomi","doi":"10.1177/10944281211058469","DOIUrl":"https://doi.org/10.1177/10944281211058469","url":null,"abstract":"<p><p>Relational event models expand the analytical possibilities of existing statistical models for interorganizational networks by: (i) making efficient use of information contained in the sequential ordering of observed events connecting sending and receiving units; (ii) accounting for the intensity of the relation between exchange partners, and (iii) distinguishing between short- and long-term network effects. We introduce a recently developed relational event model (REM) for the analysis of continuously observed interorganizational exchange relations. The combination of efficient sampling algorithms and sender-based stratification makes the models that we present particularly useful for the analysis of very large samples of relational event data generated by interaction among heterogeneous actors. We demonstrate the empirical value of event-oriented network models in two different settings for interorganizational exchange relations-that is, high-frequency overnight transactions among European banks and patient-sharing relations within a community of Italian hospitals. We focus on patterns of direct and generalized reciprocity while accounting for more complex forms of dependence present in the data. Empirical results suggest that distinguishing between degree- and intensity-based network effects, and between short- and long-term effects is crucial to our understanding of the dynamics of interorganizational dependence and exchange relations. We discuss the general implications of these results for the analysis of social interaction data routinely collected in organizational research to examine the evolutionary dynamics of social networks within and between organizations.</p>","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"26 3","pages":"524-565"},"PeriodicalIF":9.5,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/40/59/10.1177_10944281211058469.PMC10278390.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10351480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-06-27DOI: 10.1177/10944281231181642
Siwei Peng, K. Man, B. Veldkamp, Yan Cai, Dongbo Tu
For various reasons, respondents to forced-choice assessments (typically used for noncognitive psychological constructs) may respond randomly to individual items due to indecision or globally due to disengagement. Thus, random responding is a complex source of measurement bias and threatens the reliability of forced-choice assessments, which are essential in high-stakes organizational testing scenarios, such as hiring decisions. The traditional measurement models rely heavily on nonrandom, construct-relevant responses to yield accurate parameter estimates. When survey data contain many random responses, fitting traditional models may deliver biased results, which could attenuate measurement reliability. This study presents a new forced-choice measure-based mixture item response theory model (called M-TCIR) for simultaneously modeling normal and random responses (distinguishing completely and incompletely random). The feasibility of the M-TCIR was investigated via two Monte Carlo simulation studies. In addition, one empirical dataset was analyzed to illustrate the applicability of the M-TCIR in practice. The results revealed that most model parameters were adequately recovered, and the M-TCIR was a viable alternative to model both aberrant and normal responses with high efficiency.
{"title":"A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research","authors":"Siwei Peng, K. Man, B. Veldkamp, Yan Cai, Dongbo Tu","doi":"10.1177/10944281231181642","DOIUrl":"https://doi.org/10.1177/10944281231181642","url":null,"abstract":"For various reasons, respondents to forced-choice assessments (typically used for noncognitive psychological constructs) may respond randomly to individual items due to indecision or globally due to disengagement. Thus, random responding is a complex source of measurement bias and threatens the reliability of forced-choice assessments, which are essential in high-stakes organizational testing scenarios, such as hiring decisions. The traditional measurement models rely heavily on nonrandom, construct-relevant responses to yield accurate parameter estimates. When survey data contain many random responses, fitting traditional models may deliver biased results, which could attenuate measurement reliability. This study presents a new forced-choice measure-based mixture item response theory model (called M-TCIR) for simultaneously modeling normal and random responses (distinguishing completely and incompletely random). The feasibility of the M-TCIR was investigated via two Monte Carlo simulation studies. In addition, one empirical dataset was analyzed to illustrate the applicability of the M-TCIR in practice. The results revealed that most model parameters were adequately recovered, and the M-TCIR was a viable alternative to model both aberrant and normal responses with high efficiency.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41961861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-06-14DOI: 10.1177/10944281231175904
Amal Chekili, Ivan Hernandez
Gender and ethnicity are increasingly studied topics within I-O psychology, helpful for understanding the composition of collectives, experiences of marginalized group members, and differences in outcomes between demographics and capturing diversity at higher levels. However, the absence of explicit, structured, demographic information online makes applying these research questions to Big Data sources challenging. We highlight how deep neural networks can be used to infer demographics based on people's names, which are commonly found online (e.g., social media profiles, employee pages, and membership rosters), using broad international data to train and evaluate the effectiveness of these models and find that validity coefficients meet minimum reliability thresholds at the individual level ( rgender = .91, rethnicity = .80) highlighting their ability to contextualize and facilitate Big Data research. Using empirical data extracted from databases, websites, and mobile apps, we highlight how these models can be applied to large organizational data sets by presenting illustrative demonstrations of research questions that incorporate the information provided by the model. To promote broader usage, we offer an online application to infer demographics from names without requiring advanced programming knowledge.
{"title":"Demographic Inference in the Digital Age: Using Neural Networks to Assess Gender and Ethnicity at Scale","authors":"Amal Chekili, Ivan Hernandez","doi":"10.1177/10944281231175904","DOIUrl":"https://doi.org/10.1177/10944281231175904","url":null,"abstract":"Gender and ethnicity are increasingly studied topics within I-O psychology, helpful for understanding the composition of collectives, experiences of marginalized group members, and differences in outcomes between demographics and capturing diversity at higher levels. However, the absence of explicit, structured, demographic information online makes applying these research questions to Big Data sources challenging. We highlight how deep neural networks can be used to infer demographics based on people's names, which are commonly found online (e.g., social media profiles, employee pages, and membership rosters), using broad international data to train and evaluate the effectiveness of these models and find that validity coefficients meet minimum reliability thresholds at the individual level ( rgender = .91, rethnicity = .80) highlighting their ability to contextualize and facilitate Big Data research. Using empirical data extracted from databases, websites, and mobile apps, we highlight how these models can be applied to large organizational data sets by presenting illustrative demonstrations of research questions that incorporate the information provided by the model. To promote broader usage, we offer an online application to infer demographics from names without requiring advanced programming knowledge.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46027719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-19DOI: 10.1177/10944281231169942
S. Kepes, Wenhao Wang, J. Cortina
Heterogeneity refers to the variability in effect sizes across different samples and is one of the major criteria to judge the importance and advancement of a scientific area. To determine how studies in the organizational sciences address heterogeneity, we conduct two studies. In study 1, we examine how meta-analytic studies conduct heterogeneity assessments and report and interpret the obtained results. To do so, we coded heterogeneity-related information from meta-analytic studies published in five leading journals. We found that most meta-analytic studies report several heterogeneity statistics. At the same time, however, there tends to be a lack of detail and thoroughness in the interpretation of these statistics. In study 2, we review how primary studies report heterogeneity-related results and conclusions from meta-analyses. We found that the quality of the reporting of heterogeneity-related information in primary studies tends to be poor and unrelated to the detail and thoroughness with which meta-analytic studies report and interpret the statistics. Based on our findings, we discuss implications for practice and provide recommendations for how heterogeneity assessments should be conducted and communicated in future research.
{"title":"Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature","authors":"S. Kepes, Wenhao Wang, J. Cortina","doi":"10.1177/10944281231169942","DOIUrl":"https://doi.org/10.1177/10944281231169942","url":null,"abstract":"Heterogeneity refers to the variability in effect sizes across different samples and is one of the major criteria to judge the importance and advancement of a scientific area. To determine how studies in the organizational sciences address heterogeneity, we conduct two studies. In study 1, we examine how meta-analytic studies conduct heterogeneity assessments and report and interpret the obtained results. To do so, we coded heterogeneity-related information from meta-analytic studies published in five leading journals. We found that most meta-analytic studies report several heterogeneity statistics. At the same time, however, there tends to be a lack of detail and thoroughness in the interpretation of these statistics. In study 2, we review how primary studies report heterogeneity-related results and conclusions from meta-analyses. We found that the quality of the reporting of heterogeneity-related information in primary studies tends to be poor and unrelated to the detail and thoroughness with which meta-analytic studies report and interpret the statistics. Based on our findings, we discuss implications for practice and provide recommendations for how heterogeneity assessments should be conducted and communicated in future research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43622205","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-08DOI: 10.1177/10944281231169943
Juan I. Sanchez, Chen Wang, A. Ponnapalli, Hock-Peng Sin, Le Xu, M. Lapeira, Mohan Song
Mediation analysis tests X → M → Y processes in which an independent variable ( X) exerts an indirect effect on a dependent variable ( Y) through its influence on an intervening or mediator variable ( M). A preponderance of mediation studies, however, focuses on determining solely whether mediation effects are statistically significant, instead of focusing on what the results tell us about potential theoretical refinements in the mediation model. We argue in favor of employing a set of three standardized effect sizes based on variance proportions that allow researchers to compare their results with those of other mediation studies employing similar combinations of X, M, and Y variables. These standardized effect sizes constitute a set of common metrics signaling potential gaps in a mediation model, and as such provide useful insights for the theoretical refinement of mediation models in organizational research. We illustrate the utility of comparing these common-metric effect sizes using the examples of abusive and transformational leadership effects on employee outcomes as transmitted by social exchange quality.
{"title":"Assessing Common-Metric Effect Sizes to Refine Mediation Models","authors":"Juan I. Sanchez, Chen Wang, A. Ponnapalli, Hock-Peng Sin, Le Xu, M. Lapeira, Mohan Song","doi":"10.1177/10944281231169943","DOIUrl":"https://doi.org/10.1177/10944281231169943","url":null,"abstract":"Mediation analysis tests X → M → Y processes in which an independent variable ( X) exerts an indirect effect on a dependent variable ( Y) through its influence on an intervening or mediator variable ( M). A preponderance of mediation studies, however, focuses on determining solely whether mediation effects are statistically significant, instead of focusing on what the results tell us about potential theoretical refinements in the mediation model. We argue in favor of employing a set of three standardized effect sizes based on variance proportions that allow researchers to compare their results with those of other mediation studies employing similar combinations of X, M, and Y variables. These standardized effect sizes constitute a set of common metrics signaling potential gaps in a mediation model, and as such provide useful insights for the theoretical refinement of mediation models in organizational research. We illustrate the utility of comparing these common-metric effect sizes using the examples of abusive and transformational leadership effects on employee outcomes as transmitted by social exchange quality.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48021064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-07DOI: 10.1177/10944281231167839
S. Trevis Certo, Kristen Raney, Latifa Albader, John R. Busenbark
Organizational researchers have increasingly noted the problems associated with nonnormal dependent variable distributions. Most of this scholarship focuses on variables with positive values and lo...
{"title":"Out of Shape: The Implications of (Extremely) Nonnormal Dependent Variables","authors":"S. Trevis Certo, Kristen Raney, Latifa Albader, John R. Busenbark","doi":"10.1177/10944281231167839","DOIUrl":"https://doi.org/10.1177/10944281231167839","url":null,"abstract":"Organizational researchers have increasingly noted the problems associated with nonnormal dependent variable distributions. Most of this scholarship focuses on variables with positive values and lo...","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"109 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50165318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}