Pub Date : 2021-12-02DOI: 10.1177/10944281211046312
Jessica Villiger, Simone A. Schweiger, Artur Baldauf
This article contributes to the practice of coding in meta-analyses by offering direction and advice for experienced and novice meta-analysts on the “how” of coding. The coding process, the invisible architecture of any meta-analysis, has received comparably little attention in methodological resources, leaving the research community with insufficient guidance on “how” it should be rigorously planned (i.e., cohere with the research objective), conducted (i.e., make reliable and valid coding decisions), and reported (i.e., in a sufficiently transparent manner for readers to comprehend the authors’ decision-making). A lack of rigor in these areas can lead to erroneous results, which is problematic for entire research communities who build their future knowledge upon meta-analyses. Along four steps, the guidelines presented here elucidate “how” the coding process can be performed in a coherent, efficient, and credible manner that enables connectivity with future research, thereby enhancing the reliability and validity of meta-analytic findings. Our recommendations also support editors and reviewers in advising authors on how to improve the rigor of their coding and ultimately establish higher quality standards in meta-analytic research.
{"title":"Making the Invisible Visible: Guidelines for the Coding Process in Meta-Analyses","authors":"Jessica Villiger, Simone A. Schweiger, Artur Baldauf","doi":"10.1177/10944281211046312","DOIUrl":"https://doi.org/10.1177/10944281211046312","url":null,"abstract":"This article contributes to the practice of coding in meta-analyses by offering direction and advice for experienced and novice meta-analysts on the “how” of coding. The coding process, the invisible architecture of any meta-analysis, has received comparably little attention in methodological resources, leaving the research community with insufficient guidance on “how” it should be rigorously planned (i.e., cohere with the research objective), conducted (i.e., make reliable and valid coding decisions), and reported (i.e., in a sufficiently transparent manner for readers to comprehend the authors’ decision-making). A lack of rigor in these areas can lead to erroneous results, which is problematic for entire research communities who build their future knowledge upon meta-analyses. Along four steps, the guidelines presented here elucidate “how” the coding process can be performed in a coherent, efficient, and credible manner that enables connectivity with future research, thereby enhancing the reliability and validity of meta-analytic findings. Our recommendations also support editors and reviewers in advising authors on how to improve the rigor of their coding and ultimately establish higher quality standards in meta-analytic research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"716 - 740"},"PeriodicalIF":9.5,"publicationDate":"2021-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47471672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-29DOI: 10.1177/10944281211043733
Ozlem Ozkok, Manuel J Vaulont, M. Zyphur, Zhen Zhang, Kristopher J Preacher, Peter Koval, Yixia Zheng
Researchers often combine longitudinal panel data analysis with tests of interactions (i.e., moderation). A popular example is the cross-lagged panel model (CLPM). However, interaction tests in CLPMs and related models require caution because stable (i.e., between-level, B) and dynamic (i.e., within-level, W) sources of variation are present in longitudinal data, which can conflate estimates of interaction effects. We address this by integrating literature on CLPMs, multilevel moderation, and latent interactions. Distinguishing stable B and dynamic W parts, we describe three types of interactions that are of interest to researchers: 1) purely dynamic or WxW; 2) cross-level or BxW; and 3) purely stable or BxB. We demonstrate estimating latent interaction effects in a CLPM using a Bayesian SEM in Mplus to apply relationships among work-family conflict and job satisfaction, using gender as a stable B variable. We support our approach via simulations, demonstrating that our proposed CLPM approach is superior to a traditional CLPMs that conflate B and W sources of variation. We describe higher-order nonlinearities as a possible extension, and we discuss limitations and future research directions.
{"title":"Interaction Effects in Cross-Lagged Panel Models: SEM with Latent Interactions Applied to Work-Family Conflict, Job Satisfaction, and Gender","authors":"Ozlem Ozkok, Manuel J Vaulont, M. Zyphur, Zhen Zhang, Kristopher J Preacher, Peter Koval, Yixia Zheng","doi":"10.1177/10944281211043733","DOIUrl":"https://doi.org/10.1177/10944281211043733","url":null,"abstract":"Researchers often combine longitudinal panel data analysis with tests of interactions (i.e., moderation). A popular example is the cross-lagged panel model (CLPM). However, interaction tests in CLPMs and related models require caution because stable (i.e., between-level, B) and dynamic (i.e., within-level, W) sources of variation are present in longitudinal data, which can conflate estimates of interaction effects. We address this by integrating literature on CLPMs, multilevel moderation, and latent interactions. Distinguishing stable B and dynamic W parts, we describe three types of interactions that are of interest to researchers: 1) purely dynamic or WxW; 2) cross-level or BxW; and 3) purely stable or BxB. We demonstrate estimating latent interaction effects in a CLPM using a Bayesian SEM in Mplus to apply relationships among work-family conflict and job satisfaction, using gender as a stable B variable. We support our approach via simulations, demonstrating that our proposed CLPM approach is superior to a traditional CLPMs that conflate B and W sources of variation. We describe higher-order nonlinearities as a possible extension, and we discuss limitations and future research directions.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"673 - 715"},"PeriodicalIF":9.5,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48057256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-13DOI: 10.1177/10944281211058903
T. Köhler, L. Lambert
We are honored to be the next co-Editors of ORM. Under the previous editorial teams, led by Larry Williams, Herman Aguinis, Bob Vandenberg, José Cortina, James LeBreton, and Paul Bliese, ORM has been succeeding by every available metric. ORM is widely recognized as the premier outlet for methodological scholarship in the organizational sciences, and this success is due to the collaboration between past Editors, Editorial teams, and Sage. It is not possible to overstate the contributions of the past Editors, and we are excited to take over leadership of this well-established journal. We especially want to credit Paul Bliese for making the handover process an incredibly smooth one. He promised we can reach out to him anytime. Thank you, Paul. We have your phone number on speed dial. Going forward, we are going to implement a few changes to ORM’s editorship structure and increase ORM’s visibility and reach in different research communities. In this editorial, we want to provide a small preview of what we have planned.
{"title":"Inaugural Editorial","authors":"T. Köhler, L. Lambert","doi":"10.1177/10944281211058903","DOIUrl":"https://doi.org/10.1177/10944281211058903","url":null,"abstract":"We are honored to be the next co-Editors of ORM. Under the previous editorial teams, led by Larry Williams, Herman Aguinis, Bob Vandenberg, José Cortina, James LeBreton, and Paul Bliese, ORM has been succeeding by every available metric. ORM is widely recognized as the premier outlet for methodological scholarship in the organizational sciences, and this success is due to the collaboration between past Editors, Editorial teams, and Sage. It is not possible to overstate the contributions of the past Editors, and we are excited to take over leadership of this well-established journal. We especially want to credit Paul Bliese for making the handover process an incredibly smooth one. He promised we can reach out to him anytime. Thank you, Paul. We have your phone number on speed dial. Going forward, we are going to implement a few changes to ORM’s editorship structure and increase ORM’s visibility and reach in different research communities. In this editorial, we want to provide a small preview of what we have planned.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"3 - 5"},"PeriodicalIF":9.5,"publicationDate":"2021-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46409058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-18DOI: 10.1177/10944281211042388
Kyle J. Emich, Li Lu, Amanda J. Ferguson, R. Peterson, Michael McCourt
Research methods for studying team composition tend to employ either a variable-centered or person-centered approach. The variable-centered approach allows scholars to consider how patterns of attributes between team members influence teams, while the person-centered approach allows scholars to consider how variation in multiple attributes within team members influences subgroup formation and its effects. Team composition theory, however, is becoming increasingly sophisticated, assuming variation on multiple attributes both within and between team members—for example, in predicting how a team functions differently when its most assertive members are also optimistic rather than pessimistic. To support this new theory, we propose an attribute alignment approach, which complements the variable-centered and person-centered approaches by modeling teams as matrices of their members and their members’ attributes. We first demonstrate how to calculate attribute alignment by determining the vector norm and vector angle between team members’ attributes. Then, we demonstrate how the alignment of team member personality attributes (neuroticism and agreeableness) affects team relationship conflict. Finally, we discuss the potential of using the attribute alignment approach to enrich broader team research.
{"title":"Team Composition Revisited: A Team Member Attribute Alignment Approach","authors":"Kyle J. Emich, Li Lu, Amanda J. Ferguson, R. Peterson, Michael McCourt","doi":"10.1177/10944281211042388","DOIUrl":"https://doi.org/10.1177/10944281211042388","url":null,"abstract":"Research methods for studying team composition tend to employ either a variable-centered or person-centered approach. The variable-centered approach allows scholars to consider how patterns of attributes between team members influence teams, while the person-centered approach allows scholars to consider how variation in multiple attributes within team members influences subgroup formation and its effects. Team composition theory, however, is becoming increasingly sophisticated, assuming variation on multiple attributes both within and between team members—for example, in predicting how a team functions differently when its most assertive members are also optimistic rather than pessimistic. To support this new theory, we propose an attribute alignment approach, which complements the variable-centered and person-centered approaches by modeling teams as matrices of their members and their members’ attributes. We first demonstrate how to calculate attribute alignment by determining the vector norm and vector angle between team members’ attributes. Then, we demonstrate how the alignment of team member personality attributes (neuroticism and agreeableness) affects team relationship conflict. Finally, we discuss the potential of using the attribute alignment approach to enrich broader team research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"642 - 672"},"PeriodicalIF":9.5,"publicationDate":"2021-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42555071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-18DOI: 10.1177/10944281211050609
Seang-Hwane Joo, Philseok Lee, Jung Yeon Park, Stephen E. Stark
Although the use of ideal point item response theory (IRT) models for organizational research has increased over the last decade, the assessment of construct dimensionality of ideal point scales has been overlooked in previous research. In this study, we developed and evaluated dimensionality assessment methods for an ideal point IRT model under the Bayesian framework. We applied the posterior predictive model checking (PPMC) approach to the most widely used ideal point IRT model, the generalized graded unfolding model (GGUM). We conducted a Monte Carlo simulation to compare the performance of item pair discrepancy statistics and to evaluate the Type I error and power rates of the methods. The simulation results indicated that the Bayesian dimensionality detection method controlled Type I errors reasonably well across the conditions. In addition, the proposed method showed better performance than existing methods, yielding acceptable power when 20% of the items were generated from the secondary dimension. Organizational implications and limitations of the study are further discussed.
{"title":"Assessing Dimensionality of the Ideal Point Item Response Theory Model Using Posterior Predictive Model Checking","authors":"Seang-Hwane Joo, Philseok Lee, Jung Yeon Park, Stephen E. Stark","doi":"10.1177/10944281211050609","DOIUrl":"https://doi.org/10.1177/10944281211050609","url":null,"abstract":"Although the use of ideal point item response theory (IRT) models for organizational research has increased over the last decade, the assessment of construct dimensionality of ideal point scales has been overlooked in previous research. In this study, we developed and evaluated dimensionality assessment methods for an ideal point IRT model under the Bayesian framework. We applied the posterior predictive model checking (PPMC) approach to the most widely used ideal point IRT model, the generalized graded unfolding model (GGUM). We conducted a Monte Carlo simulation to compare the performance of item pair discrepancy statistics and to evaluate the Type I error and power rates of the methods. The simulation results indicated that the Bayesian dimensionality detection method controlled Type I errors reasonably well across the conditions. In addition, the proposed method showed better performance than existing methods, yielding acceptable power when 20% of the items were generated from the secondary dimension. Organizational implications and limitations of the study are further discussed.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"26 1","pages":"353 - 382"},"PeriodicalIF":9.5,"publicationDate":"2021-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45044899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1177/10944281211030648
L. J. Williams, G. Banks, R. Vandenberg
Providing developmental peer reviewers is one of the most critical services performed by researchers in the organizational sciences (Bedeian, 2003). Yet, completing helpful and constructive reviews is not easy (Epstein, 1995; Feldman, 2005). This challenge may be due, in part, to the fact that our field provides only limited formal reviewer training in graduate programs and through professional development workshops (PDWs). Much of what new reviewers learn happens through informal training with mentors (Carpenter, 2009). Without effective training, reviewers may be prone to biases in their methodological evaluations of manuscripts (Banks et al., 2016; Bedeian, Taylor, & Miller, 2010; Emerson et al., 2010) or may simply lack the expertise needed to evaluate manuscripts due to the large variety of content areas and methodological techniques being employed in research. Many editorials have been written to provide guidance for basic reviewer development (e.g., Lee, 1995). Recently, the Society for Industrial and Organizational Psychology (SIOP) and the Consortium for the Advancement of Research Methods and Analysis (CARMA) started an initiative around basic reviewer development (http://carmarmep.org/siop-carma-reviewer-series/). This ongoing training serves to introduce basic reviewer competencies (Koehler et al., 2020), recommend readings, and training videos that are freely available to help new and even experienced reviewers improve the quality of their reviews. While basic reviewer development is laudable, there is also a need for more formal training on advanced methodological topics. Hence, Organizational Research Methods along with CARMA are now introducing a new Virtual Feature Topic targeted at advanced reviewer development.
{"title":"ORM-CARMA Virtual Feature Topics for Advanced Reviewer Development","authors":"L. J. Williams, G. Banks, R. Vandenberg","doi":"10.1177/10944281211030648","DOIUrl":"https://doi.org/10.1177/10944281211030648","url":null,"abstract":"Providing developmental peer reviewers is one of the most critical services performed by researchers in the organizational sciences (Bedeian, 2003). Yet, completing helpful and constructive reviews is not easy (Epstein, 1995; Feldman, 2005). This challenge may be due, in part, to the fact that our field provides only limited formal reviewer training in graduate programs and through professional development workshops (PDWs). Much of what new reviewers learn happens through informal training with mentors (Carpenter, 2009). Without effective training, reviewers may be prone to biases in their methodological evaluations of manuscripts (Banks et al., 2016; Bedeian, Taylor, & Miller, 2010; Emerson et al., 2010) or may simply lack the expertise needed to evaluate manuscripts due to the large variety of content areas and methodological techniques being employed in research. Many editorials have been written to provide guidance for basic reviewer development (e.g., Lee, 1995). Recently, the Society for Industrial and Organizational Psychology (SIOP) and the Consortium for the Advancement of Research Methods and Analysis (CARMA) started an initiative around basic reviewer development (http://carmarmep.org/siop-carma-reviewer-series/). This ongoing training serves to introduce basic reviewer competencies (Koehler et al., 2020), recommend readings, and training videos that are freely available to help new and even experienced reviewers improve the quality of their reviews. While basic reviewer development is laudable, there is also a need for more formal training on advanced methodological topics. Hence, Organizational Research Methods along with CARMA are now introducing a new Virtual Feature Topic targeted at advanced reviewer development.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"675 - 677"},"PeriodicalIF":9.5,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49065474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1177/1094428120963830
Aaron Schecter, E. Quintane
The relational event model (REM) solves a problem for organizational researchers who have access to sequences of time-stamped interactions. It enables them to estimate statistical models without collapsing the data into cross-sectional panels, which removes timing and sequence information. However, there is little guidance in the extant literature regarding issues that may affect REM’s power, precision, and accuracy: How many events or actors are needed? How large should the risk set be? How should statistics be scaled? To gain insights into these issues, we conduct a series of experiments using simulated sequences of relational events under different conditions and using different sampling and scaling strategies. We also provide an empirical example using email communications in a real-life context. Our results indicate that, in most cases, the power and precision levels of REMs are good, making it a strong explanatory model. However, REM suffers from issues of accuracy that can be severe in certain cases, making it a poor predictive model. We provide a set of practical recommendations to guide researchers’ use of REMs in organizational research.
{"title":"The Power, Accuracy, and Precision of the Relational Event Model","authors":"Aaron Schecter, E. Quintane","doi":"10.1177/1094428120963830","DOIUrl":"https://doi.org/10.1177/1094428120963830","url":null,"abstract":"The relational event model (REM) solves a problem for organizational researchers who have access to sequences of time-stamped interactions. It enables them to estimate statistical models without collapsing the data into cross-sectional panels, which removes timing and sequence information. However, there is little guidance in the extant literature regarding issues that may affect REM’s power, precision, and accuracy: How many events or actors are needed? How large should the risk set be? How should statistics be scaled? To gain insights into these issues, we conduct a series of experiments using simulated sequences of relational events under different conditions and using different sampling and scaling strategies. We also provide an empirical example using email communications in a real-life context. Our results indicate that, in most cases, the power and precision levels of REMs are good, making it a strong explanatory model. However, REM suffers from issues of accuracy that can be severe in certain cases, making it a poor predictive model. We provide a set of practical recommendations to guide researchers’ use of REMs in organizational research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"802 - 829"},"PeriodicalIF":9.5,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120963830","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46952812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1177/1094428120967089
J. DeSimone, M. Brannick, Ernest H. O’Boyle, J. Ryu
This article encourages transparency in the reporting of meta-analytic procedures. Specifically, we highlight aspects of meta-analytic search, coding, data presentation, and data analysis where published meta-analyses often fall short in presenting sufficient information to allow replication. We identify opportunities where reviewers can request additional information or analyses that will enhance transparent reporting practices and facilitate the evaluation of quality in meta-analytic reporting. We focus on concerns specific to (or prevalent in) meta-analyses conducted in organizational research. In doing so, we reference a number of existing and emerging techniques, highlighting their contribution to meta-analysis while emphasizing key information reviewers may request. Our focus is primarily on meta-analyses, but secondary uses of meta-analytic data are also considered. We conclude by providing a checklist for reviewers in an effort to facilitate the review process as it pertains to the goals of transparency and replicability.
{"title":"Recommendations for Reviewing Meta-Analyses in Organizational Research","authors":"J. DeSimone, M. Brannick, Ernest H. O’Boyle, J. Ryu","doi":"10.1177/1094428120967089","DOIUrl":"https://doi.org/10.1177/1094428120967089","url":null,"abstract":"This article encourages transparency in the reporting of meta-analytic procedures. Specifically, we highlight aspects of meta-analytic search, coding, data presentation, and data analysis where published meta-analyses often fall short in presenting sufficient information to allow replication. We identify opportunities where reviewers can request additional information or analyses that will enhance transparent reporting practices and facilitate the evaluation of quality in meta-analytic reporting. We focus on concerns specific to (or prevalent in) meta-analyses conducted in organizational research. In doing so, we reference a number of existing and emerging techniques, highlighting their contribution to meta-analysis while emphasizing key information reviewers may request. Our focus is primarily on meta-analyses, but secondary uses of meta-analytic data are also considered. We conclude by providing a checklist for reviewers in an effort to facilitate the review process as it pertains to the goals of transparency and replicability.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"694 - 717"},"PeriodicalIF":9.5,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120967089","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41902304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1177/1094428120915516
Danni Wang, D. Waldman, Pierre A. Balthazard, Maja Stikic, Nicola M. Pless, Thomas Maak, C. Berka, Travis Richardson
In this article, we describe how neuroscience can be used in the study of team dynamics. Specifically, we point out methodological limitations in current team-based research and explain how quantitative electroencephalogram technology can be applied to the study of emergent processes in teams. In so doing, we describe how this technology and related analyses can explain emergent processes in teams through an example of the neural assessment of attention of team members who are engaged in a problem-solving task. Specifically, we demonstrate how the real-time, continuous neural signatures of team members’ attention in a problem-solving context emerges in teams over time. We then consider how further development of this technology might advance our understanding of the emergence of other team-based constructs and research questions.
{"title":"Applying Neuroscience to Emergent Processes in Teams","authors":"Danni Wang, D. Waldman, Pierre A. Balthazard, Maja Stikic, Nicola M. Pless, Thomas Maak, C. Berka, Travis Richardson","doi":"10.1177/1094428120915516","DOIUrl":"https://doi.org/10.1177/1094428120915516","url":null,"abstract":"In this article, we describe how neuroscience can be used in the study of team dynamics. Specifically, we point out methodological limitations in current team-based research and explain how quantitative electroencephalogram technology can be applied to the study of emergent processes in teams. In so doing, we describe how this technology and related analyses can explain emergent processes in teams through an example of the neural assessment of attention of team members who are engaged in a problem-solving task. Specifically, we demonstrate how the real-time, continuous neural signatures of team members’ attention in a problem-solving context emerges in teams over time. We then consider how further development of this technology might advance our understanding of the emergence of other team-based constructs and research questions.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"595 - 615"},"PeriodicalIF":9.5,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120915516","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47461126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-01DOI: 10.1177/1094428120930815
Andrew B. Speer
Performance appraisal narratives are qualitative descriptions of employee job performance. This data source has seen increased research attention due to the ability to efficiently derive insights using natural language processing (NLP). The current study details the development of NLP scoring for performance dimensions from narrative text and then investigates validity and generalizability evidence for those scores. Specifically, narrative valence scores were created to measure a priori performance dimensions. These scores were derived using bag of words and word embedding features and then modeled using modern prediction algorithms. Construct validity evidence was investigated across three samples, revealing that the scores converged with independent human ratings of the text, aligned numerical performance ratings made during the appraisal, and demonstrated some degree of discriminant validity. However, construct validity evidence differed based on which NLP algorithm was used to derive scores. In addition, valence scores generalized to both downward and upward rating contexts. Finally, the performance valence algorithms generalized better in contexts where the same qualitative survey design was used compared with contexts where different instructions were given to elicit narrative text.
{"title":"Scoring Dimension-Level Job Performance From Narrative Comments: Validity and Generalizability When Using Natural Language Processing","authors":"Andrew B. Speer","doi":"10.1177/1094428120930815","DOIUrl":"https://doi.org/10.1177/1094428120930815","url":null,"abstract":"Performance appraisal narratives are qualitative descriptions of employee job performance. This data source has seen increased research attention due to the ability to efficiently derive insights using natural language processing (NLP). The current study details the development of NLP scoring for performance dimensions from narrative text and then investigates validity and generalizability evidence for those scores. Specifically, narrative valence scores were created to measure a priori performance dimensions. These scores were derived using bag of words and word embedding features and then modeled using modern prediction algorithms. Construct validity evidence was investigated across three samples, revealing that the scores converged with independent human ratings of the text, aligned numerical performance ratings made during the appraisal, and demonstrated some degree of discriminant validity. However, construct validity evidence differed based on which NLP algorithm was used to derive scores. In addition, valence scores generalized to both downward and upward rating contexts. Finally, the performance valence algorithms generalized better in contexts where the same qualitative survey design was used compared with contexts where different instructions were given to elicit narrative text.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"572 - 594"},"PeriodicalIF":9.5,"publicationDate":"2021-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120930815","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42014110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：