Pub Date : 2026-01-19DOI: 10.1177/08944393251414170
Wen Zeng, Chandni Kumar, Sinong Zhou, Donggyu Kim, Aimei Yang, Dmitri Williams
This study investigates networked social influence in Sky: Children of the Light , a social multiplayer online game. Drawing on survey responses ( n = 9,254) and in-game data from over 660,000 players, we use an innovative graph-based machine learning approach to quantify how individuals influence others’ playtime, and regression analyses to test predictors from the COM-B model. Results show that Capability enhances influence, although excessive task focus correlates negatively with social impact; Opportunity emerges as the strongest predictor, with active social interactions significantly boosting influence; and Motivation varies by playstyle, with socializers and competitors demonstrating greater influence than narrative-focused players. By applying the COM-B model in a digital gaming context, this research highlights behavioral dimensions of player influence and employs a novel metric for quantifying interpersonal influence. These findings suggest practical implications for game design, particularly by highlighting how social interaction opportunities and different player motivations shape influence within communities.
本研究调查了社交多人在线游戏《Sky: Children of the Light》的网络社交影响。根据调查结果（n = 9254）和来自66万多名玩家的游戏数据，我们使用一种创新的基于图表的机器学习方法来量化个人如何影响他人的游戏时间，并使用回归分析来测试COM-B模型的预测因子。结果表明，能力增强了影响力，但过度的任务关注与社会影响呈负相关；机会是最有力的预测因素，积极的社会互动显著提升了影响力；动机因游戏风格而异，社交者和竞争者比专注于叙述的玩家更有影响力。通过在数字游戏环境中应用COM-B模型，本研究突出了玩家影响的行为维度，并采用了一种量化人际影响的新指标。这些发现对游戏设计具有实际意义，特别是通过强调社交互动机会和不同玩家动机如何影响社区。
{"title":"Capability, Opportunity, and Motivation in a Social Multiplayer Online Game: Player Influence Dynamics in Sky: Children of Light","authors":"Wen Zeng, Chandni Kumar, Sinong Zhou, Donggyu Kim, Aimei Yang, Dmitri Williams","doi":"10.1177/08944393251414170","DOIUrl":"https://doi.org/10.1177/08944393251414170","url":null,"abstract":"This study investigates networked social influence in <jats:italic toggle=\"yes\">Sky: Children of the Light</jats:italic> , a social multiplayer online game. Drawing on survey responses ( <jats:italic toggle=\"yes\">n</jats:italic> = 9,254) and in-game data from over 660,000 players, we use an innovative graph-based machine learning approach to quantify how individuals influence others’ playtime, and regression analyses to test predictors from the COM-B model. Results show that <jats:italic toggle=\"yes\">Capability</jats:italic> enhances influence, although excessive task focus correlates negatively with social impact; <jats:italic toggle=\"yes\">Opportunity</jats:italic> emerges as the strongest predictor, with active social interactions significantly boosting influence; and <jats:italic toggle=\"yes\">Motivation</jats:italic> varies by playstyle, with socializers and competitors demonstrating greater influence than narrative-focused players. By applying the COM-B model in a digital gaming context, this research highlights behavioral dimensions of player influence and employs a novel metric for quantifying interpersonal influence. These findings suggest practical implications for game design, particularly by highlighting how social interaction opportunities and different player motivations shape influence within communities.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"48 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146000836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-14DOI: 10.1177/08944393261416781
John Maina Karanja, Macrina Mbaika Musili
In June 2024, youth-led protests in Kenya against a controversial Finance Bill demonstrated the connection between digital technologies and political activism in the Global South. This study examines how generative artificial intelligence (GAI) shapes political participation by focusing on Kenyan Gen Z activists who used ChatGPT to create custom models: Finance_Bill_GPT, Corrupt_Politicians_GPT, and MPs_Contribution_GPT (collectively called Protest_GPT_KE). These tools simplified complex laws, exposed corruption, and mobilized young people online, allowing them to bypass traditional sources such as media and elites. However, using GAI for activism raises ethical and political concerns, including surveillance, data rights, and state repression. The study surveyed 374 Kenyan Gen Z participants, primarily in Nairobi, and used Partial Least Squares Structural Equation Modeling (PLS-SEM) to analyze the connections among AI use, tool appropriation, and political participation. Results show that ChatGPT use alone did not directly increase offline activism; its effect appeared when combined with Protest_GPT_KE and online participation. This study is one of the first to document how youth in the Global South are creatively using GAI for grassroots mobilization, demonstrating that GAI’s political influence depends on user innovation and context.
{"title":"The Dawn of Generative AI-Enabled Political Activism: How Kenyan Gen Z Used ChatGPT and Protest GPTs to Mobilize","authors":"John Maina Karanja, Macrina Mbaika Musili","doi":"10.1177/08944393261416781","DOIUrl":"https://doi.org/10.1177/08944393261416781","url":null,"abstract":"In June 2024, youth-led protests in Kenya against a controversial Finance Bill demonstrated the connection between digital technologies and political activism in the Global South. This study examines how generative artificial intelligence (GAI) shapes political participation by focusing on Kenyan Gen Z activists who used ChatGPT to create custom models: Finance_Bill_GPT, Corrupt_Politicians_GPT, and MPs_Contribution_GPT (collectively called Protest_GPT_KE). These tools simplified complex laws, exposed corruption, and mobilized young people online, allowing them to bypass traditional sources such as media and elites. However, using GAI for activism raises ethical and political concerns, including surveillance, data rights, and state repression. The study surveyed 374 Kenyan Gen Z participants, primarily in Nairobi, and used Partial Least Squares Structural Equation Modeling (PLS-SEM) to analyze the connections among AI use, tool appropriation, and political participation. Results show that ChatGPT use alone did not directly increase offline activism; its effect appeared when combined with Protest_GPT_KE and online participation. This study is one of the first to document how youth in the Global South are creatively using GAI for grassroots mobilization, demonstrating that GAI’s political influence depends on user innovation and context.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"83 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145961985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-14DOI: 10.1177/08944393251408022
Joshua Claassen, Jan Karem Höhne, Ruben Bach, Anna-Carolina Haensch
Online survey participants are frequently recruited through social media platforms, opt-in online access panels, and river sampling approaches. Such online surveys are threatened by bots that shift survey outcomes and exploit incentives. In this proof-of-concept study, we advance the identification of bots driven by Large Language Models (LLMs) through the prediction of LLM-generated text in open narrative responses. We conducted an online survey on same-gender partnerships, including three open narrative questions, and recruited 1512 participants through Facebook. In addition, we utilized two LLM-driven bots, each of which responded to the open narrative questions 400 times. Open narrative responses synthesized by our bots were labeled as containing LLM-generated text (“yes”). Facebook responses were assigned a proxy label (“unclear”) as they may contain bots themselves. Using this binary label as ground truth, we fine-tuned prediction models relying on the “Bidirectional Encoder Representations from Transformers” (BERT) model, resulting in an impressive prediction performance: The models accurately identified between 97% and 100% of bot responses. However, prediction performance decreases if the models make predictions about questions they were not fine-tuned with. Our study contributes to the ongoing discussion on bots and extends the methodological toolkit for protecting the quality and integrity of online survey data.
{"title":"Identifying Bots Through LLM-Generated Text in Open Narrative Responses: A Proof-of-Concept Study","authors":"Joshua Claassen, Jan Karem Höhne, Ruben Bach, Anna-Carolina Haensch","doi":"10.1177/08944393251408022","DOIUrl":"https://doi.org/10.1177/08944393251408022","url":null,"abstract":"Online survey participants are frequently recruited through social media platforms, opt-in online access panels, and river sampling approaches. Such online surveys are threatened by bots that shift survey outcomes and exploit incentives. In this proof-of-concept study, we advance the identification of bots driven by Large Language Models (LLMs) through the prediction of LLM-generated text in open narrative responses. We conducted an online survey on same-gender partnerships, including three open narrative questions, and recruited 1512 participants through Facebook. In addition, we utilized two LLM-driven bots, each of which responded to the open narrative questions 400 times. Open narrative responses synthesized by our bots were labeled as containing LLM-generated text (“yes”). Facebook responses were assigned a proxy label (“unclear”) as they may contain bots themselves. Using this binary label as ground truth, we fine-tuned prediction models relying on the “Bidirectional Encoder Representations from Transformers” (BERT) model, resulting in an impressive prediction performance: The models accurately identified between 97% and 100% of bot responses. However, prediction performance decreases if the models make predictions about questions they were not fine-tuned with. Our study contributes to the ongoing discussion on bots and extends the methodological toolkit for protecting the quality and integrity of online survey data.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"14 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145961791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-10DOI: 10.1177/08944393261416786
Thomas E. Dearden, Andréanne Bergeron
This study investigates international differences in hacking attempts using honeypot data. By analyzing 504,877 login attempts from 314 unique IP addresses over a 3-month period, the research aims to understand how socio-economic factors influence cybercriminal behavior. We consider prior international work on cybercrime to develop hypotheses regarding opportunity, which predict that countries with higher unemployment and poverty rates, as well as lower GDP and education expenditures, will exhibit more frequent hacking attempts. We found that higher unemployment rates and lower education expenditures correlate with an increase in the mean number of breach attempts per country. Lower education expenditures also correlate with higher success rates of breach attempts. No significant relationship was found between GDP or population below the poverty line and hacking behavior. This study highlights the role of socio-economic conditions in shaping cybercriminal activities, demonstrating that cybercrime does not occur in a vacuum but is influenced by the broader geopolitical context.
{"title":"International Differences in Windows Remote Desktop Hacking: An Analysis of Honeypot Data","authors":"Thomas E. Dearden, Andréanne Bergeron","doi":"10.1177/08944393261416786","DOIUrl":"https://doi.org/10.1177/08944393261416786","url":null,"abstract":"This study investigates international differences in hacking attempts using honeypot data. By analyzing 504,877 login attempts from 314 unique IP addresses over a 3-month period, the research aims to understand how socio-economic factors influence cybercriminal behavior. We consider prior international work on cybercrime to develop hypotheses regarding opportunity, which predict that countries with higher unemployment and poverty rates, as well as lower GDP and education expenditures, will exhibit more frequent hacking attempts. We found that higher unemployment rates and lower education expenditures correlate with an increase in the mean number of breach attempts per country. Lower education expenditures also correlate with higher success rates of breach attempts. No significant relationship was found between GDP or population below the poverty line and hacking behavior. This study highlights the role of socio-economic conditions in shaping cybercriminal activities, demonstrating that cybercrime does not occur in a vacuum but is influenced by the broader geopolitical context.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"35 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145938013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-30DOI: 10.1177/08944393251410596
Liesje C. A. van der Linden, Cedric Waterschoot, Ernst van den Hemel, Florian A. Kunneman, Antal P. J. van den Bosch, Emiel J. Krahmer
To better understand the demographic composition of people participating in commenting sections beneath online news articles, we conducted a large-scale survey ( n = 5,490) with a panel that is representative of the Dutch population – the LISS panel. We combined these data with demographic background variables and previously collected data on political views and values, to provide a detailed description of the identity of online news commenters in comparison to non-commenters. Our results show that the group of commenters contain more men (55%), and the age group of 45–54 years old has the largest share of commenters (18% for men, 13% for women). Furthermore, we found little to no differences for education levels, income, location, political preferences, and cultural background, suggesting that there is no striking overrepresentation of specific groups among online commenters in general. However, when looking at the profiles of online commenters as a function of the topic and platform of discussion, differences start to emerge for gender, age, and education levels. We found no differences related to age and gender distributions for those with a higher commenting frequency, but a higher frequency does go hand in hand with more support for national populist and far-right political parties and a lower confidence in political parties.
{"title":"Who Are the Online Commenters? A Large-Scale Representative Survey to Explore the Identity and Motivation of Online News Commenters in Comparison to Non-Commenters","authors":"Liesje C. A. van der Linden, Cedric Waterschoot, Ernst van den Hemel, Florian A. Kunneman, Antal P. J. van den Bosch, Emiel J. Krahmer","doi":"10.1177/08944393251410596","DOIUrl":"https://doi.org/10.1177/08944393251410596","url":null,"abstract":"To better understand the demographic composition of people participating in commenting sections beneath online news articles, we conducted a large-scale survey ( <jats:italic toggle=\"yes\">n</jats:italic> = 5,490) with a panel that is representative of the Dutch population – the LISS panel. We combined these data with demographic background variables and previously collected data on political views and values, to provide a detailed description of the identity of online news commenters in comparison to non-commenters. Our results show that the group of commenters contain more men (55%), and the age group of 45–54 years old has the largest share of commenters (18% for men, 13% for women). Furthermore, we found little to no differences for education levels, income, location, political preferences, and cultural background, suggesting that there is no striking overrepresentation of specific groups among online commenters in general. However, when looking at the profiles of online commenters as a function of the topic and platform of discussion, differences start to emerge for gender, age, and education levels. We found no differences related to age and gender distributions for those with a higher commenting frequency, but a higher frequency does go hand in hand with more support for national populist and far-right political parties and a lower confidence in political parties.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"13 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145893672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-30DOI: 10.1177/08944393251413796
Marko Galjak, Marina Budić
This cross-sectional study examines a generational divide in the adoption of AI for academic writing among academic researchers in Serbia. A survey of 823 social scientists analyzed usage patterns and measured age-related adoption rates through logistic regression analysis. The findings indicate that 27.2% of researchers employ AI for academic writing, with adoption rates varying significantly by age: 42.9% of researchers in their twenties use these tools, compared to 14.3% of those in their sixties. Researchers aged 23–34 were twice as likely to adopt AI writing tools as those aged 49–80. Each additional year of age reduced the odds of AI adoption by 3.8%, even when controlled for academic title, sex, and workplace type. This age effect persisted while gender and institutional context showed no significant association with adoption. The significant variation in AI adoption across age groups suggests potential shifts in academia. Senior faculty who avoid AI writing tools cannot effectively mentor graduate students who rely on them. Manuscripts now face inconsistent peer review standards; reviewers familiar with AI-assisted writing apply different criteria than those who reject it entirely. Universities face competing demands: junior researchers insist AI tools help them publish enough to secure tenure, yet senior faculty argue that students who depend on these tools never learn to construct arguments or evaluate evidence independently.
{"title":"Generational Divide in AI Adoption for Academic Writing: Evidence From Serbian Social Scientists","authors":"Marko Galjak, Marina Budić","doi":"10.1177/08944393251413796","DOIUrl":"https://doi.org/10.1177/08944393251413796","url":null,"abstract":"This cross-sectional study examines a generational divide in the adoption of AI for academic writing among academic researchers in Serbia. A survey of 823 social scientists analyzed usage patterns and measured age-related adoption rates through logistic regression analysis. The findings indicate that 27.2% of researchers employ AI for academic writing, with adoption rates varying significantly by age: 42.9% of researchers in their twenties use these tools, compared to 14.3% of those in their sixties. Researchers aged 23–34 were twice as likely to adopt AI writing tools as those aged 49–80. Each additional year of age reduced the odds of AI adoption by 3.8%, even when controlled for academic title, sex, and workplace type. This age effect persisted while gender and institutional context showed no significant association with adoption. The significant variation in AI adoption across age groups suggests potential shifts in academia. Senior faculty who avoid AI writing tools cannot effectively mentor graduate students who rely on them. Manuscripts now face inconsistent peer review standards; reviewers familiar with AI-assisted writing apply different criteria than those who reject it entirely. Universities face competing demands: junior researchers insist AI tools help them publish enough to secure tenure, yet senior faculty argue that students who depend on these tools never learn to construct arguments or evaluate evidence independently.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"39 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145895536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-24DOI: 10.1177/08944393251409744
Daniel Karell, Matthew Shu, Thomas Davidson, Keitaro Okura
Many people now use AI chatbots to obtain summaries of complex topics, yet we know little about how this affects knowledge acquisition, including how the effects might vary across different groups of people. We conducted two experiments comparing how well people recalled factual information after reading AI-generated or human-written historical summaries. Participants who read AI-generated summaries scored significantly higher on knowledge tests than those who read expert-written blog posts (Study 1) or Wikipedia articles (Study 2). These improvements were present regardless of whether readers knew the content was AI-generated or if the AI summaries were politically biased. Moreover, AI summaries improved recall across various demographic groups, including gender, race, income, education, and digital literacy levels. This suggets that using AI tools for everyday factual queries does not create new knowledge inequalities but could still amplify existing ones through differential access. Our findings indicate that the increasingly routine use of AI for information-seeking could enhance factual learning, with implications for education policy and addressing inequality.
{"title":"Generating the Past: How Artificial Intelligence Summaries of Historical Events Affect Knowledge","authors":"Daniel Karell, Matthew Shu, Thomas Davidson, Keitaro Okura","doi":"10.1177/08944393251409744","DOIUrl":"https://doi.org/10.1177/08944393251409744","url":null,"abstract":"Many people now use AI chatbots to obtain summaries of complex topics, yet we know little about how this affects knowledge acquisition, including how the effects might vary across different groups of people. We conducted two experiments comparing how well people recalled factual information after reading AI-generated or human-written historical summaries. Participants who read AI-generated summaries scored significantly higher on knowledge tests than those who read expert-written blog posts (Study 1) or Wikipedia articles (Study 2). These improvements were present regardless of whether readers knew the content was AI-generated or if the AI summaries were politically biased. Moreover, AI summaries improved recall across various demographic groups, including gender, race, income, education, and digital literacy levels. This suggets that using AI tools for everyday factual queries does not create new knowledge inequalities but could still amplify existing ones through differential access. Our findings indicate that the increasingly routine use of AI for information-seeking could enhance factual learning, with implications for education policy and addressing inequality.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"93 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-12-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145813191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-23DOI: 10.1177/08944393251410155
Taewoo Kang, Kjerstin Thorson, Tai-Quan Peng, Dan Hiaeshutter-Rice, Sanguk Lee, Stuart Soroka
This study attempts to advance automated content analysis from consensus-oriented to coordination-oriented practices, thereby embracing diverse coding outputs and exploring the dynamics among differential perspectives. As an exploratory investigation, we evaluate six GPT-4o configurations to analyze sentiment in Fox News and MSNBC transcripts on Biden and Trump during the 2020 U.S. presidential campaign. By assessing each model’s alignment with partisan perspectives, we explore how partisan selective processing can be identified in LLM-Assisted Content Analysis (LACA). The findings indicate that LLM-based partisan perspective simulations reflect politically polarized standpoints across partisan groups, revealing a pronounced divergence in sentiment analysis between Democrat-aligned and Republican-aligned persona models. This pattern is evident in intercoder-reliability metrics, which are higher among same-partisan than cross-partisan persona model pairs. Results also suggest that LLM partisan simulations exhibit stronger ideological biases when analyzing politically congruent content. This approach enhances the nuanced understanding of LLM outputs and advances the integrity of AI-driven social science research and may also enable simulations of real-world implications.
{"title":"Embracing Dialectic Intersubjectivity: Coordination of Differential Perspectives in Content Analysis With LLM Persona Simulation","authors":"Taewoo Kang, Kjerstin Thorson, Tai-Quan Peng, Dan Hiaeshutter-Rice, Sanguk Lee, Stuart Soroka","doi":"10.1177/08944393251410155","DOIUrl":"https://doi.org/10.1177/08944393251410155","url":null,"abstract":"This study attempts to advance automated content analysis from consensus-oriented to coordination-oriented practices, thereby embracing diverse coding outputs and exploring the dynamics among differential perspectives. As an exploratory investigation, we evaluate six GPT-4o configurations to analyze sentiment in Fox News and MSNBC transcripts on Biden and Trump during the 2020 U.S. presidential campaign. By assessing each model’s alignment with partisan perspectives, we explore how partisan selective processing can be identified in LLM-Assisted Content Analysis (LACA). The findings indicate that LLM-based partisan perspective simulations reflect politically polarized standpoints across partisan groups, revealing a pronounced divergence in sentiment analysis between Democrat-aligned and Republican-aligned persona models. This pattern is evident in intercoder-reliability metrics, which are higher among same-partisan than cross-partisan persona model pairs. Results also suggest that LLM partisan simulations exhibit stronger ideological biases when analyzing politically congruent content. This approach enhances the nuanced understanding of LLM outputs and advances the integrity of AI-driven social science research and may also enable simulations of real-world implications.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"1 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145813192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-03DOI: 10.1177/08944393251404052
Frederic Hilkenmeier, Marie Pelzer, Christian Stierle, Jakob Fink-Lamotte
Systematic reviews are essential for evidence synthesis but often require extensive time and resources, especially during data extraction. This proof-of-concept study evaluates the performance of Elicit , an AI tool specifically developed to support systematic reviews, in the context of a systematic review on psychological factors in dermatological conditions. We compared Elicit’s automated data extraction with manually extracted data across 43 studies and 602 data points. Both were assessed against a consensus-based ground truth. Elicit achieved an overall accuracy of 81.4%, compared to 86.7% for human reviewers—a difference that was not statistically significant. In cases where Elicit and the human reviewer extracted the same information, this information was correct in 100% of instances, suggesting that agreement between human and machine may serve as a reliable proxy for validity. Based on these results, we propose a semi-automated workflow in which Elicit functions as a second reviewer, reducing workload while maintaining high data quality. Our results demonstrate that domain-specific AI tools can effectively augment data extraction in systematic reviews, especially in settings with limited time or personnel.
{"title":"Evaluating the AI Tool “Elicit” as a Semi-Automated Second Reviewer for Data Extraction in Systematic Reviews: A Proof-of-Concept","authors":"Frederic Hilkenmeier, Marie Pelzer, Christian Stierle, Jakob Fink-Lamotte","doi":"10.1177/08944393251404052","DOIUrl":"https://doi.org/10.1177/08944393251404052","url":null,"abstract":"Systematic reviews are essential for evidence synthesis but often require extensive time and resources, especially during data extraction. This proof-of-concept study evaluates the performance of <jats:italic toggle=\"yes\">Elicit</jats:italic> , an AI tool specifically developed to support systematic reviews, in the context of a systematic review on psychological factors in dermatological conditions. We compared <jats:italic toggle=\"yes\">Elicit’s</jats:italic> automated data extraction with manually extracted data across 43 studies and 602 data points. Both were assessed against a consensus-based ground truth. Elicit achieved an overall accuracy of 81.4%, compared to 86.7% for human reviewers—a difference that was not statistically significant. In cases where <jats:italic toggle=\"yes\">Elicit</jats:italic> and the human reviewer extracted the same information, this information was correct in 100% of instances, suggesting that agreement between human and machine may serve as a reliable proxy for validity. Based on these results, we propose a semi-automated workflow in which <jats:italic toggle=\"yes\">Elicit</jats:italic> functions as a second reviewer, reducing workload while maintaining high data quality. Our results demonstrate that domain-specific AI tools can effectively augment data extraction in systematic reviews, especially in settings with limited time or personnel.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"1 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145664391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-11-29DOI: 10.1177/08944393251403501
Carina Cornesse, Julia Witton, Julian B. Axenfeld, Jean-Yves Gerlitz, Olaf Groh-Samberg
Research shows that concurrent and sequential self-administered mixed-mode designs both have advantages and disadvantages in terms of panel survey recruitment and maintenance. Since concurrent mixed-mode designs usually achieve higher initial response rates at lower bias than sequential mixed-mode designs, the former may be ideal for panel recruitment. However, concurrent designs produced high share of paper respondents relative to web respondents. Since these paper respondents have been found to be at higher risk of attrition, cause higher data collection costs, and slow down the fieldwork process, sequential mixed-mode designs may be more practical in the regular course of the panel study after recruitment. Our study provides experimental evidence on the effect of switching a panel study from concurrent to sequential mixed-mode design after the panel recruitment. Results show that this switch significantly increases the share of online respondents without harming response rates. Respondents who are pushed to the web by the design change differ significantly from respondents who continue to participate via paper questionnaires with regard to a number of socio-digital inequality correlates. This suggests that, while the share of online respondents can be increased through mode sequencing, keeping the paper mail mode option is vital for ensuring continued representation of societal subgroups.
{"title":"From Concurrent to Push-To-Web Mixed-Mode: Experimental Design Change in the German Social Cohesion Panel","authors":"Carina Cornesse, Julia Witton, Julian B. Axenfeld, Jean-Yves Gerlitz, Olaf Groh-Samberg","doi":"10.1177/08944393251403501","DOIUrl":"https://doi.org/10.1177/08944393251403501","url":null,"abstract":"Research shows that concurrent and sequential self-administered mixed-mode designs both have advantages and disadvantages in terms of panel survey recruitment and maintenance. Since concurrent mixed-mode designs usually achieve higher initial response rates at lower bias than sequential mixed-mode designs, the former may be ideal for panel recruitment. However, concurrent designs produced high share of paper respondents relative to web respondents. Since these paper respondents have been found to be at higher risk of attrition, cause higher data collection costs, and slow down the fieldwork process, sequential mixed-mode designs may be more practical in the regular course of the panel study after recruitment. Our study provides experimental evidence on the effect of switching a panel study from concurrent to sequential mixed-mode design after the panel recruitment. Results show that this switch significantly increases the share of online respondents without harming response rates. Respondents who are pushed to the web by the design change differ significantly from respondents who continue to participate via paper questionnaires with regard to a number of socio-digital inequality correlates. This suggests that, while the share of online respondents can be increased through mode sequencing, keeping the paper mail mode option is vital for ensuring continued representation of societal subgroups.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"71 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145614129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：