Pub Date : 2025-07-15DOI: 10.1177/08944393251361455
Shaojung Sharon Wang, Shiuan-Tung Chen
As geolocation tracking apps become increasingly embedded in everyday digital interactions, their role in romantic relationships remains underexplored. This study examines the use of geolocation tracking apps in romantic relationships, addressing gaps in understanding their implications for relational uncertainty while identifying key psychological antecedents of app use. Findings from users ( N = 333) challenge the assumption that geolocation tracking inherently reduces relational uncertainty, revealing no significant association with increased clarity. Instead, intensive tracking correlates with heightened definition uncertainty, suggesting that rather than reinforcing relationship security, tracking may introduce ambiguity about the relationship’s status. However, it is not associated with diminished intimacy, as couples may use it consensually for safety and reassurance. Moreover, attachment styles and jealousy predict tracking behaviors, mirroring patterns observed in social media surveillance. These findings highlight the limitations of geolocation tracking as an uncertainty-reducing tool and emphasize the psychological and relational factors that drive its use. By reframing geolocation tracking as a socially accepted yet relationally complex form of monitoring, this study advances theoretical discussions on digital surveillance and the evolving role of technology in intimate relationships.
{"title":"Digital Surveillance and Relational Uncertainty: The Role of Geolocation Tracking in Romantic Relationships","authors":"Shaojung Sharon Wang, Shiuan-Tung Chen","doi":"10.1177/08944393251361455","DOIUrl":"https://doi.org/10.1177/08944393251361455","url":null,"abstract":"As geolocation tracking apps become increasingly embedded in everyday digital interactions, their role in romantic relationships remains underexplored. This study examines the use of geolocation tracking apps in romantic relationships, addressing gaps in understanding their implications for relational uncertainty while identifying key psychological antecedents of app use. Findings from users ( <jats:italic>N</jats:italic> = 333) challenge the assumption that geolocation tracking inherently reduces relational uncertainty, revealing no significant association with increased clarity. Instead, intensive tracking correlates with heightened definition uncertainty, suggesting that rather than reinforcing relationship security, tracking may introduce ambiguity about the relationship’s status. However, it is not associated with diminished intimacy, as couples may use it consensually for safety and reassurance. Moreover, attachment styles and jealousy predict tracking behaviors, mirroring patterns observed in social media surveillance. These findings highlight the limitations of geolocation tracking as an uncertainty-reducing tool and emphasize the psychological and relational factors that drive its use. By reframing geolocation tracking as a socially accepted yet relationally complex form of monitoring, this study advances theoretical discussions on digital surveillance and the evolving role of technology in intimate relationships.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"109 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144629771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-11DOI: 10.1177/08944393251356630
Minxuan Lan, Lin Liu, Jon D. Elhai, Hanlin Zhou, Xin Gu, Zihan Su, Debao Chen
Crime is not randomly distributed but tends to occur in specific spatial clusters. The literature has published many theories to explain its underlying causes. In recent years, scholars have increasingly leveraged social media big data to enrich our understanding of crime. Among these efforts is the Broken Emotion Conjecture, which offers a novel perspective on the connection between crime and emotion. However, how this connection varies among crime types and across the geographic space remains unclear. In this study, we investigate the spatial variations of the Broken Emotion Conjecture by analyzing emotion of residents and visitors, and their associations with assaults, burglaries, robberies, and thefts in Cincinnati, OH. Through spatial statistical analyses, we find that emotional states of residents and visitors have distinct effects on crime. Specifically, after controlling for key socioeconomic and land-use factors, we observed that collective negative emotion among residents is associated with a higher likelihood of burglaries; while collective negative emotion among visitors correlated with increased risk of assault, burglary, and robbery. Notably, we found no statistically significant impact of either residents’ or visitors' negative emotion on thefts. These findings align with established criminological and psychological theories, but provide a more nuanced interpretation of the connection between emotion and crime. Our study contributes to the growing body of research on the crime-emotion relationship, supports the development of an ambient population based emotion research within criminology, and provides practical policy implications.
{"title":"Spatial Variations of the Broken Emotion Conjecture","authors":"Minxuan Lan, Lin Liu, Jon D. Elhai, Hanlin Zhou, Xin Gu, Zihan Su, Debao Chen","doi":"10.1177/08944393251356630","DOIUrl":"https://doi.org/10.1177/08944393251356630","url":null,"abstract":"Crime is not randomly distributed but tends to occur in specific spatial clusters. The literature has published many theories to explain its underlying causes. In recent years, scholars have increasingly leveraged social media big data to enrich our understanding of crime. Among these efforts is the Broken Emotion Conjecture, which offers a novel perspective on the connection between crime and emotion. However, how this connection varies among crime types and across the geographic space remains unclear. In this study, we investigate the spatial variations of the Broken Emotion Conjecture by analyzing emotion of residents and visitors, and their associations with assaults, burglaries, robberies, and thefts in Cincinnati, OH. Through spatial statistical analyses, we find that emotional states of residents and visitors have distinct effects on crime. Specifically, after controlling for key socioeconomic and land-use factors, we observed that collective negative emotion among residents is associated with a higher likelihood of burglaries; while collective negative emotion among visitors correlated with increased risk of assault, burglary, and robbery. Notably, we found no statistically significant impact of either residents’ or visitors' negative emotion on thefts. These findings align with established criminological and psychological theories, but provide a more nuanced interpretation of the connection between emotion and crime. Our study contributes to the growing body of research on the crime-emotion relationship, supports the development of an ambient population based emotion research within criminology, and provides practical policy implications.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"6 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144611139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-24DOI: 10.1177/08944393251354977
Christian Pieter Hoffmann, Daniel Bendahan Bitton, Alexander Godulla
Previous research has highlighted that encounters with deepfakes induce uncertainty, skepticism, and mistrust among audiences. In this study, we relate perceived deepfake exposure to media cynicism. Deepfakes shake users’ sense of reality, increasing a need to rely on epistemic authorities, such as journalistic media, while raising fears of manipulation. Based on uncertainty management theory, we propose that two “epistemic virtues” moderate the relationship between deepfake exposure and media cynicism: self-efficacy and intellectual humility. In a survey of 1421 German internet users, we find that perceived deepfake exposure positively relates to media cynicism. Intellectual humility does not dampen this relationship. Deepfake detection self-efficacy may be more harmful than helpful in preventing media cynicism. We discuss these findings in the context of research indicating that users tend to overestimate their ability to detect deepfakes and the challenges the novel deepfake technology poses to audience trust in a digital information ecosystem.
{"title":"The Role of Self-Efficacy and Intellectual Humility in the Relationship Between Perceived Deepfake Exposure and Media Cynicism","authors":"Christian Pieter Hoffmann, Daniel Bendahan Bitton, Alexander Godulla","doi":"10.1177/08944393251354977","DOIUrl":"https://doi.org/10.1177/08944393251354977","url":null,"abstract":"Previous research has highlighted that encounters with deepfakes induce uncertainty, skepticism, and mistrust among audiences. In this study, we relate perceived deepfake exposure to media cynicism. Deepfakes shake users’ sense of reality, increasing a need to rely on epistemic authorities, such as journalistic media, while raising fears of manipulation. Based on uncertainty management theory, we propose that two “epistemic virtues” moderate the relationship between deepfake exposure and media cynicism: self-efficacy and intellectual humility. In a survey of 1421 German internet users, we find that perceived deepfake exposure positively relates to media cynicism. Intellectual humility does not dampen this relationship. Deepfake detection self-efficacy may be more harmful than helpful in preventing media cynicism. We discuss these findings in the context of research indicating that users tend to overestimate their ability to detect deepfakes and the challenges the novel deepfake technology poses to audience trust in a digital information ecosystem.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"17 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144479202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-19DOI: 10.1177/08944393251343930
Pnina Fichman, Gordon Amidu
Online political gendertrolling is widespread, and while research shows that women are trolled more often than men, and that men troll more often than women, it is unclear if there is a difference in political gendertrolling between same- and cross-gender pairs of perpetrator-target. To address this gap, this study first examines the extent and style of gendertrolling. Then, it tests for variations based on the perpetrator’s and target’s gender and the target’s political affiliation. Using a two-by-two factorial design, with four perpetrator-target gender pairs (Women/Women, Women/Men, Men/Women, Men/Men), we performed a content analysis of 4,000 trolling comments on 40 Facebook posts that were made by 20 politicians (Men/Women, Democrats/Republicans). We found significant main and interaction effects in gendertrolling style based on the perpetrator’s and target’s genders and the target’s political affiliation. Women’s trolling styles toward men differed from the dominant trolling style, and regardless of perpetrator gender, the gendertrolling style towards women Democrats differed from the style used towards the other targets. However, we found no significant main or interaction effects in the extent of political gendertrolling in any of the four gender conditions, nor based on target’s political affiliation. Contributing to gendertrolling literature, this paper provides evidence of the complex relationships between same- and cross-gender perpetrator-target pairs.
{"title":"Political Gendertrolling","authors":"Pnina Fichman, Gordon Amidu","doi":"10.1177/08944393251343930","DOIUrl":"https://doi.org/10.1177/08944393251343930","url":null,"abstract":"Online political gendertrolling is widespread, and while research shows that women are trolled more often than men, and that men troll more often than women, it is unclear if there is a difference in political gendertrolling between same- and cross-gender pairs of perpetrator-target. To address this gap, this study first examines the extent and style of gendertrolling. Then, it tests for variations based on the perpetrator’s and target’s gender and the target’s political affiliation. Using a two-by-two factorial design, with four perpetrator-target gender pairs (Women/Women, Women/Men, Men/Women, Men/Men), we performed a content analysis of 4,000 trolling comments on 40 Facebook posts that were made by 20 politicians (Men/Women, Democrats/Republicans). We found significant main and interaction effects in gendertrolling <jats:italic>style</jats:italic> based on the perpetrator’s and target’s genders and the target’s political affiliation. Women’s trolling styles toward men differed from the dominant trolling style, and regardless of perpetrator gender, the gendertrolling style towards women Democrats differed from the style used towards the other targets. However, we found no significant main or interaction effects in the <jats:italic>extent</jats:italic> of political gendertrolling in any of the four gender conditions, nor based on target’s political affiliation. Contributing to gendertrolling literature, this paper provides evidence of the complex relationships between same- and cross-gender perpetrator-target pairs.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"56 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144328808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-13DOI: 10.1177/08944393251341201
Chiara Respi, Marco Gui, Gaetano Scaduto, Miriam Serini, Dario Pizzul, Tiziano Gerosa, Christoph Lutz
Prior research on digital inequality has highlighted the role of sociocultural resources in shaping Internet beneficial use patterns by positively impacting on online literacy. Research on privacy protection online has—at the same time—shown the emergence of a “privacy cynicism,” where concerns about privacy fail to translate into protective actions. This study investigates how education level impacts privacy protection behavior through these two different mediation paths. Using unique data from a sample of 3,156 Italian Internet users, structural equation modeling (SEM) is employed to analyze the linkages between education level, privacy literacy, privacy cynicism, and protective behaviors. Contrary to expectations, the results reveal a moderate negative impact of education level on privacy protection behaviors. This total effect is the results of two different paths exerting opposite effects on protection behaviors. While a higher education correlates with increased privacy literacy, this competence does not translate into proactive protective actions. Surprisingly, individuals with higher privacy literacy exhibit even lower levels of protection behavior, contributing to a negative indirect effect of education on privacy protection. On the other side, the indirect effect of education on behaviors through privacy cynicism operates consistently with the digital inequality framework, partially compensating the negative effect through literacy. An interpretation of privacy protection as an exception within the digital inequality framework is proposed.
{"title":"Lower Cynicism, Not Higher Literacy, Promotes Protective Behavior: Exploring the “privacy exception” in the Digital Inequality Framework","authors":"Chiara Respi, Marco Gui, Gaetano Scaduto, Miriam Serini, Dario Pizzul, Tiziano Gerosa, Christoph Lutz","doi":"10.1177/08944393251341201","DOIUrl":"https://doi.org/10.1177/08944393251341201","url":null,"abstract":"Prior research on digital inequality has highlighted the role of sociocultural resources in shaping Internet beneficial use patterns by positively impacting on online literacy. Research on privacy protection online has—at the same time—shown the emergence of a “privacy cynicism,” where concerns about privacy fail to translate into protective actions. This study investigates how education level impacts privacy protection behavior through these two different mediation paths. Using unique data from a sample of 3,156 Italian Internet users, structural equation modeling (SEM) is employed to analyze the linkages between education level, privacy literacy, privacy cynicism, and protective behaviors. Contrary to expectations, the results reveal a moderate negative impact of education level on privacy protection behaviors. This total effect is the results of two different paths exerting opposite effects on protection behaviors. While a higher education correlates with increased privacy literacy, this competence does not translate into proactive protective actions. Surprisingly, individuals with higher privacy literacy exhibit even lower levels of protection behavior, contributing to a negative indirect effect of education on privacy protection. On the other side, the indirect effect of education on behaviors through privacy cynicism operates consistently with the digital inequality framework, partially compensating the negative effect through literacy. An interpretation of privacy protection as an exception within the digital inequality framework is proposed.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"44 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144290121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-10DOI: 10.1177/08944393251349541
Mike Farjam, Hendrik Meyer, Meike Lohkamp
This paper provides a practical example and guide on how to augment or replace human coders with Large Language Models (LLMs) during content analysis. We demonstrate this by replicating and extending an influential study on environmental communication. Our setup, running locally on consumer-grade hardware, makes it feasible for university researchers operating within typical computational and legal constraints. We validate the LLM’s performance by replicating the original study’s codings, scaling the analysis to cover a tenfold increase in articles, and extending the LLM’s application to a comparable German-language corpus, comparing these results to human expert coders. We offer guidelines for instructing LLMs, validating output, and handling multilingual coding, presenting a replicable framework for future research. This paper is intended to systematically guide other researchers when integrating LLMs into their workflows, ensuring reliable and scalable coding practices. We demonstrate several advantages of LLMs as coders, including cost-effective multilingual coding, overcoming the limitations of small-sample content analysis, and improving both the replicability and transparency of the coding process.
{"title":"A Practical Guide and Case Study on How to Instruct LLMs for Automated Coding During Content Analysis","authors":"Mike Farjam, Hendrik Meyer, Meike Lohkamp","doi":"10.1177/08944393251349541","DOIUrl":"https://doi.org/10.1177/08944393251349541","url":null,"abstract":"This paper provides a practical example and guide on how to augment or replace human coders with Large Language Models (LLMs) during content analysis. We demonstrate this by replicating and extending an influential study on environmental communication. Our setup, running locally on consumer-grade hardware, makes it feasible for university researchers operating within typical computational and legal constraints. We validate the LLM’s performance by replicating the original study’s codings, scaling the analysis to cover a tenfold increase in articles, and extending the LLM’s application to a comparable German-language corpus, comparing these results to human expert coders. We offer guidelines for instructing LLMs, validating output, and handling multilingual coding, presenting a replicable framework for future research. This paper is intended to systematically guide other researchers when integrating LLMs into their workflows, ensuring reliable and scalable coding practices. We demonstrate several advantages of LLMs as coders, including cost-effective multilingual coding, overcoming the limitations of small-sample content analysis, and improving both the replicability and transparency of the coding process.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"218 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144260638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-09DOI: 10.1177/08944393251349542
Rejoice Chitengu, Silas Formunyuy Verkijika, Kelibone Eva Mamabolo
Civil unrest, encompassing protests and riots, is an increasing global concern, with incidents rising at an alarming rate, a trend that has been observed in South Africa over the years. This issue is particularly pronounced in today’s social media era, where platforms like ‘X’ (formerly Twitter) serve as powerful tools for mobilization. This raises the question: What factors drive civil unrest, and how can machine learning, using social media data, be employed to forecast such events? In response, this study had as objective to develop a hybrid machine learning model to forecast protest and riot events in South Africa using Twitter data. Employing the CRISP-DM methodology, data was collected from Twitter for the period between 2019 and 2024, resulting in 18,487 curated tweets, with associated ground truth data extracted from the ACLED database. Using this data, a hybrid model combining Bidirectional LSTM (Bi-LSTM) networks with eXtreme Gradient Boosting (XGBoost) for classification and regression tasks was developed to forecast civil unrest in South Africa. Additionally, SHapley Additive exPlanations (SHAP) were used for model explainability. The proposed model outperformed the base model, achieving an R-squared value of 33% for protests and 23% for riots in regression, along with classification accuracies of 92% for protests and 86.2% for riots. SHAP results indicated that the key predictors of unrest included sentiment-related features, tweet engagement features, regional factors, the day of the week, public holidays, and the topics being discussed. This study demonstrates the value of a hybrid model in forecasting civil unrest events and identifies key features that stakeholders can use to target their efforts more precisely in addressing civil unrest, ensuring resources are allocated where they are needed most. The study concludes with a discussion of valuable insights for stakeholders on how to leverage social media data to predict and mitigate civil unrest.
{"title":"Forecasting Civil Unrest in South Africa Using Social Media Data: A Hybrid Machine Learning Approach","authors":"Rejoice Chitengu, Silas Formunyuy Verkijika, Kelibone Eva Mamabolo","doi":"10.1177/08944393251349542","DOIUrl":"https://doi.org/10.1177/08944393251349542","url":null,"abstract":"Civil unrest, encompassing protests and riots, is an increasing global concern, with incidents rising at an alarming rate, a trend that has been observed in South Africa over the years. This issue is particularly pronounced in today’s social media era, where platforms like ‘X’ (formerly Twitter) serve as powerful tools for mobilization. This raises the question: What factors drive civil unrest, and how can machine learning, using social media data, be employed to forecast such events? In response, this study had as objective to develop a hybrid machine learning model to forecast protest and riot events in South Africa using Twitter data. Employing the CRISP-DM methodology, data was collected from Twitter for the period between 2019 and 2024, resulting in 18,487 curated tweets, with associated ground truth data extracted from the ACLED database. Using this data, a hybrid model combining Bidirectional LSTM (Bi-LSTM) networks with eXtreme Gradient Boosting (XGBoost) for classification and regression tasks was developed to forecast civil unrest in South Africa. Additionally, SHapley Additive exPlanations (SHAP) were used for model explainability. The proposed model outperformed the base model, achieving an R-squared value of 33% for protests and 23% for riots in regression, along with classification accuracies of 92% for protests and 86.2% for riots. SHAP results indicated that the key predictors of unrest included sentiment-related features, tweet engagement features, regional factors, the day of the week, public holidays, and the topics being discussed. This study demonstrates the value of a hybrid model in forecasting civil unrest events and identifies key features that stakeholders can use to target their efforts more precisely in addressing civil unrest, ensuring resources are allocated where they are needed most. The study concludes with a discussion of valuable insights for stakeholders on how to leverage social media data to predict and mitigate civil unrest.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"60 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144252137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-27DOI: 10.1177/08944393251344865
Laurence-Olivier M. Foisy, Étienne Proulx, Hubert Cadieux, Jérémy Gilbert, Jozef Rivest, Alexandre Bouillon, Yannick Dufresne
This research note addresses a methodological gap in the study of large language models (LLMs) in social sciences: the absence of standardized data extraction procedures. While existing research has examined biases and the reliability of LLM-generated content, the establishment of transparent extraction protocols necessarily precedes substantive analysis. The paper introduces a replicable procedural framework for extracting structured political data from LLMs via API, designed to enhance transparency, accessibility, and reproducibility. Canadian federal and Quebec provincial politicians serve as an illustrative case to demonstrate the extraction methodology, encompassing prompt engineering, output processing, and error handling mechanisms. The procedure facilitates systematic data collection across multiple LLM versions, enabling inter-model comparisons while addressing extraction challenges such as response variability and malformed outputs. The contribution is primarily methodological—providing researchers with a foundational extraction protocol adaptable to diverse research contexts. This standardized approach constitutes an essential preliminary step for subsequent evaluation of LLM-generated content, establishing procedural clarity in this methodologically developing research domain.
{"title":"Prompting the Machine: Introducing an LLM Data Extraction Method for Social Scientists","authors":"Laurence-Olivier M. Foisy, Étienne Proulx, Hubert Cadieux, Jérémy Gilbert, Jozef Rivest, Alexandre Bouillon, Yannick Dufresne","doi":"10.1177/08944393251344865","DOIUrl":"https://doi.org/10.1177/08944393251344865","url":null,"abstract":"This research note addresses a methodological gap in the study of large language models (LLMs) in social sciences: the absence of standardized data extraction procedures. While existing research has examined biases and the reliability of LLM-generated content, the establishment of transparent extraction protocols necessarily precedes substantive analysis. The paper introduces a replicable procedural framework for extracting structured political data from LLMs via API, designed to enhance transparency, accessibility, and reproducibility. Canadian federal and Quebec provincial politicians serve as an illustrative case to demonstrate the extraction methodology, encompassing prompt engineering, output processing, and error handling mechanisms. The procedure facilitates systematic data collection across multiple LLM versions, enabling inter-model comparisons while addressing extraction challenges such as response variability and malformed outputs. The contribution is primarily methodological—providing researchers with a foundational extraction protocol adaptable to diverse research contexts. This standardized approach constitutes an essential preliminary step for subsequent evaluation of LLM-generated content, establishing procedural clarity in this methodologically developing research domain.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"151 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144153928","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-20DOI: 10.1177/08944393251338396
Vihang Jumle, Mykola Makhortykh, Maryna Sydorova, Victoria Vziatysheva
Framing is among the most extensively used concepts in the field of communication science. The availability of digital data offers new possibilities for studying how specific aspects of social reality are made more salient in online communication, but also raises challenges related to the scaling of framing analysis and its adoption to new research areas (e.g. studying the impact of artificial intelligence-powered systems on the representation of societally relevant issues). To address these challenges, we introduce a transformer-based approach for generic news frame detection in Anglophone online content. While doing so, we discuss the composition of the training and test datasets, the model architecture, and the validation of the approach and reflect on the possibilities and limitations of the automated detection of generic news frames.
{"title":"Finding Frames With BERT: A Transformer-Based Approach to Generic News Frame Detection","authors":"Vihang Jumle, Mykola Makhortykh, Maryna Sydorova, Victoria Vziatysheva","doi":"10.1177/08944393251338396","DOIUrl":"https://doi.org/10.1177/08944393251338396","url":null,"abstract":"Framing is among the most extensively used concepts in the field of communication science. The availability of digital data offers new possibilities for studying how specific aspects of social reality are made more salient in online communication, but also raises challenges related to the scaling of framing analysis and its adoption to new research areas (e.g. studying the impact of artificial intelligence-powered systems on the representation of societally relevant issues). To address these challenges, we introduce a transformer-based approach for generic news frame detection in Anglophone online content. While doing so, we discuss the composition of the training and test datasets, the model architecture, and the validation of the approach and reflect on the possibilities and limitations of the automated detection of generic news frames.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"79 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144104618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-02DOI: 10.1177/08944393251334977
Jennifer Stromer-Galley, Brian McKernan, Saklain Zaman, Chinmay Maganur, Sampada Regmi
Systematic content analysis of messaging has been a staple method in the study of communication. While computer-assisted content analysis has been used in the field for three decades, advances in machine learning and crowd-based annotation combined with the ease of collecting volumes of text-based communication via social media have made the opportunities for classification of messages easier and faster. The greatest advancement yet might be in the form of general intelligence large language models (LLMs), which are ostensibly able to accurately and reliably classify messages by leveraging context to disambiguate meaning. It is unclear, however, how effective LLMs are in deploying the method of content analysis. In this study, we compare the classification of political candidate social media messages between trained annotators, crowd annotators, and large language models from Open AI accessed through the free Web (ChatGPT) and the paid API (GPT API) on five different categories of political communication commonly used in the literature. We find that crowd annotation generally had higher F1 scores than ChatGPT and an earlier version of the GPT API, although the newest version, GPT-4 API, demonstrated good performance as compared with the crowd and with ground truth data derived from trained student annotators. This study suggests the application of any LLM to an annotation task requires validation, and that freely available and older LLM models may not be effective for studying human communication.
{"title":"The Efficacy of Large Language Models and Crowd Annotation for Accurate Content Analysis of Political Social Media Messages","authors":"Jennifer Stromer-Galley, Brian McKernan, Saklain Zaman, Chinmay Maganur, Sampada Regmi","doi":"10.1177/08944393251334977","DOIUrl":"https://doi.org/10.1177/08944393251334977","url":null,"abstract":"Systematic content analysis of messaging has been a staple method in the study of communication. While computer-assisted content analysis has been used in the field for three decades, advances in machine learning and crowd-based annotation combined with the ease of collecting volumes of text-based communication via social media have made the opportunities for classification of messages easier and faster. The greatest advancement yet might be in the form of general intelligence large language models (LLMs), which are ostensibly able to accurately and reliably classify messages by leveraging context to disambiguate meaning. It is unclear, however, how effective LLMs are in deploying the method of content analysis. In this study, we compare the classification of political candidate social media messages between trained annotators, crowd annotators, and large language models from Open AI accessed through the free Web (ChatGPT) and the paid API (GPT API) on five different categories of political communication commonly used in the literature. We find that crowd annotation generally had higher F1 scores than ChatGPT and an earlier version of the GPT API, although the newest version, GPT-4 API, demonstrated good performance as compared with the crowd and with ground truth data derived from trained student annotators. This study suggests the application of any LLM to an annotation task requires validation, and that freely available and older LLM models may not be effective for studying human communication.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"43 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143901247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：