Modes of Tracking Mal-Info in Social Media with AI/ML Tools to Help Mitigate Harmful GenAI for Improved Societal Well Being

Abstract

A rapidly developing threat to societal well-being is from misinformation widely spread on social media. Even more concerning is ”mal-info” (malicious) which is amplified on certain social networks. Now there is an additional dimension to that threat, which is the use of Generative AI to deliberately augment the mis-info and mal-info. This paper highlights some of the ”fringe” social media channels which have a high level of mal-info as characterized by our AI/ML algorithms. We discuss various channels and focus on one in particular, ”GAB”, as representative of the potential negative impacts. We outline some of the current mal-info as an example. We capture elements, and observe the trends in time. We provide a set of AI/ML modes which can characterize the mal-info and allow for capture, tracking, and potentially for responding or for mitigation. We highlight the concern about malicious agents using GenAI for deliberate mal-info messaging specifically to disrupt societal well being. We suggest the characterizations presented as a methodology for initiating a more deliberate and quantitative approach to address these harmful aspects of social media which would adversely impact societal well being. The article highlights the potential for ”mal-info,” including disinfo, cyberbullying, and hate speech, to disrupt segments of society. The amplification of mal-info can result in serious real-world consequences such as mass shootings. Despite attempts to introduce moderation on major platforms like Facebook and to some extent on X/Twitter, there are now growing social networks such as Gab, Gettr, and Bitchute that offer completely unmoderated spaces. This paper presents an introduction to these platforms and the initial results of a semiquantitative analysis of Gab’s posts. The paper examines several characterization modes using text analysis. The paper emphasizes the developing dangerous use of generative AI algorithms by Gab and other fringe platforms, highlighting the risks to societal well being. This article aims to lay the foundation for capturing, monitoring, and mitigating these risks.