{"title":"内容下架和活动家组织:社交媒体内容审核对活动家和组织的影响","authors":"Diane Jackson","doi":"10.1002/poi3.372","DOIUrl":null,"url":null,"abstract":"Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.","PeriodicalId":46894,"journal":{"name":"Policy and Internet","volume":"3 3","pages":"0"},"PeriodicalIF":4.1000,"publicationDate":"2023-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Content takedowns and activist organizing: Impact of social media content moderation on activists and organizing\",\"authors\":\"Diane Jackson\",\"doi\":\"10.1002/poi3.372\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.\",\"PeriodicalId\":46894,\"journal\":{\"name\":\"Policy and Internet\",\"volume\":\"3 3\",\"pages\":\"0\"},\"PeriodicalIF\":4.1000,\"publicationDate\":\"2023-10-26\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Policy and Internet\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1002/poi3.372\",\"RegionNum\":1,\"RegionCategory\":\"文学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMMUNICATION\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Policy and Internet","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/poi3.372","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMMUNICATION","Score":null,"Total":0}
Content takedowns and activist organizing: Impact of social media content moderation on activists and organizing
Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.
期刊介绍:
Understanding public policy in the age of the Internet requires understanding how individuals, organizations, governments and networks behave, and what motivates them in this new environment. Technological innovation and internet-mediated interaction raise both challenges and opportunities for public policy: whether in areas that have received much work already (e.g. digital divides, digital government, and privacy) or newer areas, like regulation of data-intensive technologies and platforms, the rise of precarious labour, and regulatory responses to misinformation and hate speech. We welcome innovative research in areas where the Internet already impacts public policy, where it raises new challenges or dilemmas, or provides opportunities for policy that is smart and equitable. While we welcome perspectives from any academic discipline, we look particularly for insight that can feed into social science disciplines like political science, public administration, economics, sociology, and communication. We welcome articles that introduce methodological innovation, theoretical development, or rigorous data analysis concerning a particular question or problem of public policy.