How Automated Writing Systems Affect the Circulation of Political Information Online

Bots engage in a spectrum of behaviors online, ranging from policing Wikipedia, as mentioned above (see Krista Kennedy for a writing and rhetoric perspective on these bots), to autoresponding to customer queries, to trading stocks, or to automating shopping for items that are in extraordinarily high demand. We hone in here on the role of bots in the spread and circulation of writing, especially on social networking sites, or SNSs. On SNSs populated by hundreds of millions of people, bots can participate in what one research group called "engineered social tampering" that serve the economic and political interests of their creators (Ferrara, Varol, Davis, Menczer, and Flammini 2). Much of this manipulation comes in the form of advertising, which serves the economic interests of bot creators, but in ways no more insidious than traditional print or video ads. As long as they comply with automation policies on SNSs (e.g., Twitter's Automation rules https://support.twitter.com/articles/76915), bots such as Patron Tequila's Bot-tender, which suggests drink recipes over Direct Message, can boost brand loyalty and engagement, especially among the tantalizing commercial demographic of Millennials (Lull). While Twitter does suspend abusive bots (e.g., those that tweet too frequently, duplicate purposes, target trending topics, or direct message without permission, according to their Automation rules), the increasingly important role of bots in marketing through Twitter and Facebook messaging means that the role of bots in SNS discourse is essentially protected by commercial interests. This openness to commercial advantage comes with vulnerabilities, as James J. Brown, Jr. has argued: once a network opens itself to the outside, it can be used—and abused—by friend and foe alike (Ethical Programs). In arenas of political discourse, the influence of bots can be subtle, and even more so because they often operate in a distributed fashion that doesn't trigger banning from the SNS. Bots have been deployed as fake populations to make certain viewpoints seem far more popular than they are, and to manipulate the emotional tenor of a discussion on social media in an effort to influence users' perception of reality (Ferrara, Varol, Davis, Menczer, and Flammini 3). Often, it is socialbots that do this kind of work, and there is a burgeoning and powerful body of research on socialbots that raise crucial questions about their role in the shaping of public discourse.

Unlike the bots that crawl the web to help search engines function or monitor Wikipedia pages, socialbots are computer programs that exist on social media networks and mimic human profiles in an attempt to convince humans the bots are just more users among many. In other words, socialbots are either designed to pass the Turing Test in social media spaces—or, alternatively, they rely on social media users to not inspect their activity closely enough to tell. After establishing a presence, socialbots can engage in a variety of scripted behaviors in an attempt to manipulate public opinion. They can follow a politician en masse to make it appear as if that politician is much more popular than s/he actually is. They can attract followers and friends, use hashtags, and re-circulate or re-tweet content such as fake news to popularize it. They can repeat information such as fake news, but they can also carry on conversations with human users by mining the Web for relevant text or cloning particular social profiles to simulate legitimacy (Ferrara, Varol, Davis, Menczer, and Flammini). These socialbots introduce more complexity to information circulation and the way we are exposed to information; they are layered on the surface of social networks and through their activity attempt to influence the algorithms that determine a user's information consumption.

Socialbots have thrived on SNSs designed in ways that accommodate bot behavior. As Douglas Guilbeault explains, the vast amount of writing and images online provide the raw material for socialbot creators to generate plausible stock social media profiles and text for socialbots. Short textual exchanges, often with novel abbreviations and typos when humans write them, tend to provide cover for socialbots' infelicities with natural language. Emoticons give them seemingly complex speech with wide latitude for sensemaking. And the removal of humans from much of the editorial process means that socialbots often need to pass muster only with human-decision-simulating algorithms, such as they are themselves. Networks of socialbots following each other in social media platforms can effectively bolster their social media currency and help them pass through both human and algorithmic filters. Because of the ways they can effectively draw on resources and operate in constrained systems, these bots can be difficult to detect. But even if human or nonhuman users do detect them, the design of social networks and their emphasis on popularity incentivizes users to friend and follow bots (Guilbeault). They operate within social network systems and are well-tailored to do just that. Because socialbots thrive under these particular conditions, Guilbeault argues that we must understand them ecologically. He introduces the idea of "platform persuasion," that is, the ability of socialbots to use the automated conversation tools SNSs provide—such as sharing messages verbatim and emoticons—in order to convince users to follow them or read their messages. Their success in these systems means that it's irrelevant whether they might succeed in passing a Turing Test elsewhere. Their ability to persuade is fully enmeshed in the platform in which they operate.

We can understand Guilbeault's model in a longer tradition of rhetorical ecologies. Over the past few decades, rhetorical theory has moved away from models of persuasion that consider how a rhetor with a single text might persuade an audience and toward models of rhetorical ecologies that seek to understand how networked people experience multiple encounters with a variety of other people, texts, and objects over time (e.g., Cooper; Edbauer). And whether we conceptualize literacy practices as happening in rhetorical ecologies, communities of practice, or activity systems—to name a few influential models in literacy studies of past decades—thinking about texts as plural, interconnected, and embedded in complex systems has been a dominant trend. Rhetoric as a field has also considered how the delivery and circulation patterns of online writing shape the production practices of writers as they anticipate reaching their audiences (Ridolfo and DeVoss; Brooke; Eyman). If we think about web content through these kinds of ecological frameworks that have been part of rhetorical thinking in composition, we see how the presence of bots designed to operate in SNSs shifts the situation not only for those who wish to deploy them but also for human writers who want their words to contend for attention in this space.

---

At least since Claude Shannon's work in communication and noise, we have known that message delivery can be plagued by transmission errors and false or irrelevant information. James Gleick discusses the perpetual challenges of filtering good information from mixed channels of good, bad and irrelevant information, a process that changes but never seems to be obviated by new technologies of communication. In the calculated online writing ecology that we focus on here, we see new kinds of interference that degrades information quality—automated amplification, spoofing followers of accounts, and the burying of genuine human political discourse. And when we consider the influence of bots, we need to consider how they both amplify certain kinds of information and, at times, actively work to suppress other kinds. To echo our opening to this article, human political discourse has always been shaped by material and nonhuman conditions of writing, including technologies that amplify or suppress. We can now consider how automated amplification and suppression practices work to shape online discourse, particularly in the case of bots.

Scholarly studies of Twitter after the US midterm elections of 2010 mark some of the first efforts to measure bot participation in the spread of (mis)information. Such studies were largely produced by specialists in computer science and networked systems research with the aim of identifying characteristics of bot "abuse" in social media for the purpose of developing digital tools that would help users identify bots (http://botornot.co/ provides an example of one tool that has been since published). Bot identification can be tricky for a number of reasons, and as Guilbeault has argued, the design of social media platforms encourage kinds of discourse to which bots are well fit. The uniform templates of SNSs encourage a uniformity of composition, which means bots do not have the challenge of "creating novel forms of self-expression" (5009). In the SNS ecosystem, bots are excellent conduits for spreading (mis)information.

Indeed, some studies of the 2010 midterm elections voiced strong concern that bots were being used to spread fake news and misinformation. P. Takis Metaxas and Eni Mustafaraj tracked a 2010 instance where nine twitter accounts began messaging other users with a URL to a website that smeared one of the political candidates with fake news in a Massachusetts election (some time later the Republican group of "Swift Boat" fame outed themselves as the creators of the site). Metaxas and Mustafaraj found the bots pushing the URL were retweeted enough by humans that it reached more than 60,000 users while rising to the top of Google results in searches for the candidate's name. This prompted the authors to worry about the future exploitation of automated writing tools if actors could reach tens of thousands of users with "a few minutes worth of work, using automated scripts, and exploiting the open architecture" of sites like Twitter (6). Since 2010, bots and their deployment have grown more sophisticated and arguably more influential in political discourse as candidates increasingly rely on Facebook and Twitter to communicate to their constituents. Yet, as Douglas Guilbeault and Samuel Woolley warned on the eve of the 2016 US presidential election, the outsized influence bots could have on political outcomes has gone completely unremarked on by the US Federal Election Commission, whose job it is to oversee financing of elections.

Alessandro Bessi and Emilio Ferrara have been involved in developing bot detection software and have published the most extensive account of how bots intervened in the political discourse of the 2016 election cycle in the US. Published in the internet studies journal First Monday, their findings are worth recounting here because they offer literacy studies a detailed glimpse of the extent to which online writers were writing with and among a very active social bot population during the election. Some of the important findings relevant to our discussion: 1) Roughly 15% percent of the Twitter population they sampled consisted of bots, and those bots sent roughly 20% of the total tweets related to the election. 2) The social bots had a number of different functionalities. The authors list at least eight, including searching Twitter and retweeting relevant content, adding users tweeting about a topic to public lists, and automatically following relevant users. 3) Humans did retweet bots in a substantial number of instances, which suggested to the authors that "bots are being very effective at spreading information in the human population" (n. pag.). After an account of how bots became active participants in online election discourse, Bessi and Ferrara conclude that bots create three tangible issues for political speech: a skewed distribution of influence, further polarization of political speech, and the continued spread of misinformation. Presumably bots could also be leveraged to spread factual information too, but as we've seen of late, a hyper-polarized political climate makes it difficult to agree on what constitutes factual information.

Another concern about credibility that bots introduce to our current political climate is related to human users' followers, which are often used as a shorthand to determine a person's popularity with the public. The most high profile example is that of Donald Trump, who has cited his large Twitter following as an indicator of his popularity. Speaking at a tech conference in May 2017, Hillary Clinton charged that his follower numbers were being goosed up by fake accounts, although the claim that he had a recent surge in followers was debunked by Snopes, a prominent fact-checking website ("Did President Trump's Twitter"). Still, some web tools routinely estimate that roughly half of Trump's thirty million followers appeared to be fake, or bots—something that has been widely reported in the press (Bort). In a recent check conducted by the authors, the website Twitter Audit, which analyzes a user's followers to determine which are fake or real, indicated that 45% of Trump's are bots, 10% of Barack Obama's followers are bots, and 37% of Hillary Clinton's are bots (as of July 8, 2017). But these numbers may not be accurate: a Twitter spokesman said that the widely-cited Twitter Audit website's methods for determining bots were flawed (Johnson and Gordon), and, as discussed above, it's hard to determine just how many of the millions of followers of these politicians are fake.

Regardless of the percentage, bot followers now seem to be part of the structure of high profile accounts. It is usually unclear whether users themselves or their representatives are organizing or buying these bot followers to boost apparent popularity, or whether they are coordinated independently. There are many services that sell bot audiences to boost a user's status on Twitter or other SNSs (e.g., MonsterSocial), and since bots themselves aren't against the terms of service of SNSs such as Twitter, these services are relatively affordable and above board. The business of bots is not well-known by the public though, and so their role in boosting follower numbers can be misleading. For instance, Donald Trump has bragged about his millions of followers without noting that so many of them are suspected bots, perpetuating the impression that he has more human followers than he does (Bilton).

A study by Jacob Ratkiewicz et al. warned of the use of bots for "astroturfing," a term used to indicate a highly coordinated and well-funded campaign designed to look as if it had instead emerged from grassroots efforts. The authors argue that social networks can be exploited so that an attacker "can easily orchestrate a distributed effort" to mimic the organic spread of information through social networks (297). As an example, work by the Digital Forensic Research Lab examined the followers of fringe French-language, Russian-allied political accounts on Twitter and discovered that their followers appear to be highly active retweeters and responders, in comparison to mainstream French media accounts. These fringe followers tended to align with Russian interests or French nationalism and opposed the centrist candidate Emmanuel Macron in the 2017 French presidential election. They also appear to have been highly automated accounts (Nimmo and Czuperski).

A slightly different astroturfing example can be found preceding the US election. While he was a presidential candidate, Trump's tweets were highly visible, and an algorithm in Twitter's mobile app that prioritized the display of replies based on rapidity of response rather than popularity incentivized political actors to compose scripts that would autorespond to Trump's tweets. When the algorithm changed to (apparently) consider aspects of the accounts responding and popularity of the response, these bots disappeared or became irrelevant and thus invisible in Twitter replies (Matsakis). Guilbeault and Woolley claim that one-third of pro-Trump tweets and one-fifth of pro-Clinton tweets between the first two presidential debates originated from automated accounts, enough for either candidate's online perception to be influenced by bots. In the ways that bots interact with media outlet and human users, they can affect the tenor of political discourse through amplifying or lending credibility to fringe politics and high profile accounts. These effects shift rapidly with political events and changes in the implementations of the algorithms that bots attempt to game.    

A high-profile example illustrates how bots can demonstrably influence politics. While serving time for his crimes of digital espionage, Andrés Sepúlveda spoke to Bloomberg Businessweek about how he used digital means to hack elections in Latin America for over a decade. In addition to hacking into candidates' communication channels, he took advantage of the fact that expressions of candidate support on social media were often read as more genuine than advertisements or overt propaganda. He wielded both high-end fake Twitter profiles, which had been maintained for over a year and seemed more believable, and sheer quantities of lower-end profiles—30,000 that he could deploy to shape conversations, for example, about the 2012 Mexican presidential candidate (and winner of the election) Enrique Peña Nieto's plan to end drug violence. This dual strategy of high- and low-quality bots is illustrative of how bots can be coordinated to spread news quickly and believably. The reporters write,

[Sepúlveda] knew that accounts could be faked and social media trends fabricated, all relatively cheaply. He wrote a software program, now called Social Media Predator, to manage and direct a virtual army of fake Twitter accounts. The software let him quickly change names, profile pictures, and biographies to fit any need. Eventually, he discovered, he could manipulate the public debate as easily as moving pieces on a chessboard—or, as he puts it, "When I realized that people believe what the Internet says more than reality, I discovered that I had the power to make people believe almost anything." (Robertson, Wiley, and Willis)

The fact that many people perceive online discourse as more immediate and genuine than media reporting opens the door for political interventions such as Sepúlveda's, suggesting that the perception of online discourse is not fully accounting for the ability of bots to suppress, amplify and distort information.

As bots compose, tweet, and retweet according to whatever script they've been given, they influence the tenor of the discussion and the kind of information that reaches people not simply by motivating information and encouraging it to go viral, but also by suppressing other kinds of information. If we understand literate people as moving in and across environments saturated by the texts and objects they use to create meaning, then information suppression matters as much as degradation: the texts that don't appear in an environment can be as notable for their absence as the texts that do appear. In the liberal-democratic model of public print discourse, historical examples of textual suppression are essentially censorship and are framed in largely negative terms. Text suppression is not new, and it certainly happens on more quotidian scales as well: collections decisions for libraries, prominent displays of books on endcaps, editorials in newspapers that may not reveal all of the writer's connections, and so on. Woolley has profiled the way bots have interfered in global politics, concluding that bots have been able to "suppress free expression," suffocate democratic speech, and demobilize activist groups.

A study on the Syrian civil war shows additional ways in which this suppression happens: for example, misdirecting attention to trending world events by flooding a hashtag with irrelevant or inaccurate information. During a six-month period in April 2012, when the Syrian civil war was a globally discussed event, Norah Abokhodair, Daisy Yoo, and David W. McDonald tracked a networked group of 130 social bots that were tweeting and re-tweeting about the war. Active for thirty-five weeks, this botnet produced thousands of tweets per week the researchers collected and analyzed for content and purpose, which helped them classify bot behavior and the attempts bots make to intervene in public discourse. The botnet had "generator bots," or core accounts that tweeted thousands of times per week, tweets that were then retweeted by less active bots in the network. The tweets largely consisted of news, opinions, and spam. One particular behavior of these bots indicates how botnets can misdirect. In a number of instances, one of the bots would tweet photos of Hurricane Sandy and its distance from New York, but the bot would include the hashtag Syria, thus anyone searching the hashtag in search of tweets about the Syrian war would have to wade through the irrelevant content. The news of Hurricane Sandy propagated by the bot works to divert the audience's attention away from tweets that criticized the Syrian regime. Abokhodair, Yoo, and McDonald refer to this as "calculated misdirection," borrowing a term from magicians who attempt to divert the audience's attention while orchestrating a trick.

This kind of "misdirection" has long been a part of the political propaganda game—for example, in the Chinese government's online strategy, although in that case, the propaganda issues from people. The popularly called "50 cent party," or wumao, post coordinated, pro-government statements on Chinese social media accounts, effectively censoring statements critical of the government by providing conflicting accounts. Gary King, Jennifer Pan, and Margaret Roberts estimate that these posters account for about 448 million comments a year on social media, or about 1 post per 178 on commercial Chinese social media sites like Sina Wibo, although the number spikes during sensitive time periods. The study reports that these individuals, mostly government bureaucrats, don't engage with critical statements so much as they attempt to flood networks with pro-government sentiments. In 2016, the Correct the Record PAC launched an online campaign to counter anti-Hillary Clinton posters and promote issues most favorable to her candidacy—ostensibly to "correct" the high volume of negative rhetoric directed at her, exemplified by hashtags such as #crookedhillary. Correct the Record posters were, it appears, human, and mostly responded with positive statements about Hillary, but their open presence as paid propaganda may have undermined their efforts, especially since they were able to coordinate directly with the Clinton campaign due to a loophole in campaign finance law pertaining to online activity (Foran). Pro-government human labor is easily obtained in China and payment for pro-Hillary posters may have been a small expenditure for her well-financed campaign. But on a platform where numbers matter, bots can scale up propaganda and obfuscate political viewpoints in public discourse much more cheaply. Again, given these examples of human intervention, the difference that bots make in political discourse appears to be in terms of scale rather than strategy.

What Bots Mean for Literacy Studies

While work in digital rhetoric has long noted that writers compose with and among software programs and code that act on their writing in subtle and nuanced ways (e.g., Rieder; Leblanc), we must ask whether current automated writing systems such as bots are fundamentally different from other code-based influences. Code can define a template for a social media site that will circumscribe possible kinds of writing; scan for copyright violations in a text and automatically take it offline; or filter and flag texts based on keywords associated with, say, commercial entities or terrorism. Although we can think of the bot intervention recounted above as related to these other code-based influences on online writing, we argue that automated writing systems such as bots mean that code is making its way into the composing lives of online writers in a new way—compelling them to write alongside and contend with scripts that are engineered to attract or distract both human and algorithmic readers. Human writers must interact with a legion of programmed writers, sometimes controlled by shadowy actors manipulating the circulation patterns of text in the online writing ecology.

One of the foremost voices of sociological studies of algorithms, Gillespie is quite clear on the stakes of the dynamics profiled above: "If our participation in public life [...] is being determined, or at the very least adjudicated, by algorithmic systems, then we must know more about the assumptions upon which they are based, the information about us upon which they act, the priorities they serve, and the ways in which they shape, distort, or tip the process" ("Algorithmically Recognizable" 64). As Andrés Sepúlveda noted, the general public may perceive online discourse as more "genuine" than mass media while being largely unaware of the influences of automated writing systems in mimicking "genuine" human discourse. Mass media seems to reinforce this assumption when individual Twitter accounts are quoted in news stories, stories whose titles often begin with "The Internet is Reacting to…" or "The Internet Says…." These articles are a modern day emulation of "man-on-the-street" interviews—although it is much more difficult to verify whether the quoted writing was produced by humans. In part, we've written this article to help literacy and composition studies take a small step forward as it considers the question of what it means when the "we" of "we must know more" about algorithms represents ordinary writers and readers. Given these misperceptions, literacy and rhetoric educators may bear some responsibility to point to the algorithms driving the complicated automated systems that mimic and shape writing online.

But the pressing question that will need to be answered is how people can engage in responsible discourse in the face of rapidly evolving technologies that can be exploited and can also offer a bullhorn to the most detestable of political positions. And the answers can be in the mundane ways we approach technology in the classroom. Teachers of digital writing and rhetoric have begun to wrestle with this question in different forms, as digital tools and the ecosystems in which they function have cracked open, and made accessible, a slew of affordances that were formerly blackboxed for writers—affordances writers might have had little call to consider in the age of print. Even a simple blog provides opportunities for writers to make decisions about design (typography, color combinations, headings, structure, and so on), accessibility (metadata and captioning), and audience experience. Although digital tools provide these options, it is also possible to ignore most of the issues, compose in a WYSIWYG interface, and click to publish. Given the difficulties of writers in learning to control their syntax, thought, and evidence well enough to express themselves, bedrock activities of composition's pedagogical project, how much of the complexity of online writing should teachers be expected to know and to impart? To what extent do we rely on experts to close the exploitable gaps in the contemporary media ecosystem? What do writers need to know, and what competencies do they need to have to maintain a critical disposition to their own work? What can they give themselves permission to not know? What processes can they blackbox or use WYSIWYG interfaces to handle? Our discussion here about automated writing systems reveals that the scene is even more complicated than previously thought. Compounding the longstanding questions of digital pedagogy are now the challenges of optimizing one's writing and publishing practices to reach intended audiences and dealing with the crushingly complicated social dynamics that are part of online writing ecosystems. To what degree should writers accommodate algorithmic filters when they plan for the future circulation of their texts? How should they account for the copresence of bots in their online writing ecosystems?

These may be largely contextual questions that writers can address as they happen, and some of them writers will learn through experience and intuition. But as teachers of writing, we must consider to what degree should we be embedding questions of automated writing systems in our pedagogy. As researchers of writing, how should we account for the bots and algorithms in our online environments? We do our best to equip students with competencies they can deploy, grow, and adapt with as much flexibility as possible, but with finite time and resources, the question of what dynamics students should prepare for in digital composing is complex. And research in writing seems to add complexities to our understanding of the process of written communication with each new study published. Many of the studies we cite above suggest that automated writing systems such as bots and algorithmic filters will play an increasingly important role in online writing. Bots and their relationship to fake news have been a convenient manifestation for our larger argument here: that questions related to algorithms and automation are already a core part of literacy and play a profoundly important role in how we encounter texts as readers and writers. Our approaches to this fact in our teaching and research may take different forms in writing studies, but regardless of approach, we will need to acknowledge the robots among us.