How Automated Writing Systems Affect the Circulation of Political Information Online

This article argues that fake news is only one instantiation of a shift that literacy studies will need to reckon with to understand how people encounter texts on an everyday basis. It argues that looking at the information ecologies in which fake news circulates reveals a shift to the reliance on computational and automated writing systems to circulate texts and amplify their distribution. The article critically synthesizes existing literature and provides key examples of how algorithms and bots were deployed strategically to pollute the media ecology with fake news in the time immediately preceding the 2016 Presidential election in the United States. The argument ultimately raises a series of questions that literacy studies will need to confront given the influence of computation in contemporary information environments, including asking: how can people engage in responsible discourse in the face of rapidly evolving and exploitable technologies?

objects: the material barrier of access to writing instruments, printers' licenses, literacy education, publishers, publicity and distribution channels, and so on.The same holds true for web environments.Despite the romantic 1990s idea-one that still persists-that writing on the web circumvented "gatekeepers" to access and distribution, online publishing can be as heavily mediated by gatekeepers as the print market, although the online market can have different players and pressures: corporate giants like Amazon, algorithms that determine bestseller lists, and automated reputation measures (Laquintano).Taken further: when we write on digital platforms, it is now impossible to untangle textual writing from code writing (Vee; Rieder).And parts of that code work to mimic or automate human activity to influence the kinds of texts we encounter.
With the reminder that writing contexts are always mediated, we begin with the uncontroversial assertion that literacy is not simply about producing or decoding alphabetic text; it is also about processing written and multimodal information, judging it, assimilating it and making sense of it in relation to other texts and worlds.It follows, then, that the shift from human to algorithmic production and evaluation of texts represents an important shift in literacy practices.Working with this special issue's theme of "fake news, " this article explores the question of why many people encountered it to begin with: what's the information environment in which this "news" is able to circulate with such vigor?To be sure, fake news has an extensive history (Soll).In the supermarket checkout line, the Weekly World News has long provided us with updates about Sasquatch and vampires, but there has never been much conversation in professional composition literature about whether fake news sources like The Onion threaten the foundations of democracy.Instead, we argue here, the current iteration and debate around fake news is indicative of a larger change in our contemporary writing ecosystem, now driven by the growing importance of algorithmic systems of text production, distribution, circulation, and curation.These systems penetrate much deeper into our literate lives than tabloid stands have, and the phenomenon behind the current uptick in fake news is how we now write with and among code-based systems such as bots, algorithms, and social media platforms.
Below, we outline how code-based technologies create the ecosystem in which fake news spreads, with particular attention to how automated writing activity (i.e., bots) and algorithms on social media platforms can influence this ecosystem.These software-based systems often replace or mimic human editorial and writing practices, and so we sometimes refer to them collectively as "robot writers, " following Bill Hart-Davidson.The idea of writing "robots" also highlights the fact that information processing can be automated just as easily as the processing of physical materials.We approach our questions about these automated writing systems through a critical synthesis of a rapidly emerging body of literature that attempts to determine how bots, networks of bots, and algorithmic editorial systems influence public discourse by intervening in text distribution and circulation practices.Qualitative, quantitative, and theoretical research is now being produced in fields like media and information studies, sociology, political science, and computer science.Here, we concentrate on the issues most salient to the literacy practices of everyday people and the ways their activity might overlap with, or be influenced by, digital environments rife with robot writers as robot and human writers mingle on social media platforms.
Our exploration builds on previous work in composition studies that points to automated reading and writing practices, including computer software and scripts that attempt to grade writing (Perelman), read writing in ways that provide feedback to writers (Omizo et al.), determine rhetorical moves (Omizo and Hart-Davidson), edit writing (Kennedy), and produce writing (Brown, Ethical Programs).In the coming years, software to automate particular writing tasks will grow more sophisticated and widespread, perhaps continuing to displace human editors and writers.Noting that we are already writing in the age of robots and algorithms, we show how and why those automated writing systems helped to heat the fake news flashpoint and what implications these systems have for literacy in the twenty-first century.Anxieties as well as real changes in writing and reading practices have always accompanied changing technologies of writing, and automated writing systems of bots and algorithms offer a new slate of changes on an accelerated scale.Specifically, we examine the shift from human editorial processes to algorithmic ones and the operations of bots in social network ecosystems, two critical phenomena in the circulation of fake news online.We argue that knowing the robots are with us is critical for understanding writing and reading in our current landscape and then raise some questions that literacy educators will need to consider when they think about how they prepare writers and readers to access and interact with information online.

From Editors to Algorithms
The anxiety over so-called "fake news" is a recent installment in a longer history of reactions to changes in writing and publishing technologies and the ways that they influence how information circulates to the public.In an example made popular in Jay David Bolter's foundational Writing Space, a monk in Victor Hugo's novel Notre-Dame de Paris worried that the rise of the print book would destroy the church's authority (Bolter 1).When cheap printing became available in the 18th century and allowed more people to publish material such as political pamphlets, Alexander Pope wrote in The Dunciad of a kingdom of Dulness and Dunces overrunning Great Britain.More recently, when it became clear that the web was going to offer an unsurpassed level of cheap access to publishing, some commentators celebrated freedom from "gatekeeping" editors, while others lamented the low quality of information that was likely to circulate without vetting from publishers.
When it comes to threats to the integrity of public discourse, there are a number of differences in our present case compared with anxieties of the past.Although social networks and online forums, where much of public discourse now takes place, enable greater access to participation for everyday writers in ways not dissimilar from those of cheap presses, the current scene includes more aggressive intervention by nonhuman actors, such as bots, that generate writing.Humans are, of course, usually responsible for authoring the computational processes that generate writing such as we address here (see Brown, "The Machine"), but by making certain aspects of online writing computational, human authors can typically operate with greater speed, scale, and autonomy.Thus, automated writing systems are potentially more disruptive than the previous threats to print economy institutions that adjudicated information.
As the term "algorithm" has grown in importance in contemporary culture, it has moved away from the more specific definition that may be used by a software engineer to designate simply a "series of steps for organizing and acting on a body of data to quickly achieve a desired outcome" to include the socio-technical constellations that surround their production and deployment (Gillespie,"Algorithm" 19).As Tarleton Gillespie argues, scholars interested in the impact of computation on culture now use "algorithm" and "algorithmic" to include "not just algorithms themselves, but also the computational networks in which they function, the people who design and operate them, the data and users on which they act, and the institutions that provide these services" (25).Like these scholars, what we are most concerned about is not the precise algorithm itself but the influence of computational "procedure into human knowledge and social experience" (25).While the term "algorithmic" tends to encompass a broad sociotechnical constellation, the definition of a bot tends to be deployed more precisely: computer code designed and used to automate a variety of tasks.Social bots specifically automate tasks on social media, and they are designed to mimic humans and interact with them in order to alter human behavior.Malicious social bots are a subcategory of bots that "mislead, exploit, and manipulate social media discourse with rumors, spam, malware, misinformation, slander, or even just noise" (Ferrara et al.).
Software-based algorithms and bots play incredibly important roles in determining what kinds of information are circulated online, and to what degree.These algorithms are influenced by a mix of human and nonhuman actors: the choices of engineers and editors, readers' preferences as measured by clicks, scrolls, and shares, and groups of professionals and amateurs who try to exploit vulnerabilities in the information ecosystem for political or economic profit.When we shift to software that arbitrates, circulates, and amplifies information of any sort-including fake newssingular human writers and readers, regardless of what they have to say or want to know, are forced to play a new game, one that includes encounters with automated writing systems.
Algorithmically aided filtering affects readers as much as writers.In the month before the US presidential election, Facebook-a major site using personalized algorithmic filtering to determine what readers see-saw more engagement with fake news than with mainstream news, including a headline that the Pope had endorsed Trump (Silverman).One challenge for potential readers in this new system is that malicious actors have learned how to exploit many of the tenuous ways we've learned to discern what's true online.Even if a story has been shared a million times on social media, and if it is found on a website that looks and sounds newsy, and if it is repeatedly linked from a popular hashtag, there's no guarantee that it's a credible story.Fake news sites often emulate the look and titles of professional news sources, e.g, the "Denver Guardian, " which falsely reported that an FBI agent in the Hillary Clinton email investigation was found dead in a murder-suicide (Silverman).Bearing the trappings of news allows stories to circulate with the veneer of credibility, aided by confirmation biases and easy mechanisms for sharing, and then magnified by popularity algorithms that push them into the purview of more readers.Some enterprising young Macedonians noticed during the 2016 presidential election that their distance from the US didn't matter when they

"Software-based algorithms and bots play incredibly important roles in determining what kinds of information are circulated online, and to what degree. "
launched popular and lucrative fake news sites to collect American clicks and advertising dollars (Silverman and Alexander).The global nature of the Web means that a writer's physical location is almost irrelevant to the circulation of information (real or fake) online.Americans circulate junk or fake news at a much higher rate, but people in France, the UK, and Germany were also circulating fake or junk news prior to their recent elections (Howard et al.).
There were humans writing the major fake stories that circulated prior to these political events, and their motivation was, unsurprisingly, money.In the wake of the US presidential election upset, reporters for both National Public Radio and the Washington Post tracked down fake news writers, who admitted that money was at least part of what motivated their work.Jestin Coler, whose fake news story about people using food stamps to buy pot resulted in proposed legislation in the Colorado House, claims to write to "'infiltrate the echo chambers of the alt-right'" by posting stories that confirm biases present in that community.He and his team of writers and sites found it easy to make a story go viral: "'It was just anybody with a blog can get on there and find a big, huge Facebook group of kind of rabid Trump supporters just waiting to eat up this red meat that they're about to get served'" (Sydell).Paul Horner worried that he helped Trump win the election because of stories he planted about paid protesters, the Amish voting for Trump, and "'crazy anti-Muslim stuff, '" although he contended his writing should be likened to The Onion's (Dewey).Both Horner and Coler claim that the tendency to avoid fact-checking helped stories bolstering Trump to spread rapidly on social media sites.
These stories also found their way into our news feeds not because a human editor thought they were relevant or important; in most cases, they began circulation with human writers sending a story out to an automated network like a botnet (a group of bots working in concert and controlled by the same owner) and then, if they were lucky, triggering a platform's algorithm to feature the story more prominently in people's feeds.We might think of some fake news as a symptom of an automatic system much like flash crashes on Wall Street, the rapid market fluctuations now symptomatic of high-volume algorithmic trading practices.Numbers drive social media sites in the same way that they drive finance, and in these environments, computational systems have an edge-although they are still influenced by the usual human motivations.
The media scholar Tarleton Gillespie has called the umbrella shift in which these examples happen a shift from an editorial model of information consumption and production to an algorithmic model.The availability and increased material accessibility of computation has encouraged this shift, but much of it has happened in response to the increased volume of writing online that has demanded it.Too much writing and information circulates online for humans alone to filter it; estimates place the current number of web pages in the billions.From the perspective of literacy studies, this shift means that algorithms now play a critical role in the kinds of texts we encounter in everyday life, and in certain domains perhaps even a more critical role than other humans (though these algorithms are

"One challenge for potential readers in this new system is that malicious actors have learned how to exploit many of the tenuous ways we've learned to discern what's true online."
being influenced by human activity).As with earlier shifts in editorial and distributive mechanisms, some may argue that the quality of information has decreased as algorithms compete with editors in importance and as the volume of writing has become too much for humans to read, much less filter.The circulation and discussion over "fake" or "junk" news is one indication of this decreased quality.
This space where clicks and recommendations and trending topics are determined algorithmically is a space of "calculated publics, " according to Gillespie, by which he means that computational processes don't simply record audiences but actively produce them as well.When algorithmic logic overtakes editorial logic, we move from questions of ineffable human taste to what kinds of information can be quantified and returned to information seekers.Gillespie notes that professional editors curated, and thus directed us to, much of the reading material we encountered across the twentieth century, but as literates and information seekers reading and writing online, we are increasingly assigning that function to computer algorithms-that is, search engines, newsfeeds and recommended links-which register human activity and then return information to us based on a slew of different calculations.For example, the kinds of participatory activity common on social networks provide us with information partially based on whether those in our network find a text worthy of attention (an indicator of value), and search engines provide us with information partially based on calculating the attention a texts has received (whether other websites have linked to that content or re-posted it, etc.).
The shift from the editorial model to the algorithmic model is not complete; algorithms still index an incredible amount of information produced via editorial intervention, like newspapers, and human editors often still make decisions about when automated systems such as bots are allowed intervene in writing (e.g., to protect entries or add signatures on Wikipedia, according to R. Stuart Geiger).However, even in this incomplete state, algorithmic logic and its various implementations affect the content of what we read, even among institutions that have heavily relied on editors.Note the difference, for instance, between an online newspaper that can track clicks and time spent on each story and a paper newspaper that measures customer consumption only by subscription and letters to the editor.The online newspaper has the opportunity to employ usage data and analytics to shape future content strategies to have greater appeal to their audiences.And, at least in other domains, there is evidence that content providers are doing exactly that.Blake Hallinan and Ted Striphas, for example, document the way in which Netflix has begun to plan content and programming based on analysis they have done on audiences and usage patterns.Gillespie makes a bold assertion about the consequences of this shift to algorithmic intervention in information-seeking practices: "that we are now turning to algorithms to identify what we need to know is as momentous as having relied on credentialed experts, the scientific method, common sense, or the word of God" (168).Our information filtering processes as readers, and thus the ways we hope to reach readers, as writers, have fundamentally changed.
In the same way that human editorial processes have been traditionally hidden from ordinary readers, so too are many of the same calculations that influence the circulation of content.The online distribution systems that circulate and amplify our news stories most are often social media platforms such as Facebook, Twitter, and YouTube, which use closed algorithms that influence what users see and in what order.The Pew Research Center reported that about 62% of US adults got news on social media in early 2016, with about 44% using Facebook.Pew noted that 18% got their news from social media often, and a more recent study in the Journal of Economic Perspectives indicated that 14% of respondents found social media their "most important source" of news (Allcott and Gentzkow).Yet, in one study, 62.5% of Facebook users were unaware that their content was filtered via computational algorithms (Eslami et al.).There are some immediate risks here, not least of which is the highly tailored information customization that can insulate readers and writers from diverse viewpoints and lead to polarization.One of the starkest examples of this polarization is the "Red Feed, Blue Feed" graphic created by the Wall Street Journal (Keegan).This is an interactive web installation that allows users to view a conservative and liberal Facebook feed side by side on a variety of issues such as immigration and gun control.Eli Pariser warned of the problems of such "filter bubbles, " arguing that social networks invisibly curated our feeds to tell us what we wanted to hear and what we already believed.
Companies blackbox (deliberately hide) their algorithms for competitive advantage, but also because gaming their proprietary algorithms would be easier if they were transparent.An entire industry is built around helping people or companies to increase their visibility on Facebook and other algorithmically-governed platforms; search engine optimization (SEO) companies attempt to reverse-engineer the algorithms, guessing at what works.Their strategies last until Facebook and other social networking sites (SNSs) change their algorithms again-and what we have then is a Red Queen's race, where both sides move fast but essentially stay in the same place.For instance, although Facebook says their algorithmic techniques have improved at removing recidivist accounts that promote terrorism, they note that "[t]his work is never finished because it is adversarial, and the terrorists are continuously evolving their methods too" (Bickert and Fishman).While algorithms change often, Nick Seaver points out that we cannot think of a platform's algorithm as being unitary even at a given point of time.Platforms often employ multiple algorithms at once-different ones for different groups of users, or through A/B testing, where trial algorithms essentially undergo live beta testing."Personalization" of search engines such as Google mean that different results are returned to people based on their browsing history, which likely has more to do with targeted advertising than enhanced user experience (Feuz, Fuller, and Stalder).Moreover, even if we had full access to the algorithms governing social media sites, the use of complex machine learning in the development of these algorithms means that even their engineers cannot fully explain their behavior (Seaver).Understanding the workings behind the distribution systems of social media platforms is further complicated by the fact that they are not fully automated systems.They are instead hybrid, as the platforms enlist human labor to filter, report on, and censor content such as graphic videos or incendiary speech (Roberts).Google, Facebook, and Twitter have all considered or deployed methods to crowdsource the review of the huge volume of user-generated content they deliver in order to combat false information (Dwoskin).
Thus, in the current literacy scape, editorial logic and algorithmic logic converge in foggy wayswhich are difficult for individual users to know-and then together condition the information we encounter online.This includes the editorial decision-making processes of corporate media giants, the technological implementations of Silicon Valley behemoths, the widely distributed activity of humans processing information, and the history and activity of the individual user.With the deep integration of algorithms into major sites where we write and receive news, we are experiencing a significant shift in how we are exposed to information and how we circulate the texts that are important to us.But it is seemingly a messy time where models of information publishing and delivery overlap and compete, where values of public discourse compete and conflict with values of individual users and the corporations who create platforms for participation.For this reason, Gillespie notes that "we cannot simply study algorithms and their effects; rather, we must study the interactions between providers of information and algorithmic assessors of information, sometimes a confluence of interests and sometimes a contest, and the results that these interacting forces generate" ("Santorum" 75).
This complex intersection between algorithms, effects, and contested interests is perhaps nowhere more apparent than in the ways that minority groups are represented and discussed online.As Safiya Umoja Noble argues, human decision, shaped as it is by racial stereotypes and cultural assumptions, gets embedded in search algorithms through feedback for ranked results, and is then delivered in a naturalized, seemingly objective package-and thus can be even more damaging than if it were presented in an obviously racist way.For example, Kabir Alli's YouTube video of his Google search for "three black teenagers, " which resulted in mugshots, and "three white teenagers, " which resulted in stock photos of happy teens, went viral in 2016, and many accused Google of racism.Google's algorithms reflect a racist society, and Noble contends that Silicon Valley's lack of diversity is one reason algorithms like Google's fail to recognize and counteract such racist results.People depend on search results to be accurate, and Noble says "'they are when you are looking for the hours for the local Starbucks.But when you are looking for information, ideas or concepts, the algorithm often fails us deeply and yet society does not really see that the algorithm as a failure except when these kinds of egregious moments happen'" (Guynn, sic).Human review has problems, too, especially in the context of SNSs that serve international communities with widely differing cultures and norms.A recent report from ProPublica noted from Facebook's internal training documents that human adjudication of account banning and post censorship was governed by rules that protected large groups such as white men, but not what Facebook considers "subgroups" such as black children or female drivers, demonstrating the strange and sometimes disturbing balance human and algorithms make in filtering information in online and by default international spaces (Angwin).
To be sure, there are small-scale user attempts to control the quality of information that flows through the web, much of which employs algorithmic logic even as it is subjected to it.To maintain their integrity and userbase, online communities must grapple with increased volume of writing, some of it by bots or human spammers taking advantage of the ways that an algorithmic logic filters information.Online writers develop conventions of "micro-celebrity" to help minimize their exposure to low value content (boyd and Marwick), and self-publishing book communities-in concert with website owners-develop numerous conventions to help circulate valuable content and suppress chaff (Laquintano).With help from website owners, these conventions can be enforced in the code of the community's online forum and also with social norms and crowdsourced filtering.
Additionally, community responses to automated writing often utilize automated methods similar to the ones that threaten them: they fight bots with bots.Wikipedians have developed a number of tactics to control the quality of information on the site.The site keeps revision histories they can reference when examining factual disputes, bots patrol the site for signs of vandalism, and pages for controversial topics are locked and can only be modified by trusted editors.Thus, human writing and code writing can be united in a mission to preserve a community's discourse and identity.
What the fake news epidemic of the 2016 US presidential election showed was that these distributed and hybrid gatekeeping mechanisms are currently quite vulnerable to exploitation, especially if communities of ill-informed readers are actively circulating content that will confirm their bias.This is how the exploitability of these systems and the openness of web and social media publishing led to a bunch of Macedonian teenagers generating clickbait fake news for Americans for cash (Silverman and Alexander).Alice Marwick and Rebecca Lewis have recently reported in a white paper on a number of tactics used for media manipulation, and some groups have been extremely deft in creating strategies to exploit vulnerabilities that exist in the algorithmic model (34)(35)(36)(37)(38)(39)(40).They document a number of network effects that allow media manipulators to channel the attention of traditional media outlets.Manipulators will create a hoax or float a conspiracy with the hopes it makes its way through the media chain, from blogs to local media outlets to, perhaps, national media (39).For the manipulators who can exploit attention in a number of ways, how the media frames the event matters less than whether it receives attention at all (39).With the incredible time and attention paid to conspiracy theories before and after the election, the fringe element of media manipulators, who likely had disproportionate influence on public debate, showed us exactly why contemporary writers need to be aware of the strengths and vulnerabilities of the tools in the current media ecology.Moreover, the emergence of media manipulation and its highly distributed nature evaded the same kind of derision in the press that the established media attracted, even if some of the platforms like Facebook were mildly criticized for their complicity.
In the next section, we look more specifically at one automated writing tactic that manipulators use to get media attention: bots.Bots are small software applications programmed to run scripts to perform automated tasks on the Internet.They represent one way to take advantage of our current shift in information-filtering practices and provide an excellent example of how technologically aware writers must be as they orient themselves to the new online writing ecosystem.

Bots in the New Ecosystem of Online Writing
Bots engage in a spectrum of behaviors online, ranging from policing Wikipedia, as mentioned above (see Krista Kennedy for a writing and rhetoric perspective on these bots), to autoresponding to

"What the fake news epidemic of the 2016 US presidential election showed was that these distributed and hybrid gatekeeping mechanisms are currently quite vulnerable to exploitation, especially if communities of ill-informed readers are actively circulating content that will confirm their bias. "
customer queries, to trading stocks, or to automating shopping for items that are in extraordinarily high demand.We hone in here on the role of bots in the spread and circulation of writing, especially on social networking sites, or SNSs.On SNSs populated by hundreds of millions of people, bots can participate in what one research group called "engineered social tampering" that serve the economic and political interests of their creators (Ferrara,Varol,Davis,Menczer,and Flammini 2).Much of this manipulation comes in the form of advertising, which serves the economic interests of bot creators, but in ways no more insidious than traditional print or video ads.As long as they comply with automation policies on SNSs (e.g., Twitter's Automation rules https://support.twitter.com/articles/76915), bots such as Patron Tequila's Bot-tender, which suggests drink recipes over Direct Message, can boost brand loyalty and engagement, especially among the tantalizing commercial demographic of Millennials (Lull).While Twitter does suspend abusive bots (e.g., those that tweet too frequently, duplicate purposes, target trending topics, or direct message without permission, according to their Automation rules), the increasingly important role of bots in marketing through Twitter and Facebook messaging means that the role of bots in SNS discourse is essentially protected by commercial interests.This openness to commercial advantage comes with vulnerabilities, as James J. Brown, Jr. has argued: once a network opens itself to the outside, it can be used-and abused-by friend and foe alike (Ethical Programs).In arenas of political discourse, the influence of bots can be subtle, and even more so because they often operate in a distributed fashion that doesn't trigger banning from the SNS.Bots have been deployed as fake populations to make certain viewpoints seem far more popular than they are, and to manipulate the emotional tenor of a discussion on social media in an effort to influence users' perception of reality (Ferrara,Varol,Davis,Menczer,and Flammini 3).Often, it is socialbots that do this kind of work, and there is a burgeoning and powerful body of research on socialbots that raise crucial questions about their role in the shaping of public discourse.
Unlike the bots that crawl the web to help search engines function or monitor Wikipedia pages, socialbots are computer programs that exist on social media networks and mimic human profiles in an attempt to convince humans the bots are just more users among many.In other words, socialbots are either designed to pass the Turing Test in social media spaces-or, alternatively, they rely on social media users to not inspect their activity closely enough to tell.After establishing a presence, socialbots can engage in a variety of scripted behaviors in an attempt to manipulate public opinion.They can follow a politician en masse to make it appear as if that politician is much more popular than s/he actually is.They can attract followers and friends, use hashtags, and re-circulate or re-tweet content such as fake news to popularize it.They can repeat information such as fake news, but they can also carry on conversations with human users by mining the Web for relevant text or cloning particular social profiles to simulate legitimacy (Ferrara, Varol, Davis, Menczer, and Flammini).These socialbots introduce more complexity to information circulation and the way we are exposed to information; they are layered on the surface of social networks and through their activity attempt to influence the algorithms that determine a user's information consumption.
Socialbots have thrived on SNSs designed in ways that accommodate bot behavior.As Douglas Guilbeault explains, the vast amount of writing and images online provide the raw material for socialbot creators to generate plausible stock social media profiles and text for socialbots.Short textual exchanges, often with novel abbreviations and typos when humans write them, tend to provide cover for socialbots' infelicities with natural language.Emoticons give them seemingly complex speech with wide latitude for sensemaking.And the removal of humans from much of the editorial process means that socialbots often need to pass muster only with human-decision-simulating algorithms, such as they are themselves.Networks of socialbots following each other in social media platforms can effectively bolster their social media currency and help them pass through both human and algorithmic filters.Because of the ways they can effectively draw on resources and operate in constrained systems, these bots can be difficult to detect.But even if human or nonhuman users do detect them, the design of social networks and their emphasis on popularity incentivizes users to friend and follow bots (Guilbeault).They operate within social network systems and are well-tailored to do just that.Because socialbots thrive under these particular conditions, Guilbeault argues that we must understand them ecologically.He introduces the idea of "platform persuasion, " that is, the ability of socialbots to use the automated conversation tools SNSs provide-such as sharing messages verbatim and emoticons-in order to convince users to follow them or read their messages.Their success in these systems means that it's irrelevant whether they might succeed in passing a Turing Test elsewhere.Their ability to persuade is fully enmeshed in the platform in which they operate.
We can understand Guilbeault's model in a longer tradition of rhetorical ecologies.Over the past few decades, rhetorical theory has moved away from models of persuasion that consider how a rhetor with a single text might persuade an audience and toward models of rhetorical ecologies that seek to understand how networked people experience multiple encounters with a variety of other people, texts, and objects over time (e.g., Cooper; Edbauer).And whether we conceptualize literacy practices as happening in rhetorical ecologies, communities of practice, or activity systemsto name a few influential models in literacy studies of past decades-thinking about texts as plural, interconnected, and embedded in complex systems has been a dominant trend.Rhetoric as a field has also considered how the delivery and circulation patterns of online writing shape the production practices of writers as they anticipate reaching their audiences (Ridolfo and DeVoss; Brooke; Eyman).If we think about web content through these kinds of ecological frameworks that have been part of rhetorical thinking in composition, we see how the presence of bots designed to operate in SNSs shifts the situation not only for those who wish to deploy them but also for human writers who want their words to contend for attention in this space.

How Bots Interfere with Human Political Discourse
At least since Claude Shannon's work in communication and noise, we have known that message delivery can be plagued by transmission errors and false or irrelevant information.James Gleick discusses the perpetual challenges of filtering good information from mixed channels of good, bad and irrelevant information, a process that changes but never seems to be obviated by new technologies of communication.In the calculated online writing ecology that we focus on here, we see new kinds of interference that degrades information quality-automated amplification, spoofing followers of accounts, and the burying of genuine human political discourse.And when we consider the influence of bots, we need to consider how they both amplify certain kinds of information and, at times, actively work to suppress other kinds.To echo our opening to this article, human political discourse has always been shaped by material and nonhuman conditions of writing, including technologies that amplify or suppress.We can now consider how automated amplification and suppression practices work to shape online discourse, particularly in the case of bots.
Scholarly studies of Twitter after the US midterm elections of 2010 mark some of the first efforts to measure bot participation in the spread of (mis)information.Such studies were largely produced by specialists in computer science and networked systems research with the aim of identifying characteristics of bot "abuse" in social media for the purpose of developing digital tools that would help users identify bots (http://botornot.co/provides an example of one tool that has been since published).Bot identification can be tricky for a number of reasons, and as Guilbeault has argued, the design of social media platforms encourage kinds of discourse to which bots are well fit.The uniform templates of SNSs encourage a uniformity of composition, which means bots do not have the challenge of "creating novel forms of self-expression" (5009).In the SNS ecosystem, bots are excellent conduits for spreading (mis)information.
Indeed, some studies of the 2010 midterm elections voiced strong concern that bots were being used to spread fake news and misinformation.P. Takis Metaxas and Eni Mustafaraj tracked a 2010 instance where nine twitter accounts began messaging other users with a URL to a website that smeared one of the political candidates with fake news in a Massachusetts election (some time later the Republican group of "Swift Boat" fame outed themselves as the creators of the site).Metaxas and Mustafaraj found the bots pushing the URL were retweeted enough by humans that it reached more than 60,000 users while rising to the top of Google results in searches for the candidate's name.This prompted the authors to worry about the future exploitation of automated writing tools if actors could reach tens of thousands of users with "a few minutes worth of work, using automated scripts, and exploiting the open architecture" of sites like Twitter (6).Since 2010, bots and their deployment have grown more sophisticated and arguably more influential in political discourse as candidates increasingly rely on Facebook and Twitter to communicate to their constituents.Yet, as Douglas Guilbeault and Samuel Woolley warned on the eve of the 2016 US presidential election, the outsized influence bots could have on political outcomes has gone completely unremarked on by the US Federal Election Commission, whose job it is to oversee financing of elections.
Alessandro Bessi and Emilio Ferrara have been involved in developing bot detection software and have published the most extensive account of how bots intervened in the political discourse of the 2016 election cycle in the US.Published in the internet studies journal First Monday, their findings are worth recounting here because they offer literacy studies a detailed glimpse of the extent to which online writers were writing with and among a very active social bot population during the election.Some of the important findings relevant to our discussion: 1) Roughly 15% percent of the Twitter population they sampled consisted of bots, and those bots sent roughly 20% of the total tweets related to the election.2) The social bots had a number of different functionalities.The authors list at least eight, including searching Twitter and retweeting relevant content, adding users tweeting about a topic to public lists, and automatically following relevant users.3) Humans did retweet bots in a substantial number of instances, which suggested to the authors that "bots are being very effective at spreading information in the human population" (n.pag.).After an account of how bots became active participants in online election discourse, Bessi and Ferrara conclude that bots create three tangible issues for political speech: a skewed distribution of influence, further polarization of political speech, and the continued spread of misinformation.Presumably bots could also be leveraged to spread factual information too, but as we've seen of late, a hyper-polarized political climate makes it difficult to agree on what constitutes factual information.
Another concern about credibility that bots introduce to our current political climate is related to human users' followers, which are often used as a shorthand to determine a person's popularity with the public.The most high profile example is that of Donald Trump, who has cited his large Twitter following as an indicator of his popularity.Speaking at a tech conference in May 2017, Hillary Clinton charged that his follower numbers were being goosed up by fake accounts, although the claim that he had a recent surge in followers was debunked by Snopes, a prominent fact-checking website ("Did President Trump's Twitter").Still, some web tools routinely estimate that roughly half of Trump's thirty million followers appeared to be fake, or bots-something that has been widely reported in the press (Bort).In a recent check conducted by the authors, the website Twitter Audit, which analyzes a user's followers to determine which are fake or real, indicated that 45% of Trump's are bots, 10% of Barack Obama's followers are bots, and 37% of Hillary Clinton's are bots (as of July 8, 2017).But these numbers may not be accurate: a Twitter spokesman said that the widely-cited Twitter Audit website's methods for determining bots were flawed (Johnson and Gordon), and, as discussed above, it's hard to determine just how many of the millions of followers of these politicians are fake.
Regardless of the percentage, bot followers now seem to be part of the structure of high profile accounts.It is usually unclear whether users themselves or their representatives are organizing or buying these bot followers to boost apparent popularity, or whether they are coordinated independently.There are many services that sell bot audiences to boost a user's status on Twitter or other SNSs (e.g., MonsterSocial), and since bots themselves aren't against the terms of service of SNSs such as Twitter, these services are relatively affordable and above board.The business of bots is not well-known by the public though, and so their role in boosting follower numbers can be misleading.For instance, Donald Trump has bragged about his millions of followers without noting that so many of them are suspected bots, perpetuating the impression that he has more human followers than he does (Bilton).
A study by Jacob Ratkiewicz et al. warned of the use of bots for "astroturfing, " a term used to indicate a highly coordinated and well-funded campaign designed to look as if it had instead emerged from grassroots efforts.The authors argue that social networks can be exploited so that an attacker "can easily orchestrate a distributed effort" to mimic the organic spread of information through social networks (297).As an example, work by the Digital Forensic Research Lab examined the followers of fringe French-language, Russian-allied political accounts on Twitter and discovered that their followers appear to be highly active retweeters and responders, in comparison to mainstream French media accounts.These fringe followers tended to align with Russian interests or French nationalism and opposed the centrist candidate Emmanuel Macron in the 2017 French presidential election.They also appear to have been highly automated accounts (Nimmo and Czuperski).
A slightly different astroturfing example can be found preceding the US election.While he was a presidential candidate, Trump's tweets were highly visible, and an algorithm in Twitter's mobile app that prioritized the display of replies based on rapidity of response rather than popularity incentivized political actors to compose scripts that would autorespond to Trump's tweets.When the algorithm changed to (apparently) consider aspects of the accounts responding and popularity of the response, these bots disappeared or became irrelevant and thus invisible in Twitter replies (Matsakis).Guilbeault and Woolley claim that one-third of pro-Trump tweets and one-fifth of pro-Clinton tweets between the first two presidential debates originated from automated accounts, enough for either candidate's online perception to be influenced by bots.In the ways that bots interact with media outlet and human users, they can affect the tenor of political discourse through amplifying or lending credibility to fringe politics and high profile accounts.These effects shift rapidly with political events and changes in the implementations of the algorithms that bots attempt to game.A high-profile example illustrates how bots can demonstrably influence politics.While serving time for his crimes of digital espionage, Andrés Sepúlveda spoke to Bloomberg Businessweek about how he used digital means to hack elections in Latin America for over a decade.In addition to hacking into candidates' communication channels, he took advantage of the fact that expressions of candidate support on social media were often read as more genuine than advertisements or overt propaganda.He wielded both high-end fake Twitter profiles, which had been maintained for over a year and seemed more believable, and sheer quantities of lower-end profiles-30,000 that he could deploy to shape conversations, for example, about the 2012 Mexican presidential candidate (and winner of the election) Enrique Peña Nieto's plan to end drug violence.This dual strategy of high-and low-quality bots is illustrative of how bots can be coordinated to spread news quickly and believably.The reporters write, [Sepúlveda] knew that accounts could be faked and social media trends fabricated, all relatively cheaply.He wrote a software program, now called Social Media Predator, to manage and direct a virtual army of fake Twitter accounts.The software let him quickly change names, profile pictures, and biographies to fit any need.Eventually, he discovered, he could manipulate the public debate as easily as moving pieces on a chessboard-or, as he puts it, "When I realized that people believe what the Internet says more than reality, I discovered that I had the power to make people believe almost anything." (Robertson, Wiley, and Willis) The fact that many people perceive online discourse as more immediate and genuine than media reporting opens the door for political interventions such as Sepúlveda's, suggesting that the perception of online discourse is not fully accounting for the ability of bots to suppress, amplify and distort information.
As bots compose, tweet, and retweet according to whatever script they've been given, they influence the tenor of the discussion and the kind of information that reaches people not simply by motivating information and encouraging it to go viral, but also by suppressing other kinds of information.If we understand literate people as moving in and across environments saturated by the texts and objects they use to create meaning, then information suppression matters as much as degradation: the texts that don't appear in an environment can be as notable for their absence as the texts that do appear.In the liberal-democratic model of public print discourse, historical examples of textual suppression are essentially censorship and are framed in largely negative terms.Text suppression is not new, and it certainly happens on more quotidian scales as well: collections decisions for libraries, prominent displays of books on endcaps, editorials in newspapers that may not reveal all of the writer's connections, and so on.Woolley has profiled the way bots have interfered in global politics, concluding that bots have been able to "suppress free expression, " suffocate democratic speech, and demobilize activist groups.
A study on the Syrian civil war shows additional ways in which this suppression happens: for example, misdirecting attention to trending world events by flooding a hashtag with irrelevant or inaccurate information.During a six-month period in April 2012, when the Syrian civil war was a globally discussed event, Norah Abokhodair, Daisy Yoo, and David W. McDonald tracked a networked group of 130 social bots that were tweeting and re-tweeting about the war.Active for thirty-five weeks, this botnet produced thousands of tweets per week the researchers collected and analyzed for content and purpose, which helped them classify bot behavior and the attempts bots make to intervene in public discourse.The botnet had "generator bots, " or core accounts that tweeted thousands of times per week, tweets that were then retweeted by less active bots in the network.The tweets largely consisted of news, opinions, and spam.One particular behavior of these bots indicates how botnets can misdirect.In a number of instances, one of the bots would tweet photos of Hurricane Sandy and its distance from New York, but the bot would include the hashtag Syria, thus anyone searching the hashtag in search of tweets about the Syrian war would have to wade through the irrelevant content.The news of Hurricane Sandy propagated by the bot works to divert the audience's attention away from tweets that criticized the Syrian regime.Abokhodair, Yoo, and McDonald refer to this as "calculated misdirection, " borrowing a term from magicians who attempt to divert the audience's attention while orchestrating a trick.This kind of "misdirection" has long been a part of the political propaganda game-for example, in the Chinese government's online strategy, although in that case, the propaganda issues from people.The popularly called "50 cent party, " or wumao, post coordinated, pro-government statements on Chinese social media accounts, effectively censoring statements critical of the government by providing conflicting accounts.Gary King, Jennifer Pan, and Margaret Roberts estimate that these posters account for about 448 million comments a year on social media, or about 1 post per 178 on commercial Chinese social media sites like Sina Wibo, although the number spikes during sensitive time periods.The study reports that these individuals, mostly government bureaucrats, don't engage with critical statements so much as they attempt to flood networks with pro-government sentiments.In 2016, the Correct the Record PAC launched an online campaign to counter anti-Hillary Clinton posters and promote issues most favorable to her candidacy-ostensibly to "correct" the high volume of negative rhetoric directed at her, exemplified by hashtags such as #crookedhillary.Correct the Record posters were, it appears, human, and mostly responded with positive statements about Hillary, but their open presence as paid propaganda may have undermined their efforts, especially since they were able to coordinate directly with the Clinton campaign due to a loophole in campaign finance law pertaining to online activity (Foran).Pro-government human labor is easily obtained in China and payment for pro-Hillary posters may have been a small expenditure for her well-financed campaign.But on a platform where numbers matter, bots can scale up propaganda and obfuscate political viewpoints in public discourse much more cheaply.Again, given these examples of human intervention, the difference that bots make in political discourse appears to be in terms of scale rather than strategy.

What Bots Mean for Literacy Studies
While work in digital rhetoric has long noted that writers compose with and among software programs and code that act on their writing in subtle and nuanced ways (e.g., Rieder; Leblanc), we must ask whether current automated writing systems such as bots are fundamentally different from other code-based influences.Code can define a template for a social media site that will circumscribe possible kinds of writing; scan for copyright violations in a text and automatically take it offline; or filter and flag texts based on keywords associated with, say, commercial entities or terrorism.Although we can think of the bot intervention recounted above as related to these other code-based influences on online writing, we argue that automated writing systems such as bots mean that code is making its way into the composing lives of online writers in a new way-compelling them to write alongside and contend with scripts that are engineered to attract or distract both human and algorithmic readers.Human writers must interact with a legion of programmed writers, sometimes controlled by shadowy actors manipulating the circulation patterns of text in the online writing ecology.
One of the foremost voices of sociological studies of algorithms, Gillespie is quite clear on the stakes of the dynamics profiled above: "If our participation in public life [...] is being determined, or at the very least adjudicated, by algorithmic systems, then we must know more about the assumptions upon which they are based, the information about us upon which they act, the priorities they serve, and the ways in which they shape, distort, or tip the process" ("Algorithmically Recognizable" 64).As Andrés Sepúlveda noted, the general public may perceive online discourse as more "genuine" than mass media while being largely unaware of the influences of automated writing systems in mimicking "genuine" human discourse.Mass media seems to reinforce this assumption when individual Twitter accounts are quoted in news stories, stories whose titles often begin with "The Internet is Reacting to…" or "The Internet Says…." These articles are a modern day emulation of "man-on-the-street" interviews-although it is much more difficult to verify whether the quoted writing was produced by humans.In part, we've written this article to help literacy and composition studies take a small step forward as it considers the question of what it means when the "we" of "we must know more" about algorithms represents ordinary writers and readers.Given these misperceptions, literacy and rhetoric educators may bear some responsibility to point to the algorithms driving the complicated automated systems that mimic and shape writing online.
But the pressing question that will need to be answered is how people can engage in responsible discourse in the face of rapidly evolving technologies that can be exploited and can also offer a bullhorn to the most detestable of political positions.And the answers can be in the mundane ways we approach technology in the classroom.Teachers of digital writing and rhetoric have begun to wrestle with this question in different forms, as digital tools and the ecosystems in which they function have cracked open, and made accessible, a slew of affordances that were formerly blackboxed for writers-affordances writers might have had little call to consider in the age of print.Even a simple blog provides opportunities for writers to make decisions about design (typography, color combinations, headings, structure, and so on), accessibility (metadata and captioning), and audience experience.Although digital tools provide these options, it is also possible to ignore most of the issues, compose in a WYSIWYG interface, and click to publish.Given the difficulties of writers in learning to control their syntax, thought, and evidence well enough to express themselves, bedrock activities of composition's pedagogical project, how much of the complexity of online writing should teachers be expected to know and to impart?To what extent do we rely on experts to close the exploitable gaps in the contemporary media ecosystem?What do writers need to know, and what competencies do they need to have to maintain a critical disposition to their own work?What can they give themselves permission to not know?What processes can they blackbox or use WYSIWYG interfaces to handle?Our discussion here about automated writing systems reveals that the scene is even more complicated than previously thought.Compounding the longstanding questions of digital pedagogy are now the challenges of optimizing one's writing and publishing practices to reach intended audiences and dealing with the crushingly complicated social dynamics that are part of online writing ecosystems.To what degree should writers accommodate algorithmic filters when they plan for the future circulation of their texts?How should they account for the copresence of bots in their online writing ecosystems?
These may be largely contextual questions that writers can address as they happen, and some of them writers will learn through experience and intuition.But as teachers of writing, we must consider to what degree should we be embedding questions of automated writing systems in our pedagogy.As researchers of writing, how should we account for the bots and algorithms in our online environments?We do our best to equip students with competencies they can deploy, grow, and adapt with as much flexibility as possible, but with finite time and resources, the question of what dynamics students should prepare for in digital composing is complex.And research in writing seems to add complexities to our understanding of the process of written communication with each new study published.Many of the studies we cite above suggest that automated writing systems such as bots and algorithmic filters will play an increasingly important role in online writing.Bots and their relationship to fake news have been a convenient manifestation for our larger argument here: that questions related to algorithms and automation are already a core part of literacy and play a profoundly important role in how we encounter texts as readers and writers.Our approaches to this fact in our teaching and research may take different forms in writing studies, but regardless of approach, we will need to acknowledge the robots among us.