Electronic Errata: Digital Publishing, Open Review, and the Futures of Correction
PAUL FYFE
It is hardly possible to write a history of information separately from a history of the corruption of the press.
—Walter Benjamin, Charles Baudelaire: A Lyric Poet in the Era of High Capitalism
In writing about mid-nineteenth century newspapers, Walter Benjamin notes the prevalence of the réclame, a paid publisher’s advertisement printed instead as an editorial notice and hidden within the miscellany of the page. For Benjamin, this “corruption of the press” was so widespread as to necessarily inform any “history of information” (28). But Benjamin’s insight can also apply by corrupting the very word “corrupt” to mean something like “error.” As bibliographers and textual critics well know, it is hardly possible to write the history of information without attention to errors, accidents, variants, and changes—the dynamics of corruption and correction that pattern the history of published print. As this essay will argue, it is also hardly possible to write the future history of information—an enterprise in which the essays in this volume are very much involved—separately from those same dynamics as they manifest, in new ways as well as old, in online and digital publishing. But we have yet to do so.
As Steven Berlin Johnson argues, “[t]he history of knowledge conventionally focuses on breakthrough ideas and conceptual leaps. But the blind spots on the map, the dark continents of error and prejudice, carry their own mystery as well…. These questions, too, deserve their own discipline—the sociology of error” (Ghost Map, 15). This essay undertakes such questions for the current moment in electronic publishing by looking to the history of printing errors and the labor of correction. That history offers valuable perspectives on the proliferating effects of “accidentals” and error within the automation, syndication, and algorithmic relations of the web. It further informs how scholarly publishing’s digital futures might (or might not) deal with the issue, including open review, crowdsourced or distributed corrections, automated redundancy systems in libraries, and intelligent computing agents. The history of error also opens some theoretical perspectives (sometimes noted as lacking in digital humanities discussions), suggesting that, even if we do not dedicate technology or labor to these issues, the digital humanities needs to reckon conceptually with what, in a different context, John Unsworth called “the importance of failure.” Scholarly publishing will inevitably change, but before its print-based model totters into the grave we need clearer commitments about the error proofing it has traditionally undertaken as well as the consequences of reinventing or abandoning such functions altogether as we move to new formats.1
Certainly, the discussions about the digital transformation of publishing, especially (though not exclusively) the publishing of critical work and scholarly resources, have mapped an array of important issues whose problems and opportunities need to be resolved, including peer review, credentialing, collaboration, intellectual property, multimodal textuality, encoding standards, access, and sustainability.2 But largely absent from these discussions, or playing only a minor role, is the fate or future of copyediting, fact checking, the often thankless tasks of verification and correction that usually happen behind the publishing scenes. This omission is at once entirely consistent with the historical instability of copyediting (as I will suggest) and uniquely problematic from a contemporary perspective. While “Jhon Milton” is an easy enough error to catch (nonetheless his e-book Areopagitica is currently for sale on Amazon [Trettien]), the typographical, factual, and citation errors that characterize any publishing endeavor, online or in print, will not be resolved by casual or professional use in the ways commentators have envisioned.3 Electronic errata have potentially cascading effects that we would do well to consider in imagining futures for digital publishing and networked scholarly communication.4 So, also, do new regimes of correction have limitations and theoretical compromises that we should not ignore.
Before situating this argument historically, I want to make two major caveats about the present moment. Discussions of online publishing and digital projects have in fact deeply considered the problems of “error” in terms of preservation and their own technical obsolescence, including everything from hardware to browser interoperability to the degradation of links or “link rot.” Second, there has been abundant attention to the errors of digitization, OCR, and metadata cataloguing, exemplified in Geoffrey Nunberg’s ongoing critique of the Google Books initiative and projects like MIT’s “culturomics” initiative that depend on the integrity of Google’s data. Furthermore, any programmer worth her silicon knows the simple equation of “garbage in, garbage out” (GIGO), which underwrites the integrity of data and code. My argument is not that we are ignoring error but rather that we have not sufficiently considered error correction as a structural feature and theoretical premise within the transition to digital publishing.
This is not a conservative argument seeking a return to the good old ways. In fact, ignoring the particulars or correction ironically threatens to replicate a conservative notion of textuality that digital publishing was supposed to displace. Nor is this just scholarly nitpicking. If publishing errors seem mundane or even trivial compared to the greater transformations in digital scholarship, they have the potential for aggregate effects on the network and discourse we are producing. It may be that we decide that such errata do not warrant much technical concern or that they should fall down the scale of scholarly values as recalibrated by digital or open publishing. But these positions need to be articulated; we need clearer plans for the obsolescence of academic correction. Even if this essay floats a balloon that is quickly shot down, it will have drawn the necessary fire to a target that has hitherto flown beneath the radar. We begin with a trip backward, a historical detour that is also a shortcut to the present.
The Disappearance of the Reading Boy
I will hazard a generalization that getting copyedited is a consistently unsettling experience in academic life—at least in a professional life which has been governed by a regime of writing and peer review. On the otherwise mostly pleasant trajectory of, say, submitting a finished article, having it acknowledged or accepted, making further substantive revisions, and then sending it off for publication, the copyedited proofs one receives can come as a bit of a shock. Such was my reaction on receiving copyedits on an accepted article: I thought I was done, having scrupulously checked the text each time before submission and having survived peer review and its recommendations. But starkly indicated on the returned manuscript pages were anonymous testimonies to surprising grammatical inconsistencies, the occasional sloppiness of citation, lurking typos, and the flat-out misquoting of sources, several of them arcane. Why hadn’t I seen them? Why had no one pointed them out before? And who was behind all this, doing the work on the page and in the library to check all this text and references?
There is no copyeditor listed on the masthead of the journal issue in which that article was published.5 Perhaps it was the editor or assistant editor. More likely it was an (unlisted and unsalaried) editorial assistant—and probably a graduate student employed by the publisher for preprint quality control that is distinctly separate from the editorial functions of peer review. During my time as a graduate student, I actually had such a job, working to check facts, confirm citations, and copyedit quoted materials for a peer-reviewed journal—specifically, for Postmodern Culture (PMC), the first peer-reviewed academic journal to publish entirely online and for free. As forward thinking as PMC was in relation to emergent trends in digital publishing and open access, it preserved the very traditional protocols of peer review and correction as well as the distinction between them. Indeed, they had to be kept intact for PMC to leverage its more revolutionary proposal of open digital publishing.
With all due respect, this was a boring and tedious job. The workflow requires a different kind of attention than reading the articles or reviewing their arguments. Instead, the assistant confirms the page number, bibliographical citation, and textual accuracy of every piece of quoted, referenced, or footnoted material in a given manuscript. The assistant scurries all over the stacks in the library, hoarding armloads of books to check a mere fragment in any one of them. The assistant becomes the scourge of the interlibrary loan department, winging out endless requests for copies of materials the library does not have. From today’s vantage point, the assistant seems like the motor arm in a Rube Goldberg reinvention of a database, scrambling the physical world for the items that a digitized network would so easily trace. Databases and electronic versions of sources might resolve some of these logistical difficulties, but they invite other obstacles, including access policies, the integrity of digitized materials, and coverage. Then and now, the editorial assistant has to undertake a serious amount of mundane labor. I am very grateful someone did that for me, whoever you are.
Such an assistant or fact-checking process is not necessarily a part of every scholarly publishing venture. But the assistant embodies a particular attitude toward correction whose institutional evolution is worth considering. Though the messy details of book history may seem remote from the digital transformation of publishing, they set the conceptual terms and labor conditions that have reached a crucial moment of redefinition. The anonymity and transient labor of the journal fact-checker or copyeditor is itself a historical legacy of the publishing business at large. Proofreading has a history as long as writing itself, but, in print publishing, the figure of a specialized in-house proofreader only emerges in the seventeenth century in continental Europe, subsequently adopted in the mid-eighteenth century by English printers. This was the “corrector”: typically only employed by larger publishing houses, correctors were “not as a rule trained printers, but were men of education specially employed, sometimes on a part-time basis” (Gaskell, 172). Then, as now, the corrector occupied an ambiguous middle ground between editorial and production, between the managerial and technical realms. In Philip Gaskell’s hierarchy of labor of the English book trade to 1800, the corrector falls right on the threshold between business owners, printing masters, and overseers (all with fixed wages) and the journeymen, apprentices, and juvenile runners who worked for them (all paid piecemeal) (171–74). Though expected to be well educated and to exert editorial influence on the copy, they were not in the salaried echelon of a house’s editorial or commercial management. Though paid piecemeal, working with in-process sheets and collaborating with the compositors and printers assembling them, they were not allied to the working press. The corrector overlapped with each domain and properly belonged to neither.
Even when a press did not employ correctors, such as in smaller houses, their very absence suggests the professional ambiguities and shifting responsibilities of the corrector’s role. In these cases, “correction was normally carried out by the master or a senior journeyman; less commonly the corrector was the author or the author’s representative” (Gaskell, 111). In other words, correction could happen on either side of the masters/men or publisher/author divides. Sometimes, publishing firm partners and senior editors got involved in the process, as when houses undertook significantly complicated or expensive work (Gaskell, 113). On the other end of the spectrum, correction was already taking place during the very setting of type: “it had been the compositor’s duty to correct or normalize the spelling, punctuation, and capitalization (known nowadays as the ‘accidentals’) of the manuscript” and ensure the integrity of the “substantives” (Gaskell, 110, 111). As the compositor slotted lead type into his stick, he was silently doing the typographical and editorial work that the corrector would come to check. A corrector was required because the compositor had incentives to overlook his own mistakes, since doing so would be charged to his time.
In underwriting the added value of error-free print, the corrector’s allegiance was to the management, but his typical work partner hailed from the opposite side of the labor hierarchy. This was the “reading boy” or “copyholder” who would read aloud from the manuscript copy while the corrector checked the printed proofs. He would then return the proofs (in various states: the all-important first proof, revise proofs, press proofs) to the compositor, marked up in the standardized graphical codes of copyediting which “remain essentially unchanged today” (Gaskell, 113). What has definitively changed is the corrector’s workflow: checking a printed proof while listening to the reading boy, then switching roles as physical fatigue warranted. The reading was a professional performance with its own auditory conventions; a reader pronounced grammatical marks and used sing-song inflections for variations in type: capitalization, emphasis, conjunctive and possessive apostrophes, and so on.6 It also announced a publisher’s relation to source text that would profoundly change by 1900, as manuscripts increasingly came into the house having already been edited for errors, or otherwise declaring themselves as proofed copytext to begin with—with no small thanks to the increasing popularity of typescript. The jobs of compositors and correctors changed: they were no longer silently normalizing the “accidental” errors of the text but instead obliged to preserve accidentals and substantive errors in the proofs just as they appeared in manuscript. Attending that change was a shift from aural to ocular correction, or the visual collation of a work in different stages of production. As a consequence, “the reading-boy disappeared in English printing houses around 1900” (Gaskell, 353).
A century later, his partner the corrector has not fared much better, particularly since the advent of new media, which, as I will argue later, shifts correction even further from aural to ocular into synthetic or automated realms. But correctors, reading boys, editorial assistants, and copyeditors are losing their places for more conspicuous reasons. Though this is not exclusively an economic issue, the financial impact of new media upon old media, as well as coinciding economic downturns, have made the considerable costs of correction seem like an unnecessary burden. Having always occupied a tenuous middle ground between management and production, editorial positions of various kinds have been subject to significant cuts.7 From a different perspective, emerging enthusiasm about unfiltered publishing online has privileged speed, accessibility, and distribution over the comparatively tedious labor of editorial nitpicking. In effect, the corrector’s function has been dismissed or elsewhere absorbed by publishers and independent content producers across the continuum of scholarly publishing. But the consequences for its seeming disappearance or reinvention are significant.
Revaluing Review and Correction
As with the death of the author, the disappearance of the corrector raises the specter of what might be called, after Foucault, the corrector function: a relation of value between a printed text and its typographical, factual, and stylistic integrity. The historical quirks and ambiguous status of the corrector underscore how such a function is constantly being negotiated through the economic, social, and technological conditions of the present, or within the discourse of publishing. According to a variety of observers, that discourse is experiencing a paradigmatic shift with the advent of digital publishing in various commercial, civilian, and scholarly forms. So, too, are its structuring relations and values in flux—very much including correction and the notion of a published or finished text. Looking at the genealogy of the corrector, we can still find its functions in the present (such as with copyeditors, graduate editorial assistants, etc.), but what are its futures? Will the prepublication function of correction continue to make sense? Or how will it evolve?
For scholarly publishers and university presses, digital publishing has heralded a major transformation in their businesses. But attention to correction, in either its traditional or its future incarnations, seems notably absent from some of the more conspicuous forums in which university presses have worried or speculated about their futures.8 While the industry is quick to celebrate its role in “improving work through peer review/editing,” the particular values or necessities of correction go relatively unmentioned (Brown et al., 36). Peter Dougherty suggests in his “Checklist for Scholarly Publishers” in transitioning to the digital future that “[e]ditorially, our work is cut out for us”; however, attention to the editorial integrity of copy remains implicit at best. A recent Association of American University Presses report on Sustaining Scholarly Publishing suggests several new aspects of correction including “[q]uality assurance and filtration, presentational enhancement, metadata crafting and maintenance, …[and] establishing authoritative versions,” but they remain abstractions in the background of the report’s abiding concern for business models (4). Taken together, these forecasts skew toward the managerial and technical domains, toward modified business models and digital production strategies. Obviously, scholarly publishers have some serious and pressing concerns, but the declining attention to correction starts with publishers themselves.
From a different perspective, the inattention to correction seems symptomatic of changing attitudes about its necessity. In an important blog post titled “The Social Contract of Scholarly Publishing,” Dan Cohen recounts a conversation with his mentor and collaborator Roy Rosenzweig about a book manuscript they had recently finished. In an unguarded moment, Cohen asks, “So, what now?” Isn’t the book for all intents and purposes finished? Couldn’t it be published online immediately? “Isn’t it 95% done? What’s the last five percent for?” Rosenzweig’s response sketches out what Cohen calls “the magic of the last stage of scholarly production between the final draft and the published book.” It amounts to the mystification of the scholarly fetish. As Rosenzweig replies, “What happens now is the creation of the social contract between the authors and the readers. We agree to spend considerable time ridding the manuscript of minor errors, and the press spends additional time on other corrections and layout, and readers respond to these signals—a lack of typos, nicely formatted footnotes, a bibliography, specialized fonts, and a high-quality physical presentation—by agreeing to give the book a serious read.” With a quick jerk, Rosenzweig pulls the curtain of this social contract, exposing the hoary, wizard-like scholar as a fabrication in smoke. The “magic” of scholarly prestige is produced by the business of publishing. By contrast, click an “Update” button in WordPress and poof! It’s gone—and with it, all the social, disciplinary, or institutional credibility of our faith in the 5 percent. Cohen wants to change this equation, as do a host of scholars imagining (and actively realizing) the possibilities of near-instantaneous, inexpensive, networked scholarly communication on the web. In a follow-up post titled “Open Access Publishing and Scholarly Values,” Cohen completes the arc of the argument: the social contracts of peer-reviewed print scholarship are rapidly expiring, and digitally open-access scholarship is driving the reformulation of scholarly values at large. Forget the 5 percent. The “added value” of the traditional scholarly publishing model no longer adds up.
In these posts, Cohen does not address what futures might be afforded to the corrector function. Perhaps there are none. It is certainly possible to think about copyediting and fact checking as tactical elements in the cost-recovery model of closed or for-profit publishing. In other words, such correctness is not scholarly integrity ipso facto but as much a by-product of the historical development of publishing as the material features of the book, notions of authorship, and peer review. Corrected and copyedited texts are saleable and signify in a reputational economy because they have been processed in ways that users are unlikely to replicate or undertake themselves. Correction is the “added value” of publication, part of an increasingly outmoded business plan or a contractual relic of scholarly values that ought to be renegotiated. In an e-mail, Cohen suggests that “we could regain the lost opportunity cost of other scholarly endeavors” by simply forgiving minor errors, as is commonly practiced on the open web, since the “true quality” of such work will remain intact.
Cohen and Tom Scheinfeldt have experimented with dispensing the 5 percent altogether—and then some. With Hacking the Academy, they crowdsourced a book in one week, inviting contributions from anyone in a range of forms and media. Cohen and Scheinfeldt collected many of them for an open-access volume published by University of Michigan’s digitalculturebooks series. Scheinfeldt reports that he and Cohen are responsible for all the copyediting, which is still laborious but out of necessity less than is usually undertaken for academic volumes. As Scheinfeldt speculates, this situation will likely become the norm: in addition to more speed, “less nitpicking overall, more work thrown back on the editor/author, and more responsibility on the reader to read texts critically for factual and other errors.” If, by 1900, texts declared themselves correct on arrival at a publisher, we may soon see correction become a postpublication process, if it happens at all. The first shift eliminated the reading boy and changed the dominant mode of correction from aural to ocular. The current shift may eliminate the editorial nitpicker entirely, displacing correction onto the reader or to autocorrecting functions of networks.
Scheinfeldt is sanguine about the economics of this situation, including its uncompensated outsourcing of work, but he is also optimistic about its potentials. Why belabor the 5 percent when a new model of publishing repays it twice over with intellectual and social dividends? Shana Kimball—editor of the Journal of Electronic Publishing as well as head of publishing services, outreach and strategic development at the University of Michigan’s Scholarly Publishing Office—thoughtfully asks, “[m]ight scholars decide that speed, agility, and network effects of open, web-based forms of scholarly communication trump perfection and error-correction in some cases? Perhaps the quality, say, of self-published scholarly blogs would be good enough for certain purposes, when it’s not worth the time and expense (and these functions are quite expensive) of careful editorial nit-picking.” In this scenario, other functions of correction or “value” might emerge: not governed by commercial publishers or closed peer-review practices but opened to the collective input and dynamic adjustments of the web.
Kathleen Fitzpatrick’s Planned Obsolescence: Publishing, Technology, and the Future of the Academy is perhaps the most comprehensive and eloquent assessment of this situation—as well as an interesting performance of its own argument online. Fitzpatrick’s target is peer review: the conventional system of prepublication peer evaluation that functions variously—according to whom one asks in what discipline—to sort through the mass of works in progress, to improve or add intellectual value to scholarship, to control quality and prevent errors, and to serve as a gatekeeper and credentialing mechanism for academia. In tangling with these aspects of digital publication, Planned Obsolescence does not explicitly engage the functions of correction that, as we have seen, are often kept distinct from editorial direction and the peer-review process. Like scholarly publishers themselves, Fitzpatrick’s attention is drawn elsewhere—again, justifiably so but in ways that are also suggestive of how debates about digital publishing and peer review are largely taking place on different terrain.
These debates have instead concentrated on new forms of review including in-process open review, peer-to-peer review, postpublication review, no review at all, or automated/synthetic review as an effect of usage and search engine ranking algorithms. Planned Obsolescence was itself subject to both traditional peer review through New York University (NYU) Press and open review on the web, as the draft was published online with CommentPress, allowing for open user comments by page and paragraph. To Fitzpatrick’s great credit, she posted both of her official reader’s reports (by Lisa Spiro, with permission) along with exposing her prepublication manuscript to public scrutiny and commentary. Examining these documents and their assembled commentary reveals a genuine intellectual generosity in each mode, and Fitzpatrick acknowledges the contributions of each in improving the manuscript. So, too, does each mode offer the occasional correction or copyeditorial note on typos and style, but only sporadically. Fitzpatrick reports that the open review did offer a fair bit of such suggestions but that commenters primarily engaged with the argument—which is exactly what she wanted (“Re: open review”). The revised manuscript was itself sent out for professional copyediting by NYU Press.
This method of open review is seeing more trials elsewhere. The journal Shakespeare Quarterly (SQ) has undertaken several rounds of preprint open review on the web. The comments during its initial experiment, like those for Planned Obsolescence, favor an engagement with the argument rather than pointing out errors. The “FAQ” for the second trial advises potential reviewers that “the most useful feedback is that which an author can use to strengthen an argument,” though short comments are fine. According to associate editor Sarah Werner, SQ is neither expecting nor relying on open review to provide stylistic, citational, and factual correction. It happens in places but is still conducted after the review trial is over: “each article is fact-checked by either the Managing Editor or the Production Editor, and the article is copy-edited by the Managing Editor and the Editor.” Werner suggests that “both of those are things that even the best, most careful writers have a hard time doing for their own work.” Furthermore, “fact-checking provides reassurance that the argument the author is making is reliable and gives the necessary tools for other scholars to do their own assessment with the sources.” Just as PMC kept its familiar editorial foundations upon which to innovate, SQ remains a fairly traditional academic journal, committed to the added intellectual values of editorial work. Werner worries that “the biggest danger of scholarship moving toward online and open forms is that editing might get thrown out with the bathwater.” The question is, can (or will) nontraditional digital and open publishing absorb the corrector functions still currently in place?
The writer Seth Godin has quipped that “finding people to fix your typos is easy” as “most people have been brainwashed into believing that their job is to copyedit the world, not to design it.” But such a presumption about the ubiquity of online correction trivializes its labor and intellectual functions. In an e-mail, Fitzpatrick writes “that this kind of editorial work still remains important” but expresses hope that “a fair bit of it might usefully be crowdsourced” (“Re: open review”). Cohen agrees that “some of it could be crowdsourced, but there is something to having one or two people really work hard on polishing a text” (“Re: open review”). It is hard work, and not everyone is volunteering for it. Fitzpatrick is extremely aware of the sustainability challenges of crowdsourced labor and indeed has done much to focus discussions of open review and digital publishing on this important concern. Stevan Harnad, a pioneer of open review in the sciences, explains the strains upon its sustainability this way: “The expert population in any given speciality [sic] is a scarce resource, already overharvested by classical peer review, so one wonders who would have the time or inclination to add journeyman commentary services to this load on their own initiative, particularly once it is no longer a rare novelty, and the entire raw, unpoliced literature is routinely appearing in this form first.” Not least for this reason, Harnad concludes that “open peer commentary [is] an extremely powerful and important supplement to peer review, but certainly no substitute for it” (original emphasis). While Harnad does not specifically engage questions of correction either, his argument can make sense of correction’s labor by analogy: open correction might supplement, but not substitute, for its disciplined undertaking. Indeed, the particular aspects of correction’s labor may make it more difficult to sustain or crowdsource than peer review. While a commenter might be compelled to offer their time and expertise “to strengthen an argument” (“FAQ”), are they likely to undertake the tedious nonreading aspects of factual accuracy and citation checking?9 Or, as Silvia Hunter points out, to have the professional background to protect authors from legal issues like copyright infringement or libel? Are you, dear reader, checking the accuracy of all my quotes and references as you go? Or, conversely, are you prepared to relinquish them to the voluntary or automated curation of the open web or to their own scholarly obsolescence?
New Schemes for Correction
Various scenarios are already in play for the implementation, reinvention, or dismissal of the duties of formal scholarly publishing as it migrates to digital and online forms. For example, the platform for Open Journal Systems (OJS) comes with built-in roles for a wide range of editorial functions for users to customize. Its workflow replicates the traditional structures of peer-reviewed scholarly publishing, distinguishing between editorial guidance, peer review, and copyediting by the sequential transactions of electronic documents (Willinsky et al.). In Gerry Coulter’s report on “Launching (and Sustaining) a Scholarly Journal on the Internet: The International Journal of Baudrillard Studies,” he explains how a thematically focused, online academic journal with low financial overhead can function. Coulter claims that “someone has to be responsible, during the final proofreading of articles,” to adhere to standards of spelling, citation, and encoding. He and the assistant editor, Mary Ellen Donnan, copyedit “each paper before and after the HTML process …to ensure consistency and quality.”10 The distinction Coulter maintains between peer review and proofreading is suggestive of another useful distinction between professional gatekeeping and bibliographic control. Especially with a widely distributed, polyglot international journal published using standard web encoding, Coulter wants bibliographic consistency not simply as a matter of quality or value or credibility but also as a function of access and scholarly interoperability. In other words, correction may have a unique scholarly value that emerges from the conditions of open, international, online publishing.
Other systems of correction are more decentralized, distributed across a broad user base of experts and amateurs alike. The classic example is, of course, Wikipedia, whose dynamic textuality and versioning invite a different understanding of correction as curation.11 Theoretically, no centralized authority dictates the standards of bibliographic control, which are instead continually negotiated by a community of users along with standards of accuracy, coverage, consistency, and so on.12 The 2007 ITHAKA report suggests an analogous future for university publishing in which “continuous publishing ([or] ‘dynamic content’) will enable scholars to continually update or correct ‘published’ works” (Brown et al., 14). Such distributed and dynamic correction might not even have to take self-conscious or scholarly forms. It can also be outsourced, as with the example of reCAPTCHA, wherein users recommend accurate transcriptions of ambiguous snippets of texts that their sources—the New York Times and Google Books—cannot effectively deal with (“What Is reCAPTCHA?”). The corrector function of reCAPTCHA is practically clandestine, integrated into unrelated user processes (such as log-in events). In this case, bibliographic control only depends on any particular user’s assessment of what standard characters a string of OCRed text might index. These projects run on the web’s “cognitive surplus,” in Clay Shirky’s phrase. But depending on that surplus to stimulate the editing of academic work might presume a larger crowd than can reasonably be sourced.
Responding to this challenge, other distributed systems encourage users to participate in editing source texts through social media and gaming elements. For example, the National Library of Finland’s “Digitalkoot” or “Digital Volunteers” project enlists users to recommend transcriptions for otherwise ambiguous OCRed text from the library’s newspaper archive (“Digitalkoot”). Once registered on the site, users can choose to play several typing games in the style of lightweight anytime games like the hugely popular mobile app Angry Birds. Some projects balance the expected interest of users in the source material with other kinds of incentives. In the Transcribe Bentham project, users join a community to transcribe images of manuscripts that are then reviewed by editors and will be aggregated into The Collected Works of Jeremy Bentham in print. The “Benthamometer” registers the project’s progress as a whole while individual users accumulate “points” in competition with one another. The online Bite-Size Edits Project, spawned by the blog Book Oven, also uses a points scheme to entice its users to recommend improvements to snippets of creative writing. Users can cash in their accumulated points for selected books. Dickens Journalism Online is conducting a closed trial of distributed proofreading among academics, appealing to their interest in the material to generate a high-quality online resource. Project Gutenberg’s distributed proofreaders project was based on a similar concept of engaged volunteerism (Hart).
Other models look past user interactions to experiment with automating the labor of correction. In reviewing current approaches to digital preservation, Fitzpatrick considers the LOCKSS model (Lots of Copies Keeps Stuff Safe) pioneered by Stanford University Libraries. In this model, digital materials from participating libraries in various forms, editions, versions, and formats are aggregated across “an open source, peer-to-peer, decentralized digital preservation infrastructure” (“What is the LOCKSS program?”). Fitzpatrick explains that, “as the project’s name suggests, the redundancy of its distributed files creates a safety net for the material” (Planned Obsolescence, 40). But while LOCKSS is a distributed system, it also exerts its own central bibliographic control: a LOCKSS Box crawls for content, which it then compares to similar content on the network of other libraries’ LOCKSS boxes, automatically repairing differences or damage. These automatic corrections result from peer-to-peer “opinion polls”—an automated relational system that synchronizes content based on the predominance of certain copies (Maniatis et al.). The LOCKSS system preserves the integrity of content on the level of bits and bytes and also automatically migrates content formats to stave off obsolescence. While LOCKSS primarily deals with things already published, the CLOCKSS (Controlled LOCKSS) initiative archives “all journal content to which publishers allow access, both presentation files and source files” (Planned Obsolescence, 40). CLOCKSS adds to the complex relations of postpublished versions of files some of the various states of their evolution. In each case, as Fitzpatrick suggests, “the network is self-correcting” (Planned Obsolescence, 40).
Correction is not the only strategy to deal with error: search and recommendation engines like Google and Reference Extract are practicing error avoidance. In other words, these engines source the opinions and usage events of a community to derive some measure of credibility in returning and ranking results. These kinds of systems have already been compared to automated peer review (O’Malley, “Googling”), though they are more complex in accumulating data about the entire network. They do not just build in recommendations but map the patterns of what data are accessed by whom and why. That information can supply normalizing functions like automatic search suggestions, spelling correction, and citation indexing that restore a measure of correction to the work of peer review.
These examples are not meant to exhaust the current or potential approaches to correction in digital publishing or even to “error” in code, rich media, or metadata. Instead, they map a spectrum of attention to correction: from reinventing the traditional labor of academic publishing in digital environments, to distributing that labor openly and widely, to delegating it to automated agents, to dismissing its relative importance within the grander scheme of what open, digital publishing can achieve. None of these positions predominates; indeed, that might suggest an unlikely consensus that correction doesn’t amount to a “stop press” problem but can be resolved according to various needs. I will argue otherwise, not to claim that copyedited textual integrity is an independent scholarly value to be preserved at all costs in whatever media we operate, but rather to suggest how errata manifests and proliferates in its newer media and production contexts. Because the enterprise of digital publishing is remediating scholarly methods as well as resources, the evolving functions of correction have an important role in promoting the textual productivity and network effects of our published work.
The Function of Error
The web remains a textual medium. Its abundant multimodal data are inscribed by a simple character set with which we are typographically and numerically fluent—as are the protocols, ontologies, and schemata by which the web is constituted. The relational integrity of such data on and within the web is key, and facilitating the relation and interchange of data has been among the primary goals of the “semantic web.” The challenges are formidable, including (in the definition of the P2P Foundation) “resolving or mediating potential heterogeneities” from the proliferating and “discrete categories of potential mismatches.” Even beyond the problem of “semantic mediation” or resolving “what does the data mean (its semantics),” Mike Bergman suggests that we still face “crucial challenges in …reasoning over that data (inference and pragmatics), and whether the data is authoritative or can be trusted. These are the daunting—and largely remaining challenges—of the Semantic Web.” At the risk of oversimplifying the technical specificities and complexities of this undertaking, I would suggest that these goals can be usefully related to the corrector function: resolving mismatches, clarifying semantic relations, and authorizing trust in a textual media and its relational economy of ideas. If that analogy holds, then correction has a future that is vital to the evolution of the web.
Almost everyone concerned with digital publishing acknowledges that it must mean more than finding an electronic equivalence with print. The future of scholarly publishing gets right to the heart of the semantic web. In a talk titled “The Glass Box and the Commonplace Book,” Steven Johnson outlined what he considers two paths for the future of text: text in “glass boxes” (closed, nonreactive forms for looking at) and text in “commonplace books” (open, dynamic forms for interacting with). We can easily relate these metaphors to scholarly publishing, though Johnson is more broadly interested in the systemic intelligence of the web itself. He proposes a notion of “textual productivity” for how the web semantically processes and networks texts through its cascading algorithms, completely beyond the scope of any given writer or designer. Publishing in open forms facilitates these effects. Johnson argues that, “by creating fluid networks of words, by creating those digital-age commonplaces, we increase the textual productivity of the system…. [This] may be the single most important fact about the Web’s growth over the past fifteen years.”
For many scholars, this argument only underscores why scholarly publishing should open itself to the serendipitous productivity of the semantic web. But the relative inattention to correction in new publishing models may impede the very textual productivity by which the semantic web works. Because networked scholarly communication will increasingly operate through structured metadata and automated relational processes, it is imperative to consider the particular hindrances of errors within this scheme. In this sense, correction works more as “semantic debugging” for the scholarly network. Because new models of scholarly publishing are making fewer affordances for correctors, we ought to imagine the outlooks for a historical function upon which the technical promises of the future may significantly depend.
Consider again the entirely commonplace errors of misquoting or incorrectly citing sources in a research article.13 In my copyedited manuscript, I was stunned that someone tracked through databases and far-flung print sources to confirm the referential integrity of my text, the accuracy of quoted materials. Had that not happened, those errors have the potential to proliferate through a “textual productivity” all their own.14 Perhaps open review would have caught them, but I doubt it. Perhaps they would be discovered and altered through usage in a “continuous” or “dynamic” publishing model, but there is low traffic on the long tail of academic inquiry.15 Furthermore, many research sources (especially those containing historical objects) are themselves relatively untrafficked or only available offline. It could be argued that these citations could be compared to electronic surrogates of their sources or similar citations or quotes published elsewhere, generating a “correct” version through redundancy—the principle underwriting LOCKSS as well as search engine page ranking. Of course, the power of redundancy works both ways, as errors or contested representations can themselves proliferate enough to take the spotlight.16
It may become increasingly important for citations to be proofread rigorously. In the sciences at least, open publication is earning practitioners and institutional capital through an architecture of recognition: “[t]he most important driver for this change is the availability of highly accurate bibliographic and citation databases” (Hindawi, 100). Accuracy here is key, and it depends on relations that databases formalize, which in turn depend on cataloguing, description, and citational integrity, which themselves depend on the contingencies of input. Citations are prone to error as well as to intentional manipulation. For example, Jöran Beel and Bela Gipp have researched how academic content can be optimized for search engines as well as how it can exploit their relational architecture. In the first case, they emphasize the importance of good metadata and consistency in citations, names, special characters, and so on, which can otherwise hurt rankings (Beel, Gipp, and Eilde, 184). In the second, they used a “fake book,” “fake articles,” “fake authors,” and “nonsensical text” to successfully manipulate the information and rankings in online directories (Beel and Gipp). The automated network of citations, whether generated by academic-themed search engines, portals, and archives or else constituted by the associative functions of the open web at large, will significantly be shaped by the proliferation of data in accurate or perverse forms.
The interoperability of conventional citation styles (e.g., MLA, APA, Chicago, etc.) presents another challenge, as the included data can vary between them. Among the potential solutions is to describe the data capaciously enough to encompass the differences of styles and unique usages or to assign universal references to cited documents. But the universality of online referencing has problems all its own considering the tenuousness of electronic linkage. As Fitzpatrick argues, “[o]ur digital publications thus must employ a system of bibliographic identification that allows object identifiers to resolve dynamically into the correct URL as materials move” (Planned Obsolescence, 38). She notes the increasing popularity of the Handle System and digital object identifiers (DOI), which can be integrated with “resolvers” to access local pathways and resources for users in any variety of circumstances. Universal identifiers and automated resolution are the twin arms in a new system of bibliographic control. Errors or contingencies can be accommodated on the fly.
We are seeing “self-correcting networks” that improve as their data expand, including distributed or automated textual correction, networks of citation and resource access, and digital object storage systems like LOCKSS. But automation has issues. As even the casual user of word processing software knows, the auto-correct function should give us pause. Auto-correction is as liable to produce errors in “normalizing” data as it is in correcting them. This may particularly apply to cases, such as in some humanities research, where the objects or lexicons are often well outside the norm or the horizons of digital access. Of course, with the pace of digitization, that situation will change. It is possible to imagine artificial intelligence agents as the next generation of auto-complete software, able to deal with acute semantic puzzles and the remotest bits of brainy trivia. Automation would certainly answer some of the labor challenges of correction.17 It is also possible to imagine, instead of an agent, automated correction as an emergent property of the information commons as a whole.
Or it is possible to imagine that, once displaced into automated environments, we will simply stop caring. Or so copyeditors worry. For Adam Deville, a professor and editor of an academic journal himself, the decreasing interest in and practice of correction amounts to nothing less than an “epistemological crisis” (170). Hunter worries about “the possibility of a total breakdown in scholarly communication” (7). These anxieties can be forgiven of a professional community that has seen its fortunes decline. Such academic apocalypse is unlikely: there will be minor breakdowns along the new routes of networked scholarly communication, and from these we are learning how to renegotiate the editorial and corrector functions for the digital future. In doing so, we need to consider not merely the problems of error but the editorial theory that the digital futures of publishing articulate or come to embody whether by intention, emergence, or neglect.
In Radiant Textuality, Jerome McGann considers the implications of social text theory for cultural objects on the web. McGann argues that texts in any media are n-dimensional, but the web exposes textual radiance in strikingly evident and often unpredictable ways. The arc of McGann’s thinking can be traced back to his intervention in A Critique of Modern Textual Criticism, wherein he challenged a prevailing theory of copytext and the practice of editorial idealism. They had combined to produce textual editions that synthesized various sources and relied upon editorial judgments of authorial intention or best possible variants. But because the web is unbound, online editions can manifest all the witnesses and variants available to digitization. The Rossetti Archive was designed to test exactly this, and its editorial procedures required multiple and redundant levels of proofreading by its employees.18 The results are, in the language of the archive, “trusted objects”—wherein “trust” describes scholarly integrity for users as well as interoperability with the tools being concurrently developed by the Networked Infrastructure for Nineteenth-Century Electronic Scholarship (NINES), including collation, annotation, and exhibition software. The archive’s bibliographic control is not antithetical to radiance but instead helps to expose its textual manifestations.
The trouble with dynamic curation, textual redundancy engines like LOCKSS, and any self-correcting network is that they threaten to resuscitate an editorial idealism on the web. Furthermore, the automation of correction can displace editorial judgments into the synthetic functions of server polling or algorithms whose logic may be increasingly difficult to extract or kept private by commercial entities. Instead of a network of radiant texts, these functions may produce a “copyweb” of idealized content. From another angle, the editorial forgiveness of errors for the sake of scholarship’s “true quality” also risks the problematic interventions of twentieth-century bibliography. As Philip Gaskell declared, “[l]et us carry out the author’s intentions whenever we can, but not to the extent of taking pride in reproducing the manifest inadequacies of his accidentals” (359). But for the web and its semantic, structured, relational futures, the accidentals are often essential.
Simultaneously radiant and self-idealizing, dynamically new and obsolete, the web challenges us to reset our editorial thresholds within its spectrum and to grapple with the economic, logistical, and theoretical consequences of doing so. As has been argued about digital preservation, so for digital publishing our biggest challenges are not primarily technical but social (Kirschenbaum, 21). And in whatever professional networks, collaborations, open fields, or institutional arrangements we engineer to address them, correction—or at least a better appreciation of its consequences—must have a place.
NOTES
1. This is an argument about scholarly labor and attention—not necessarily an argument against error, which has its own unique functions, as Johnson suggests. Kathryn Schultz argues in her recent book Being Wrong: Adventures in the Margin of Error that “wrongness is a vital part of how we learn and change” (5). According to Daniel Rourke, “errors, noise and mistake” might encode some of the more exciting possibilities of the web’s technical and creative evolution.
2. Kathleen Fitzpatrick’s Planned Obsolescence offers a terrific survey of these topics and the discussions they have generated.
3. Trettien offers a fascinating tour of “zombie editions” of Milton and their assorted errors. See Esposito for a related discussion of the errors in the public domain copy of Pride and Prejudice preloaded on Amazon’s Kindle, driving from the original print source, digital scans, OCR, and metadata.
4. What counts as an error? Rather than precisely defining or categorizing them, this essay considers errors in a differential relation with the labor of correcting them, a relation that changes with historical and production contexts.
5. That is not unusual, according to long-time academic copyeditor Sylvia Hunter: “[c]opy (or manuscript) editors are rarely mentioned on journal mastheads; we seldom make our way into authors’ acknowledgement footnotes; we toil in silence, invisible” (8).
6. One might imagine the great Victor Borges’s routine on “Phonetic Pronunciation.”
7. Some telling perspectives from the field: As Merill Perlman, former chief of copy desks at the New York Times, sarcastically complains, “Who needs copy editors? They just get in the way. Without them, blog postings can go up faster; without them, readers get a writer’s words unfiltered; without them, a publication can save lots of money” (Thomas). Sylvia Hunter suggests “that some scholarly journal publishers in the United States were considering, or had already begun, phasing out the crucial stages between final manuscript submission and typesetting. In other words—neatly defeating all of our own well-reasoned arguments about the value added to research by scholarly publishers—slapping a nice cover on exactly the same text that the author has already posted on her Web site” (7). Adam Deville blames electronic writing and publishing for putting increasing pressure on the decreasing resources of copyediting. My own experience as an “editor” at a new media company came to a close in the early 2000s when nearly the entire editorial department was laid off; the management and the technical staff survived.
8. I take the following as representative: an American Council of Learned Societies report Crises and Opportunities: The Futures of Scholarly Publishing; “University Press Forum: Variations on a Digital Theme,” in the Journal of Scholarly Publishing 38, no. 4 (July 2007); the ITHAKA organization’s report on “University Publishing in a Digital Age” (2007); a special issue of the Journal of Electronic Publishing on “Reimagining the University Press” (vol. 13, no. 2, Fall 2010); and the Association of American University Presses report on Sustaining Scholarly Publishing: New Business Models for University Presses (March 2011).
9. I put this to an unscientific test, carelessly selecting a humanities article on Philica.com (“The instant, open-access Journal of Everything”) to work on. But instead of engaging in the “transparent peer-review” the website seeks from its users, I put on the old editorial assistant hat and turned up a handful of errors: typos in citations, capitalization errors, absent words, sentence fragments, and so on. I will risk not sharing the specifics, as it seems churlish to identify an author who really is not at fault: everyone makes these mistakes and struggles to catch their own.
10. In his article “Scholar as E-Publisher,” Thomas H. P. Gould concurs that long-tail niche journals will still require editorial and technical staff that are separate from open peer review functions but does not identify correction (442).
11. See Fitzpatrick, Planned Obsolescence (31) for a fuller discussion of the concept.
12. Wikipedia’s own openness looks a little different in practice. As David Golumbia has pointed out, a small core of under two hundred Wikipedia editors now operate just like any other editors, maintaining the standards of bibliographic control across the site.
13. For a more expansive and colorful list of the kinds of errors academic copyeditors testify to continually finding, see Adam Deville (169) and Sylvia Hunter (8–9).
14. There is a counter argument that errors in print are even more insidious: “When some of these things sit around in the scientific literature for a long time, they can do damage: they can influence what people work on, they can influence whole fields,” according to David Goldstein, the director of Duke University’s Center for Human Genome Variation (Mandavilli).
15. A telling quote from a discussion of open review in the sciences: “‘Most papers sit in a wasteland of silence, attracting no attention whatsoever,’ says Phil Davis, a communications researcher at Cornell University in Ithaca, New York, and executive editor of The Scholarly Kitchen, a blog run by the Society for Scholarly Publishing in Wheat Ridge, Colorado” (Mandavilli).
16. “Google bombs” offer an interesting example of the intentional manipulation of collective agreement. Even with the best of intentions (or without intention at all), the “group think” of a specialized online community can amplify errors with cascading effects, as Sunstein has argued (75).
17. In a Twitter post on March 2, 2011, Tim Carmody (@tcarmody) asks, “Can we put Watson from Jeopardy to work correcting bad OCR? (I’m looking at you, Google Books.)”
18. Including a version of aural sing-song proofreading of transcription and markup done by assistants in pairs, which ironically (though unintentionally) adapts the proofing practices that were possibly undertaken for Rossetti’s own books.
BIBLIOGRAPHY
Alonso, Carlos J., Cathy N. Davidson, John M. Unsworth, and Lynne Withey. Crises and Opportunities: The Futures of Scholarly Publishing. New York: American Council of Learned Societies, 2003.
Association of American University Presses. Sustaining Scholarly Publishing: New Business Models for University Presses. New York: Association of American University Presses, 2011. http://mediacommons.futureofthebook.org/mcpress/sustaining/.
Bartlett, Rebecca A., Richard Brown, Kathleen Keane, Bruce Wilcox, Niko Pfund, and Thomas Bacher. “University Press Forum: Variations on a Digital Theme.” Journal of Scholarly Publishing 38, no. 4 (July 2007): 211–28. http://muse.jhu.edu/journals/scp/summary/v038/38.4bartlett01.html.
Beel, Jöran, and Bela Gipp. “Academic Search Engine Spam and Google Scholar’s Resilience Against it.” Journal of Electronic Publishing 13, no. 3 (December 2010). Web http://hdl.handle.net/2027/spo.3336451.0013.305.
Beel, Jöran, Bela Gipp, and Erik Eilde. “Academic Search Engine Optimization (ASEO).” Journal of Scholarly Publishing 41, no. 2 (January 2010): 176–90. http://muse.jhu.edu/journals/journal_of_scholarly_publishing/v041/41.2.beel.html.
Benjamin, Walter. Charles Baudelaire: A Lyric Poet in the Era of High Capitalism. Translated by Harry Zohn. London; New York: Verso, 1973.
Bergman, Mike. “What Is the Structured Web?” AI3. July 18, 2007. http://www.mkbergman.com/390/what-is-the-structured-web/.
Brown, Laura, Rebecca Griffiths, and Matthew Rascoff. University Publishing in a Digital Age. New York: ITHAKA, 2007.
Cohen, Dan. “Open Access Publishing and Scholarly Values.” Dan Cohen’s Digital Humanities Blog. May 27, 2010. http://www.dancohen.org/2010/05/27/open-access-publishing-and-scholarly values/.
———. “Re: open review, digital publishing, and copyediting.” E-mail to Paul Fyfe. March 9, 2011.
———. “The Social Contract of Scholarly Publishing.” Dan Cohen’s Digital Humanities Blog. March 5, 2010. http://www.dancohen.org/2010/03/05/the-social-contract-of-scholarly publishing/.
Coulter, Gerry. “Launching (and Sustaining) a Scholarly Journal on the Internet: The International Journal of Baudrillard Studies.” Journal of Electronic Publishing 13, no. 1 (Winter 2010). http://hdl.handle.net/2027/spo.3336451.0013.104.
Deville, Adam. “Sinners Well Edited.” Journal of Scholarly Publishing 39, no. 2 (January 2008): 168–73.
“Digitalkoot: Electrifying Our Cultural Heritage.” Helsinki: National Library of Finland, 2011. http://www.digitalkoot.fi/.
Dougherty, Peter J. “Reimagining the University Press: A Checklist for Scholarly Publishers.” Journal of Electronic Publishing 13, no. 2 (Fall 2010). http://hdl.handle.net/2027/spo.3336451.0013.202.
Esposito, Joseph. “The Terrible Price of Free: On E-reading Jane Austen via Google’s Ebooks.” The Scholarly Kitchen. March 14, 2011. http://scholarlykitchen.sspnet.org/2011/03/14/the-terrible-price-of-free-on-e-reading-jane-austen-via-googles-ebooks/.
““FAQ.” Open Review: “Shakespeare and Performance.” Shakespeare Quarterly. February 2011. http://mediacommons.futureofthebook.org/mcpress/shakespearequarterlyperformance/faq/.
Fitzpatrick, Kathleen. Planned Obsolescence: Publishing, Technology, and the Future of the Academy. Media Commons, 2002–present. http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/.
———. “Re: open review, digital publishing, and copyediting.” E-mail to Paul Fyfe. February 16, 2011.
Gaskell, Philip. A New Introduction to Bibliography. New Castle, Del.: Oak Knoll, 1972.
Godin, Seth. “Who Will Say Go?” Seth Godin’s Blog. March 1, 2011. http://sethgodin.typepad.com/seths_blog/2011/03/who-will-say-go.html.
Gould, Thomas H. P. “Scholar as E-Publisher.” Journal of Scholarly Publishing 41, no. 4 (July 2010): 428–448. http://muse.jhu.edu/journals/journal_of_scholarly_publishing/v041/41.4.gould.html.
Harnad, Stevan. “The Invisible Hand of Peer Review.” Exploit Interactive 5 (April 2000). http://www.exploit-lib.org/issue5/peer-review/.
Hart, Michael. “Gutenberg: The History and Philosophy of Project Gutenberg.” Project Gutenberg. August 1992. http://www.gutenberg.org/wiki/Gutenberg:The_History_and_Philosophy_of_Project_Gutenberg_by_Michael_Hart.
Hindawi, Ahmed. “2020: A Publishing Odyssey.” Serials 22, no. 2 (July 2009): 99–103. http://knol.google.com/k/2020-a-publishing-odyssey#.
Hunter, Sylvia. “Why Copy Editors Matter.” Journal of Scholarly Publishing 36, no. 1 (October 2004): 6–14. http://muse.jhu.edu/journals/journal_of_scholarly_publishing/v036/36.1hunter.html.
Johnson, Steven Berlin. The Ghost Map. New York: Riverhead, 2006.
———. “The Glass Box and the Commonplace Book.” stevenberlinjohnson.com. April 23, 2010. http://www.stevenberlinjohnson.com/2010/04/the-glass-box-and-the-commonplace-book.html.
Kimball, Shana. “Re: open review, digital publishing, and copyediting.” E-mail to Paul Fyfe. February 20, 2011.
Kirschenbaum, Matthew G. Mechanisms: New Media and the Forensic Imagination. Cambridge, Mass.: MIT Press, 2008.
Lankes, R. David, ed. Reference Extract. Syracuse University, 2011. http://www.referencextract.org/.
Mandavilli, Apoorva. “Peer Review: Trial by Twitter.” Nature 469 (January 19, 2011): 286–87. http://www.nature.com/news/2011/110119/full/469286a.html.
Maniatis, Petros, et al. “Preserving Peer Replicas by Rate-Limited Sampled Voting.” In ACM SIGOPS Operating Systems Review, 44–59. New York: ACM, 2003. http://cs.rochester.edu/meetings/sosp2003/papers/p140-maniatis.pdf.
McGann, Jerome J. A Critique of Modern Textual Criticism. 1983. Charlottesville; London: University Press of Virginia, 1992.
———. Radiant Textuality: Literature After the World Wide Web. New York: Palgrave, 2001.
McGuire, Hugh, Andy MacDonald, and Janina Szkut, eds. Bite-Sized Edits. Montreal, 2011. http://bookoven.com/projects/bitesizeedit/.
Nunberg, Geoffrey. “Counting on Google Books.” Chronicle of Higher Education. December 16, 2010. http://chronicle.com/article/Counting-on-Google-Books/125735/.
———. “Google’s Book Search: A Disaster for Scholars.” Chronicle of Higher Education. August 31, 2009. http://chronicle.com/article/Googles-Book-Search-A/48245/.
O’Malley, Mike. “Academic Editor 2.0.” The Aporetic. March 3, 2011. http://theaporetic.com/?p=1679.
———. “Googling Peer Review.” The Aporetic. October 19, 2010. http://theaporetic.com/?p=446.
P2P Foundation. “Semantic Web.” http://p2pfoundation.net/Semantic_Web.
Pochoda, Phil. “UP 2.0: Some Theses on the Future of Academic Publishing.” Journal of Electronic Publishing 13, no. 1 (Winter 2010). http://hdl.handle.net/2027/spo.3336451.0013.102.
Rourke, Daniel. “Errors in Things and the ‘Friendly Medium.’” Research Seminar Series in Open Media. Coventry School of Art and Design. February 15, 2011. http://machinemachine.net/text/ideas/errors-in-things-and-the-friendly medium.
Scheinfeldt, Tom. “Re: open review, digital publishing, and copyediting.” E-mail to Paul Fyfe. March 8, 2011.
Schulz, Kathryn. Being Wrong: Adventures in the Margin of Error. New York: HarperCollins, 2010.
Shaw, John, and Michael Slater, eds. Dickens Journalism Online (DJO). University of Buckingham, 2006–present. http://www.buckingham.ac.uk/djo.
Shirky, Clay. Here Comes Everybody: The Power of Organizing Without Organizations. New York: Penguin, 2008.
Sunstein, Cass R. Infotopia: How Many Minds Produce Knowledge. New York: Oxford University Press, 2006.
Thomas, Ashley. “The Importance of Copy Editing, Copy Editors: Some Folks Just Don’t Get It. Points to Ponder from Merrill Perlman.” Temple University Editing Residency (blog). July 8, 2008. http://djnf08.wordpress.com/2008/07/08/the-importance-of-copy-editing-copy-editors-some-folks-just-dont-get-it-points-to-ponder-from-merrill-perlman/.
Transcribe Bentham. Bentham Project, University College of London. 2010–present. http://www.ucl.ac.uk/Bentham-Project/transcribe_bentham.
Trettien, Whitney. “Zombie Editions: An Archaeology of POD Areopagiticas.” diapsalmata. December 15, 2010. http://blog.whitneyannetrettien.com/2010/12/zombie-editions-archaeology-of-pod.html.
Unsworth, John. “Documenting the Reinvention of Text: The Importance of Failure.” Journal of Electronic Publishing 3, no. 2 (1997). http://hdl.handle.net/2027/spo.3336451.0003.201.
Werner, Sarah. “Re: open review, digital publishing, and copyediting.” E-mail to Paul Fyfe. March 9, 2011.
“What Is the LOCKSS Program?” LOCKSS Program. Stanford University, 2004–present. http://lockss.stanford.edu/lockss/Home.
“What Is reCAPTCHA?” Google. Stanford, Calif., 2011. http://www.google.com/recaptcha/learnmore.
Willinsky, John, Kevin Stranack, Alec Smecher, and James MacGregor. “Open Journal Systems: A Complete Guide to Online Publishing.” 2nd ed. Burnaby, B.C., Canada: Simon Fraser University Library, 2010. http://pkp.sfu.ca/ojs/docs/userguide/2.3.3/index.html.