PART I ][ Chapter 1

Gender and Cultural Analytics: Finding or Making Stereotypes?

Laura Mandell

Feminist cultural theorists, and perhaps cultural theorists more generally, are discouraged from exploring cultural analytics, or “the computational study of culture,”[1] whenever data miners state their conclusions in terms of “males” and “females.”[2] In the fields of cultural analytics, computational linguistics, and quantitative sociology, such work typically appears in articles with “gender” in the title—paradoxically because “male” and “female” are biological sex terms rather than gender terms.[3] As a keyword for academic sociological and psychological articles, “gender” rose in popularity starkly over the 1980s, from 50 articles between 1966 and 1970 to 1,450 articles between 1980 and 1985 (Haraway, “Gender,” 57). It was of a political moment: as Donna Haraway tells us, gender was “a concept developed to contest the naturalization of sexual difference in multiple arenas of struggle” (53) by feminist writers of the 1980s, culminating in Joan Scott’s seminal 1986 essay, “Gender: A Useful Category of Historical Analysis.” Gender is historicizable because it is culturally constructed. When psychoanalyst Robert Stoller first coined the term “gender identity” in 1967, he contended “that sex was related to biology (hormones, genes, nervous system, morphology), and gender was related to culture (psychology, sociology)” (Haraway, “Gender,” 55).[4] Therefore, using the biologistic binary “male” and “female” (M/F) to speak of gender is nearly an oxymoron.

Feminist cultural critics might be tempted to simply walk away from cultural analytics altogether rather than engage with what I call “M/F”—that is, the conflation of gender with sex and the naturalization of a culturally constructed binary opposition. But walking away is a bad idea. During the 1980s, feminist sociologists critiqued patriarchal notions of objectivity primarily by turning away from quantitative analysis toward the qualitative (Gorelick, 465; Hartsock, 232). For Joey Sprague and Mary Zimmerman, the failure of feminism to have as transformative an impact on sociology as it has in had “in ‘anthropology,’ ‘history,’ and ‘literary criticism’” resulted from feminist sociologists’ preference for qualitative analysis (84). Quantitative analyses in sociology proceeded without feminist input at great cost, leaving the field susceptible to “bad science” (Haraway, “Gender,” 55) and “bad description” (Marcus, Love, and Best, 6).

In addition to ensuring that feminist issues are addressed in the field, investigating M/F computational analyses also gives us the opportunity to revisit a blindness pervasive in 1980s gender analysis. For one thing, we now commonly accept that genders are not reduced to only two, nor are they wholly distinct from sex: cisgender, transgender, and genderqueer, for example, designate not masculine or feminine gender identities, but instead the relationship a person has to his or her assigned gender (LaBrada). Furthermore, a person’s sex/gender, as so-called biological sex is now sometimes described, is no longer taken as a given: it is “assigned” to a person by medical and familial institutions. That a person is given a sex/gender after being born is made salient in cases of chromosomal and genital variation, but gender assignment nonetheless happens in every instance. The video montage for the opening credits of Jill Soloway’s comedy-drama web television series, Transparent, does an excellent job of displaying women and men more or less easily assuming their assigned gender roles or, alternately, stepping outside them—so much so that, when a clip of a young girl joyously twirling her skirt appears in the montage, one realizes that she is celebrating her own capacity to fully embrace and express her incipient womanhood: hers is a moment of cisgender triumph—that is, a feeling of triumph for an achievement.

This understanding of sex, gender, and their relation owes much to the work of feminists in the 1990s, who extended into thinking about sex categories the work done by feminists in the 1980s to question the role of nature in determining gender categories. Judith Butler, for example, disturbed the boundaries of the sex/nature versus gender/culture opposition in Gender Trouble (1990): “Gender ought not to be conceived merely as the cultural inscription of meaning on a pregiven sex (a juridical construction); gender must also designate the very apparatus of production whereby the sexes themselves are established” (7). The “apparatus of production” is for Butler not only a legal and medical intervention. The biological binary M/F opposition is a textual and a numerical one: it is written, printed, spoken—discursive.

Some currently publishing data miners, sociolinguists, and sociologists uncritically conflate sex with gender, regressing to the pre-1980s intellectual milieu in which the M/F categories were treated as designating natural, transparent, biological categories. But some do not. For example, studies by Mark Olsen (“Qualitative Linguistics and Histoire des Mentalités: Gender Representation in the Trésor de la Langue Française,” 1992) as well as by Ted Underwood and David Bamman (“The Instability of Gender,” 2016), theorize gender as a category in the making, rather than as a preexisting fact. Using data-mining techniques on the ARTFL corpus (texts gathered for the project, American and French Research on the Treasury of the French Language[5]), Olsen has shown that stereotypical associations with the category of woman change over time (“Qualitative Linguistics,” 13). This work by Olsen, as well as that by Underwood and Bamman, demonstrates that computational methods offer the opportunity to denaturalize gender categories. But to capitalize on this opportunity, we must remain attentive to where and how computational data analytics crosses the line into naïve empiricism—often unwittingly—by presenting conclusions about “male” and “female” modes of thinking and writing as if the M/F terms were simple pointers to an unproblematic reality, transparently referential and not discursively constituted.

My argument in this chapter is that, in some quantitative cultural analyses, the category of gender has been biologized, binarized, and essentialized in a trend that I call “stereotyping,” with an eye on the historical and material practice of stereo-typography. Stereotyping as a material practice was invented before the printing process became industrialized (1725 in England, 1735 in France). It froze pages made from movable type into unalterable plates of textual data for reprinting and was adopted early on by the French printing house known as Didot (Hodgson 15–18). The term for “stereotype” in French is cliché.[6] Here, I show how data-mining techniques are sometimes used to produce clichés. But my goal is less to dismiss quantification than it is to argue that, insofar as the results of data mining are movable and manipulable, cultural analytics can serve us well in historicizing gender categories. Examining the gender problem in quantitative analysis reveals both opportunities afforded by digital media and how we might miss them.

Text mining devolves from process into fixed results via stereotyping, whether literally imprinted on paper or figuratively frozen into cliché. In contrast, acknowledging that numbers have to be read—that is, resisting the view that statistical results are meaningful in any self-evident way—takes place by close reading of computational results and often manipulating parameters, adjusting algorithms, and rerunning the data. Dynamically reading, generating, and analyzing numbers are essential to making interpretive quantification possible and thus to exploring how the M/F binary is constituted in any given historical moment. This chapter examines two instances of gender analysis, performed via stylometry, word-frequency analysis, and topic modeling, that veer toward naïve empiricism because they take the distinction between M/F to be transparent. Imbuing cultural analytics with feminist theory, I argue here, shows us that we can animate numerical processes rather than fixing their results as stereotype. New technologies and carefully created reading processes can transform the statistical analysis of cultural artifacts into dynamic readerly engagement.

Bias and Statistics

Matthew Jockers undertakes what he calls “macroanalysis,” he says explicitly, in order to strip literary criticism of its interpretive bias. In his book of the same name, he defines macroanalysis as “computational approaches to the study of literature” (4).[7] Despite later conciliatory gestures, Jockers unequivocally discredits more interpretive literary approaches, including close reading, in the book’s first pages: “Close reading is not only impractical as a means of evidence gathering in the digital library, but big data render it totally inappropriate as a method of studying literary history” (7). For Jockers, close reading is “totally inappropriate” because it unleashes “irrepressible biases.” Here, he takes a peculiar theoretical step that allows him to implicitly claim that biases can be dismantled by text mining: “Observation is flawed in the same way that generalization from the specific is flawed.” According to this line of reasoning, interpretive generalization fails only as a result of working with too small a sample size (6–7). For example, Ian Watt’s interpretation of the rise of the novel, Jockers says, turns out to be “correct on a number of points” despite being performed on too small a number of texts (7, emphasis added). His own assessment, because it is based on more texts, has therefore incontrovertibly established Watt’s thesis, despite many critical studies that contest it.[8]

Before discussing Jockers’s conclusions concerning gender and literary style, I want to look closely at his methods. One is stylometry, which the O.E.D. defines as “the statistical analysis of variations between one writer or genre and another.” In Macroanalysis, however, stylometry is used to analyze variations between “male” and “female” writers of novels. Stylometrists typically delineate styles using very small words, not those that are related to the topic: the difference between the styles of James Madison and Alexander Hamilton as visible in the Federalist Papers, for instance, hinges on how often one uses “a” and the other “and” (Jockers, 92). Julia Flanders describes the work of John Burrows, another proponent of stylometric analysis, as discovering “distinctions of idiolect” usually invisible to readers; for instance, the frequency with which “‘inert words like ‘of’ and ‘the’” are used (56). At first, Jockers sticks somewhat to the standard method: he describes the two features that most distinguish “the gender signal” as the higher usage of feminine pronouns and of the colon and semicolon. Jockers acknowledges that feminine pronouns are used more in novels written about women, so this stylistic feature is more about the content of novels written by women than about their style. But more problematically, as textual editors well know, punctuation (“accidentals,” in the parlance of textual editing) is often manipulated by editors of print editions. What, we might well ask, were the genders of the editors?

Jockers further deviates from Burrows’s method in producing lists of “features” used by female authors, by which he means specific vocabularies or keywords: “It is also seen that women authors show higher usage of the words heart and love—not surprising in the sense that this fact is consistent with our stereotypes of ‘feminine’ prose” (Macroanalysis, 93). Here, Jockers veers away from stylometry into simple counting. But there is a bigger problem. Not all data miners would attribute to a dataset spanning 200 years “our stereotypes.” This phrase controverts the very point Joan Scott was making by introducing the term “gender” into cultural history: that gender associations change over time.[9]

A study by Jan Rybicki seeks to build on Jockers’s work. Rybicki uses stylometrics to determine the gender of twelve anonymous authors in a collection of fiction held by the Library at Chawton House, which used to belong to Jane Austen’s brother Edward and now proclaims itself to be “home to early women’s writing.”[10] Rybicki supplements this small corpus of forty-six novels with novels written by canonical authors, including men. He claims that he is able to sort these novels by the gender of their authors by using the same technique that sorted the canonical authors by gender in order to determine the genders of the anonymous authors in the Chawton House dataset. He too relies on the frequency of substantive vocabulary rather than of inert words. His conclusion resembles that of Jockers: “The Chawton House authors exhibit a decided preference for words that would be most readily and stereotypically associated with sentimentalist feminine fiction, and the fact that the stereotype is visible in vocabulary statistics makes it, perhaps, less of a stereotype” (754). Among the gendered characteristics that Rybicki has imagined as “less of a stereotype” is the finding that “men like virtues,” by which he apparently means to suggest that men really do always care more about virtue than do women. His vocabulary statistics, however, show women using “virtue” and “virtuous” frequently and both men and women using “honest” (9). More obviously than in Jockers’s analysis, the stereotype is in the eye of the beholder.[11]

Leaving aside Jockers’s and Rybicki’s reliance on stereotypes to explain their results, it is worth exploring whether the results themselves can ever be taken as objective. Results achieved by Mark Olsen contradict those of Jockers and Rybicki: he finds no such thing as a “feminine style” in his data. Although contradiction itself cannot overturn claims for objectivity, Olsen’s method reveals the extent to which Rybicki’s and Jockers’s results are based on a preexisting assumption. Olsen deliberately does not look at word frequency (“Écriture féminine”). For Olsen, what distinguishes feminine writing is not vocabulary because it is tied to topic. For him, a gendered style consists in using the same words, but in a different way. He therefore examines the contexts of specific words, concluding that the meanings of words do not diverge based on gender. For Olsen, then, women and men make deliberate decisions about how to use specific words. Therefore, Olsen was doing something other than counting words according to an author’s gender because he believes topic to be the overriding factor, not gender, in determining which terms will be used. Olsen’s articulated premise for his study reveals Jockers’s and Rybicki’s underlying assumption: choices about diction are not the product of writerly decisions related to topic, but instead are based on being either M or F. As will become more and more clear, Jockers’s and Rybicki’s results confirm not an objective reality, but the premise of their own studies.

The criterion of scientific objectivity is reproducibility.[12] Before finding his own “female” words, Rybicki attempted to sort a subset of his corpus by using Jockers’s list of “features best distinguishing male and female authors” (94, qtd. Rybicki, 5) and a second list created by James Pennebaker in a book called The Secret Life of Pronouns. Neither list worked; he had to generate a new list of “gender signals” from his own corpus. Rybicki attributes the failure of Pennebaker’s wordlist to effectively sort gender in Rybicki’s own corpus to the fact that the two corpora cover different time spans. But he also observes that Jockers’s corpus contains “some of the authors in my [canonical] corpus” (5). Rybicki mitigates the failure of Jockers’s list by noting certain overlaps with his own (5) and concludes that his own list “makes sense in terms of Pennebaker’s and Jockers’s findings” (7). But the fact remains that Rybicki could not use either of the two lists to accurately sort his corpus. His attempt to replicate Pennebaker’s and Jockers’s findings failed.

Clearly, the ability to distinguish gender via particular words depends upon the construction of a particular corpus. Here it is important to notice a moment of clarity in Rybicki’s richly contradictory article. It is not simply that there are biases intrinsic to data selection (see also Koolen and van Cranenburgh, 14–15), but that irreproducibility suggests the instabililty of discursively constructed gender categories: “Literary scholarship has never voiced it as clearly as quantitative research has that the notions of ‘writing like a man’ and ‘writing like a woman’ are in such a constant state of flux that they risk becoming, in fact, quite problematic” (14). While one might argue with Rybicki’s assessment of “literary scholarship”—Toril Moi’s Sexual/Textual Politics of 1985 summarized all the work in the field that problematized “writing like a woman”—his claim needs to be trumpeted loudly among data miners: it is “literary evolution” and not bodies that assign keywords to M/F.

Rybicki concludes by noting the “inability, in this study, to produce a stable ‘canon’ of male and female keywords that would survive a change of corpus or shifts in literary evolution, or both” (14). He proudly attributes this insight to “quantitative research,” which is better, he says, than ordinary “literary scholarship” in figuring out that “writing like a woman” changes through time. Why, then, is this claim so inconspicuous as to not even inform the title of his article (“Vive la différence!”) or the main argument—that he can determine the gender of anonymous authors in the Chawton House collection?

Gender difference tends to be widely broadcasted, and anything that obfuscates it downplayed. For instance, Macroanalysis made the news because it “shows that, on a whole, female authorship is detectable by objective measures rather than just human intuition” (the Smithsonian’s Smart News, Nuwer). In her meta-analysis of psychological gender studies, Janet Sibley Hyde argues that the popular press picks up on and touts gender differences, even in studies that conclude there are very few marked differences. Studies of psychological gender traits have, since 1879, been reporting more similarities than differences. Most striking, in The Psychology of Sex Differences, Eleanor Maccoby and Carol Jacklin argued that most recorded gender differences did not hold up to scrutiny, and yet their work is reported in textbooks and reviews as discovering difference, not similarity. In her analysis of this fact, Hyde observes, “The mass media and the general public are captivated by findings of gender differences” (581).

In her own work, Hyde has advanced what she calls “the gender similarities hypothesis, which holds that males and females [sic] are similar on most, but not all, psychological variables” (581, emphasis added). Compiling and comparing many studies of gender difference, Hyde concludes that her gender similarity hypothesis proves true except in the case of some physical differences (throwing distance) and some aspects of sexuality (590). Stereotypical differences such as higher aggression levels and better mathematical capacities turn out to be attributable to men only in supervised tests; that is, when women know they are being watched or tracked for evidence of their gender (589; s.a. Cole, 178). Hyde’s gender similarities hypothesis is consistent with linguist Kate Harrington’s deeper reading of the statistics concerning gender differences in language. Harrington has looked more closely at the numbers generated by gendered discourse analysis, examining rather large differences in standard deviations between those classified in each gender. However, if one leaves out the extreme cases—those who rank most highly in gendered characteristics—most women resemble most men: “the often cited difference [is] dependent on the few and not representative of the many” (96). Major differences in standard deviation between the (allegedly) two genders can indicate that a dataset warrants further investigation, but the standard deviation will not be markedly different if there are exaggerators in both groups: the “range of individual variation . . . needs to be interpreted” (96, Harrington’s emphasis). Moreover, Harrington finds that those people who grossly exaggerate their own “stereotypical” gendered discourse are members of specifically identifiable groups. It is the intersection of identities that matter, not gender alone: any “speaker’s use of [allegedly gendered discourse forms] has more to do with individual social context than with gender as a social category” (101). The “individual social contexts” identified by Harrington as displaying the largest gendered differences were teenage girls (97) and men who seemed especially threatened by the study of gender differences in which they were participating (100). “The only overt comment on ‘doing gender’ in my data,” Harrington says, arises in an explicit discussion of masculinity using the now infamous genre of discourse, “Real men don’t do X” (99–100).

In my view, the exaggerators indicate, again, that gender identity is not given, but is triumphantly attained through the course of developmental acculturation (in the case of the teens) and sometimes is an insecure achievement (in the case of the threatened men). People who adopt and exaggerate stereotypically gendered traits (as they understand them) are worried that others might see their gender as ambiguous. That gender panic may intervene in sociological studies offers insight into why differences are seized on and trumpeted by the popular press. Gender is bifurcated into two categories, and people fit themselves into one or the other with difficulty and trepidation. That process requires continuously reassuring ourselves that identifying (as) M or F is no trouble at all. Reportage of firm gender demarcations is an apotropaic gesture designed to ward off anything that undermines a symbolic difference essential to the Western world’s self-understanding.[13]

Given this discussion of gender studies, what can we conclude about the extent to which statistical reading techniques can mitigate irrational symbolic investments? Statistics, Isabelle Stengers notes, are deeply conservative in how they build on previous results (n. 10, 263). For instance, Bayesian probability, the foundation of latent Dirichlet allocation (a form of topic modeling, another statistical technique, discussed later in this chapter), determines events to be more probable if similar events have already happened. More generally, probability contributes to the biases of computer-assisted racial profiling (Hardt; O’Neil) and even Google search returns (Noble).

However, the discipline of statistics has developed methods for ameliorating bias. Ben Merriman notices Franco Moretti’s and Jockers’s tendency to avoid using known statistical methods for reading their results. In particular, they rely a great deal, Merriman says, on statistical significance for their arguments, when in fact statistical significance proliferates everywhere one looks for it. If a dataset of novels were categorized according to novelists with long noses and novelists with short noses, we would find statistically significant differences; they simply would not be meaningful differences. Although many researchers turn to qualitative analysis to determine the substantive value of a difference (Fitz-Gibbon, 48), there are also quantitative methods for estimating whether a difference is meaningful or its “effect size” (Coe). The effect size, or “how much difference a difference makes (the magnitude of the effect)” (Fitz-Gibbon, 45), is typically used for “meta-analyses” such as Hyde’s, but it need not be used only for comparing multiple statistical results from different studies. To be fair to Moretti and Jockers, determining effect size is neither typical nor easy. As Robert Coe (1) tells us, “despite the fact that measures of effect size have been available for at least 60 years (Huberty, 2002), and the American Psychological Association has been officially encouraging authors to report effect sizes since 1994—but with limited success (Wilkinson et al., 1999).”[14] What the need for measuring effect size indicates, however, is that, as Merriman puts it, “patterns may be statistically significant but substantively trivial: the magnitude of the differences (the ‘effect size’) may be extremely small, or it could disappear entirely if the researcher tweaks the model” (16). The irreproducibility of gendered keywords confirms Merriman’s hypothesis.

But even after determining effect size, identifying statistical trends does not remove bias. On the contrary, it reveals the biases of the paradigm according to which a thing or event is counted as data (Forrester, 36, qtg. Hesse, 286). Statisticians well know that biases are reproduced through classification systems (Cartwright and Bradburn, 5). Comparing the work done by Burrows, Flanders, and Olsen to Jockers and Rybicki (discussed on pp. 6–8 above) reveals a bias toward biological determinism.[15]

“Gender” was proposed as an analytic category in the 1980s specifically to avoid the deterministic bias. According to Donna Haraway, it “was developed as a category to explore what counts as a ‘woman’” through time (“Gender,” 67). But setting aside “as a ‘woman’” from Haraway’s statement for a moment, What counts?[16] The question becomes axiological: What really has value for us, and why? I sketched out earlier some thoughts about the symbolic weight and psychological drama that obfuscate for us the conceptual inadequacy of the M/F categories. There is also a material side to the redacted question: What has been charged with the procedure of counting? It would be very easy to answer, in the case of cultural analytics, that what are responsible for counting are data-mining techniques and data miners. But that answer rings true only if the data itself is not at all actively participating in counting, unlike Harrington’s interviewees, who very actively generate a feedback loop. Are there feedback loops built into data? Has it been collected, sorted, and even produced according to the categories M/F? It is to answer those questions, in the case of text-mining novels, that I now turn.

Data as Capta

In a keynote collaborative presentation with Julia Flanders at the Boston Area Days of Digital Humanities Conference in 2013, Jockers explained the difference between the visualizations that he originally generated for Macroanalysis and those finally published in the book: in the course of revision, Jockers discovered that there were duplicate novels in his dataset and that three-volume novels had not been stitched together to count as one novel instead of three. Changing his dataset also altered a dramatic visualization of gender difference included in an earlier publication (“Computing and Visualizing,” Figure 3), transforming it into the less distinguishing visualization that appears both in Macroanalysis (Figure 9.3.4) and in Smithsonian Smart News (Nuwer).[17] In other words, the stylistic differentiation between M/F genders no longer looked so significant once Jockers tweaked his dataset.

Writing on the subject of the visual effect of data visualizations more generally (Graphesis, 129), Johanna Drucker argues that the “representational force” of the image “conceals what the statistician knows very well—that no ‘data’ pre-exist their parameterization. Data are capta, taken not given, constructed as an interpretation of the phenomenal world, not inherent in it” (128, emphasis in the original). Datasets are never random samples: they are gathered by humans according to parameters, however complex, hidden, unconscious, and subject to chance they may be.

Drucker’s formulation of data as “capta” cuts two ways. First, why collect data according to the categories M/F? Capturing data as M/F will create statistically significant differences between M/F, but are they significant? And more importantly, what exactly do they signify? Second, thinking about data as capta encourages us to examine the selection process in aggregating data. The shape of Jockers’s dataset was decisive in determining his results, as is clear from Rybicki’s proudly avowed inability “to produce a stable ‘canon’ of male and female keywords that would survive a change of corpus, or shifts in literary evolution, or both” (14).

It is a virtue of Rybicki’s stylometric analysis of the Chawton House Collection of novels that he openly describes the process of tweaking his dataset—changing, over the course of his experiment, which novels by men he would include or exclude—to get his algorithm to successfully differentiate authors marked as M/F. Rybicki tells us that he removed from his corpus the “troublemakers” who disturbed his results too deeply because it is “notoriously too difficult to gender classify” (3). That is, gender classification does not work without help.

Rybicki’s adjustments are not surreptitious: he is trying to get his algorithm to work through acts “of honest and well-founded cherry-picking” (6). In a 2013 article, Alan Liu quotes a question broached in 2011 by Ryan Heuser and Long Le-Khac: “How do we get from numbers to meaning?”[18] Liu argues that this “meaning problem” (“What is the Meaning,” 411) is not simply a matter of interpretation after data has been generated, describing abstractly what we can see in the practical way that Rybicki works:

Any quest for stable method in understanding how knowledge is generated by human beings using machines founders on the initial fallacy that there are immaculately separate human and machinic orders, each with an ontological, epistemological, and pragmatic purity. . . . Digital humanities method . . . consists in repeatedly coadjusting human concepts and machine technologies until . . . the two stabilize each other in temporary postures of truth that neither by itself could sustain. (“What Is the Meaning,” 416)

Adjusting the dataset to get the best results is simply what data miners do to achieve “temporary postures of truth.” This is good practice, not a crime, and it is only problematic insofar as the data miners present their findings as “objective truth,” obscuring to us (and sometimes to themselves) their own role in massaging data and adjusting the parameters of their algorithms to make them work. It would be very easy to break Rybicki’s algorithm, to prevent “highly adventurous content” from being associated with “M” and sentimentality with “F,” simply by adding novels to the dataset.[19] Data amassed, which is to say data, is always capta: counted or captured by someone.

Gender/Genre: What Counts?

To arrive at a working method for assessing the genders of the anonymous authors in his dataset, Rybicki must omit certain other authors from his collection of writers with known genders: this shows us how genre, as much as gender, muddies the picture. Genre affects the algorithm’s ability to distinguish M/F in three ways. First, “feminine” sentimental literature and “masculine” criticism incorporate stylistic features that are discernible and imitable. Rybicki had to eliminate William Godwin, who wrote in the genre of Jacobin novels that are typically sentimental in style. His results also show Samuel Richardson’s epistolary novels, written largely in the voices of his eponymous heroines, veering toward “F.” I asked Jan Rybicki to use his algorithms to perform a stylometric analysis of Mary Wollstonecraft’s two anti-Jacobin, sentimentalist novels along with two essays, her two Vindications, in relation to texts by two men. Both of Wollstonecraft’s essays are, to my sensibility, very Johnsonian in style, characterized by Latinate diction and complicated syntax. I asked him to test at least in one case the effect of genre on gender differentiation, and he produced the result shown in Figure 1.1.

Wollstonecraft’s sentimental anti-Jacobin novels most resemble Godwin’s sentimental anti-Jacobin novels, as indicated by proximity in this bootstrap consensus tree, whereas her essays most resemble Johnson’s writings. In fact, Wollstonecraft chose to write both her Vindications in the Latinate, Johnsonian style that dominated the essay genre because it had, at her moment, considerable cultural authority. The literary critic Nancy Miller points out that two other eighteenth-century novelists, Frances Burney and Françoise d’Issembourg d’Happoncourt Graffigny, assume a similar style when, in prefaces to their novels, they take on the style of a critic (51–52). Insofar as one pins a genre to a gender, then a woman writing in a Latinate style (essays, criticism) or a man writing a Jacobin sentimental novel is writing in drag.

The case of genre helps us understand more about gender analysis; that is, about figuring out “what counts as ‘woman’” (Haraway, “Gender,” 67). It shows us that features of both gender and genre, while highly discernible, are also highly imitable. Writing in the sentimentalist genre can be accomplished through study. For instance, Rybicki notices one parody of “female writing” by William Beckford (7). Despite his awareness of the possibility of parody, Rybicki argues that one of the anonymous Chawton House novels is “correctly” identified as written by a man because its preface deprecates “‘womanish and weak minds’” (13, emphasis in the original). It could be a parody, but it also could be written by a woman pretending to write as a man. In fact, Wollstonecraft originally published her Vindication of the Rights of Men anonymously. In it, she disparages Edmund Burke, whose Reflections on the Revolution in France she is attacking, as “effeminate.” That kind of slur certainly helped disguise her gender and with it her identity. Deprecating femininity is a gender marker that women find easy to imitate in order to establish their authority. (We can see from Wollstonecraft’s second Vindication that she did honestly devalue “effeminacy,” which she analyzed as learned rather than innate, pointing out that the soldiers of her time and class behaved with as much effeminacy as women.) Anyone can adopt gendered modes of behavior, just as anyone can write in genres stereotypically labeled M/F.

In writing, if we define “gender” as a category-in-the-making (Haraway, Modest_Witness, 35), as a set of conventions for self-representation that are negotiated and manipulated, more or less consciously, then gender approaches nearly to the definition of genre. (If we take out “self” from the preceding sentence’s proposed definition of gender, the result, “a set of conventions for representation,” is a definition of genre.) Learning to write in a specific genre is almost as difficult as learning to encode one’s self-expression according to a gender—and in some instances, the tasks largely overlap.[20] In person, one sends gender signals with body movements and gestures, speech, and writing. Whereas in the case of gender, we think of imitating its forms as “passing” or “drag,” in the case of genre, unless it is a case of parody, we think of it as, well, writing. In composing her two Vindications, Wollstonecraft is not imitating an essay writer; she is writing essays. When Rybicki says in his essay that Austen “writes more like a man” (9), maybe that too is only evidence of Austen writing.

Am I arguing, then, that M/F is a bogus binary, produced by data collectors and data miners, without corresponding to any distinction actually found in reality? There are two answers to this question. First, yes: I believe the gender binary is not really about gender. But if it is wrong, how is Jockers’s stylometric analysis able to determine a novel’s gender correctly in 80 percent of the cases, while identifying genre correctly in only 67 percent? That fact justifies Jockers’s statement that “gender . . . is a strong signal” (Macroanalysis, 93). However, if we look at the numbers more closely, what Jockers is calling “the genre signal” is weaker than what he is calling “the gender signal” because his algorithm is trying to sort texts into eleven genre categories, whereas there are only two gender categories: M/F. His algorithm is actually eight times better than chance at determining genre, but only 1.6 times better when determining gender.

I would rather roll dice that are 1.6 times more likely to give me a winning number.[21] But I would maintain that the algorithm sorting texts into M/F is not sorting only gender at one moment, and only genre at another, when it is required to sort in two categories as opposed to eleven. The gender/genre signals, as Rybicki’s work subsequently shows, are inextricably intertwined. In other words, when stylometric analysis guesses 1.6 times better, it is guessing about what I would call “textual gender,” the stylistic and textual features associated with gendered genres. It is by now a critical commonplace that gender construction is discursive: it takes place in writing, in everything from medical documents and legal cases to novels. And many, many books and articles examine the relationship between gender and genre. Here, however, I am proposing that, when gender analysis operationalizes data-mining algorithms, separating gender from other markings (genre, era of composition) is not possible: historical time and genre are not incidental to, but constitutive of, gender.

We could break the algorithm’s capacity to produce “a strong gender signal” by simply increasing the number of gender categories to be sorted. Experts in the field could create metadata to generate a completely new taxonomy to replace the tired M/F binary: “men writing as men,” “women writing as women, “women writing as men,” “men writing as women,” “unspecified (anonymous) writing as men,” “unspecified writing as women,” “men writing as men (byline) in the voice of a woman (woman narrator),” “men writing as unspecified (anonymous byline) in the voice of a woman,” “women writing as men (byline) in a voice of unspecified,” etc.—whatever categories are presented by title pages, prefaces, narrators’ discourses, and research into authorship attribution found.[22]

Breaking down the strength of a signal is not the same as disrupting computational capacity, which, as Ted Underwood argues, no longer depends on simple classification schemes:

Maybe in 1967 it was true that computers could only handle exclusive binary categories. I don’t know: I was divided into a pair of gametes at the time myself.[23] But nowadays data models can be as complex as you like. Go ahead, add another column to the database. No technical limit will stop you. Make categories perspectival, by linking each tag to a specific observer. Allow contradictory tags. If you’re still worried that things are too simple, make each variable a probability distribution. The computer won’t break a sweat. (“The Real Problem with Distant Reading”)

Computation enables complexity.

Additionally, the computer screen affords interactivity. Text-mining parameters can be manipulated by viewers, as they are in Voyant windows.[24] The authors of Voyant, Geoffrey Rockwell and Stéfan Sinclair, have also created Methodi.ca,[25] through which algorithms can be altered. The new “Data” section of CA: Journal of Cultural Analytics allows not only for sharing data but also for articulating the principles of data collection.[26] Visualization tools are being built that will allow running and rerunning data, each time changing the underlying parameters used for text mining in order to test their effects, and that also make it easy to drill down to the text itself for close reading when questions arise.[27] Digitization forces on us neither rigid quantification systems nor distant reading. Far from it: digital media can foster the mixing of qualitative and quantitative methods and represent the results of analysis from multiple perspectives.

The second answer to the question posed earlier is no: M/F is not a bogus binary. Eighty percent is a very good rate of correctness in data mining. But exactly what has been correctly measured? Feminist literary historians have known for a long time that the early novel was indeed a place where gender norms were formed (Miller, 47; Gelfand and Switten, 443). Furthermore, this construction of gender took place at the very same moment, and using the very same literary devices, as the moment and methods through which novelistic genres were formed. As seen in Harrington’s analysis of speech patterns described earlier, people partly create and partly confirm stereotypes. Harrington’s exaggerators seized on existing classifiers and amplified them. Thus, the M/F classification process provides a feedback loop: people wishing to exaggerate their gender will repeatedly send signals that are then further confirmed after the fact.

As novelists attempted to create gender norms, those norms became more imitable, more easily parodied,[28] and therefore more easily adopted throughout the culture. Novelists were counting and reproducing as many gender and genre features as they could. Underwood and Bamman’s study of gendered norms in novels written between 1780 and 1989 finds a general trend: over time “gender [becomes] harder and harder to predict.” After a certain historical moment, perhaps after the Victorian era (Jockers and Kirilloff), novels turned to tasks other than stereotyping the M/F binary. We could use stylometry to find this out. But stylometry will only become a valuable tool of literary criticism if we admit that categories such as gender are being constructed both by the measurer and the measured. Once this process is accepted, we might then be able to use stylometry to experiment with new taxonomies of gender, such as the one adumbrated earlier. If trained on taxonomies generated by experts in feminist literary history, who have familiarity with the time period, the guesses that an algorithm might make about gender, although a weaker signal because sorting into a greater number of categories, would optimize human-computer interactions. Algorithmic sorting would sometimes reveal our blindnesses, surprising us with insight, and sometimes reveal its own stupidity, marking those moments when human thinking surpasses computational logic. It would never simply be “right” or “wrong.”

In Modest_Witness, Haraway protests against what she calls “virile witness,” testimony as an act of opposition. The modest witness, by contrast, though no less biased as a human being, attempts to identify the terms of self-representation without forcing them into the form of aggressive opposition.[29] Rewriting gender as sex, wrongly understood as a simple binary opposition, transforms it into a weak signal that might be statistically “correct.” Yet, with categories multiplied, we would be able to see what gendered characteristics were most salient to the people writing during a specific time period, finding their gender cues rather than always our own. The M/F opposition is implicitly and always aggressive in the sense of promoting oneself at the expense of others, sometimes others who lived in the past.[30] Computer screens as opposed to stereotyped pages easily afford the fluid exploration of parameters and taxonomies, through which many sorts of experiments can be tested: interactive visualizations can give us not objective answers rooted in aggressively reductive oppositions, but parallax, multiple perspectives for viewing a very complex reality. In analyzing gender, the computational analysis of culture should never be understood as an objective observer measuring an inert reality. Rather, it should be understood as a means of counting people counting. Gender analysis thus provides an important test case for cultural analytics, allowing the field to show how self-critical and historically reflexive it can be. But it does more.

Transforming the print humanities into the digital humanities offers us an opportunity.[31] It is a very short step from “typically” to “stereotypically,” as short as it is from printed to stereotyped pages, which are notoriously difficult to tell apart (Shillingsburg). Thawing typographical fixity via computer screens, moving from private marginalia to public, scholarly, dynamic interaction, offers us the chance to contest discursive constructions like the sex/gender/sexuality system at every moment of their construction.

Notes

Bibliography