PART III ][ Chapter 20

Videographic Criticism as a Digital Humanities Method

Jason Mittell

When faced with the task of defining digital humanities to an outsider, I like to chart out three different but potentially overlapping modes of scholarship: humanistic study of digital cultural materials and practices, humanities scholarship that uses digital platforms for dissemination and presentation, and computational methods applied to humanistic topics. Such distinctions resonate fairly well within most humanistic fields, where analyzing digital culture is typically an innovative direction, and the distinction between digital dissemination and methodology makes sense. Within my own field of film and media studies, however, things are a bit more muddy.

The humanistic study of digital culture pervades the field of film and media studies, especially now that most media objects are either born in digital form or reborn, as with analog films being redistributed as DVDs or online streaming. Additionally, the vast majority of contemporary media are primarily produced through digital means, as analog recording technologies have withered away. While most film and media historians would not call themselves “digital scholars,” we cannot ignore how the digitization of moving images has transformed our field. Digital dissemination of film and media scholarship has certainly been significant, especially through blogs and other forms of writing that move beyond typical formal academic formats. Innovative venues for multimedia publishing, such as the journal Vectors and the platform Scalar, allow text, moving images, and sound to coexist in the same digital spaces, thus helping media scholars situate our critical writing on the same plane as our objects of analysis.

Film and media scholars have been slower to explore the kinds of computational techniques, such as data mining and network visualization, that have flourished in the fields of literary studies and history. In part, this stems from the challenges of transforming moving images and sounds into data that can be treated algorithmically: it is comparatively easier, both in terms of user expertise and computing power, to quantify the written word using accessible tools and scripts than it is to quantify the multiple variables of complex images and sounds that change over time. Additionally, media studies has a long history of differentiating itself from its quantitative cousin discipline of mass communication, so embracing computational methods might seem like a retreat from its qualitative and interpretive distinctiveness. Although still on the margins of the field, computationally based film and media scholarship has worked to extend quantitative research both in forms that predate digital humanities, such as the Cinemetrics project, which quantifies film editing patterns as inspired by Barry Salt’s statistical stylistics work starting in the 1970s, and in newly emerging areas, as with Kevin Ferguson’s work on volumetric cinema discussed in Chapter 28 in this volume (see also Heras).

Parallel to this work, film and media scholars have also developed a mode of digital scholarship that at first blush might not seem to belong to the digital humanities per se: videographic criticism. At its most straightforward, videographic criticism is the expression of scholarly ideas via moving images and sound in audiovisual form. While such work derives from other forms of media production, such as documentaries about film and media, experimental essay films, and conceptual art pieces reusing film footage, the past decade has seen a set of norms and practices coalesce around what are often called “video essays.” Unlike the more varied multimodal designs featured on Vectors, video essays typically resemble the more familiar format of a linear online video; such works are typically shorter than thirty minutes and incorporate sound and images from the films and media being analyzed, often using a voice-over narration to deliver an argument. While most video essays are focused on discussing formal features and interpretations of films, they can explore a wide range of topics, theoretical approaches, or objects of study. Some scholars have pushed back against video essays that are overtly explanatory, which at their least innovative resemble illustrated lectures, by suggesting that videographic works should embrace the poetic and aesthetic possibilities of film and media, and be more creative and expressive in their rhetoric. As Christian Keathley argues, videographic criticism needs to straddle the explanatory and poetic registers, aiming to produce a “knowledge effect” composed of sound and image, rather than just traditional argumentation best suited to the written word (see also Grant; López and Martin). It was in this spirit of exploration and experimentation that Keathley, Catherine Grant, Drew Morton, Christine Becker, and I cofounded [in]Transition as a journal of videographic criticism in 2014, publishing scholarly videos with open peer review via the MediaCommons scholarly network.[1] [in]Transition seeks to publish a broad range of styles and approaches to videographic criticism, while using its open peer review to discuss how scholarship in video form might fit within and advance scholarly discourse.

How might we situate videographic criticism in relation to digital humanities? While those of us who have helped develop and promulgate this mode of scholarship in the 2010s did not do so within the typical contexts of DH centers, labs, programs, or interdisciplinary clusters, there is clearly productive overlap. Videographic criticism has received both direct and indirect support from DH funders: MediaCommons was among the first recipients of National Endowment for the Humanities DH funding, and the NEH recently funded three Institutes for Advanced Topics in the Digital Humanities on producing videographic criticism that Keathley and I first convened in 2015 (see Keathley and Mittell). But few videographic creators have situated their work as part of the broader field of digital humanities, and videographic work is not positioned as an approach typically found within DH practice.[2] I contend that videographic scholarship is a distinct DH method, one that has the potential to transform how we engage with, analyze, and convey ideas about moving image texts; such a method can both make digital humanities more open to film and media scholars and offer a new mode of digital scholarship to all scholars interested in moving images, regardless of their disciplinary home.

Returning to my initial tripartite classification of digital humanities, it seems clear that videographic criticism is an example of the first two modes: most videographic works examine cultural objects that were either born or reborn digital, such as films and television programs; and they depend on digital dissemination via online video sites, with the transformative possibilities inherent in new forms of peer review and open access. But is videographic criticism a computational method? I contend that it is, although we need to regard “method” in a broader context than in which the term is typically used.[3] For many DH approaches, computational methods take cultural works that are typically researched via interpretation, contextualization, and other forms of close qualitative analysis and transform them into quantifiable datasets that can then be analyzed algorithmically, classified and sorted, and rendered as visualizations, maps, or other new ways of looking at cultural works. Videographic criticism does include such a transformation of a cultural work into data, but few would regard this process as a method analogous to text mining or digital mapping: ripping a DVD and importing it into a video-editing application like Adobe Premiere seems on its face no more of a methodological transformation than scanning a book into a machine-readable PDF.

And yet, there is a key conceptual transformation that occurs when shifting a read-only video into an editing platform. In its DVD or streaming form, a film is a bound object that asserts its own coherence; we treat it as something with its own internal design, and traditional criticism is supposed to engage with the work on those terms through analysis and interpretation. But once a film is imported into video-editing software, it becomes something else: an archive of sounds and moving images. Importing a film into editing software enables a conceptual transformation that allows critics to see and hear it differently. Just as DH methods like text mining, distant reading, and cultural analytics allow scholars to see patterns across works, videographic criticism enables us to break the seal that binds a film as a finished work and then engage with its component parts. Even if you do not make something new with the sounds and images imported into the editing platform, you can still discover something new by exploring a film via this new interface.

A good example of such transformative insight comes from one of the most prolific and accomplished videographic critics, Kevin B. Lee. In his video essay “Viewing between the Lines: Hong Sang-soo’s The Day He Arrives,” Lee strives to make sense of a Korean film that seems to purposely present an incoherent and inconsistent narrative world. He uses his editing platform to break the film down into its component sixty-five shots and to organize them visually to highlight various locations, repetitions, and patterns. In addition to presenting his critical observations derived from that computationally enabled analysis, he includes screen-captured footage of his Final Cut Pro interface in his essay to reveal the methods he used to understand the film (see Figure 20.1).

Likewise, in his piece analyzing the 2014 race for the Best Actress Oscar, Lee uses an editing timeline to visualize Sandra Bullock’s unconventional performance in Gravity, highlighting how much of her performance consists of her body and breath rather than voice (see Figure 20.2).

Lee did not produce these videos within the methodological terrain of digital humanities per se, but he is engaging with recent developments in technologically enabled visualization and quantitative methods.[4] These videos clearly employ a software platform to computationally transform a cultural object and reveal its elements that would be hard to discern with an analog eye. Is there a better description of what DH methods do?[5]

Another example of discovery via the computational transformation of film into a digital video-editing platform comes from a newer videographic critic. As part of our 2015 summer NEH workshop, film scholar Liz Greene was working on a video about sound in David Lynch’s films, a topic she has researched and written about for years (see “Bringing Video into the Digital Domain”). While manipulating footage in Premiere, she discovered uncanny similarities between the timing of sounds and images in the opening sequences of The Elephant Man and Blue Velvet. The result was “Velvet Elephant,” a juxtaposition of these two sequences that experientially conveys her discovery and enables viewers to watch and hear the parallels between the otherwise quite different films. I would not call the resulting video “scholarship” per se, as it lacks analysis, argumentation, or context; however, it is most certainly “research,” sharing a discovery that could only be realized through Greene’s computational transformation of the two films.

An insight gained from such transformations is, in itself, research. Too often, the humanities frames “research” as the finished products of scholarship: the book, the essay, the lecture. But research and its associated methodologies are not limited to the final product of scholarship: rather, the processes of discovery and experimentation are often the more exciting and insightful parts of scholarly endeavors, and it is in such processes where methodologies are used and developed. In the sciences, exploratory research consists of the processes that are undertaken in the lab, using methods and techniques to discover new things and test ideas; in the humanities, that research is how we look at, read, connect, and think through the various cultural objects and practices that interest us. For instance, Stephen Ramsay suggests that experimentation with a computational tool applying graph theory to Shakespeare’s scene structure was not attempting to prove anything in particular, but “to say something new, provocative, noteworthy, challenging, inspiring—to put these texts back into play as artifacts reconstituted by re-reading” (“In Praise of Pattern,” 189). For videographic criticism, similar research happens within the editing interface, where we learn how to see a film in a new, unanticipated way. Such a computational transformation highlights that digital humanities need not be quantitative in its methods: while any software has a quantitative core, working with a video editor allows one to computationally explore the cultural realm of images and sounds, not just shot lengths, proportions, and color temperatures.

This transformation of finished media objects into a manipulable library of sounds and images is the first way that videographic criticism constitutes a methodological shift. Any strong work of videographic criticism builds on something learned through the editing process itself, not merely expressing ideas that were fully formed before engaging with the media in its transformed state. In the video editor, decoupling audio and video, clipping footage into patterns that differ from the original’s edit, creating new juxtapositions between shots and sounds, and resequencing footage in contradiction to the narrative design all allow us to observe and discover something new about the original film. We could stop there, writing up these computationally enabled discoveries via more traditional analog means of the written word.[6] But once a film is imported into an editing platform, the possibilities of “writing” with sounds and images allows us to express this research via a new rhetoric that both draws on the affective and aesthetic experiences of viewing films and, like writing about literature, expresses critical ideas in the same medium of the object of analysis itself. Videographic critics like Lee and Greene aim to assemble the insights gained by computationally transforming a film into an archive of sounds and images, into something new: an expression of critical ideas via the tools of cultural production. Thus the second methodological shift in videographic criticism concerns the expressive modes both enabled and required in videographic work.

Images are typically a less denotative and more expressive communication medium than the written word, especially within the norms of academic writing. Creating scholarship with moving images and sounds forces critics to engage with the aesthetic and affective dimensions of their expression far more than do typical written works; they must draw on artistic practice as well as scholarly analysis. Additionally, expressing ideas with video enables more possibilities to use irony, ambiguity, sensation, and other poetic elements than does academic prose. While scholars Robert Ray and Greg Ulmer have urged a more poetic approach to academic writing for decades, their calls have largely been unheeded within film and media studies, perhaps because the norms of academic writing are too sedimented to broadly embrace such a radical approach to expression and associative analysis. However, videographic work foregrounds the poetics of the source material, embraces the aesthetics of images and sounds as forms of meaning-making, and takes advantage of the newness of the format to allow for more creative possibilities without being beholden to entrenched norms of typical academic rhetoric.

Is it fair to call such a mode of critical rhetoric a “methodology”? Perhaps not in isolation, but such rhetorical expansiveness does allow scholars to express different types of ideas and make new forms of arguments that recursively use media to critique themselves; such rhetoric thus has a methodological effect in enabling new ideas to develop. For instance, Michael Talbott’s video “Encounters” narrates an unusual trip to Argentina over a compilation of clips from Argentine cinema; via this juxtaposition, he conveys ideas about the role of cinema as a form of virtual travel and what it means to “have been” to a place. Such ideas could be argued via academic writing, but they could not be expressed and experienced in the same way; in writing, they would lose the affective tone that is distinct to his videographic expression and conveys the feelings within Talbott’s analysis rather than just the concepts. My own video, “Adaptation.’s Anomalies,” begins at the limits of writing, taking up two aspects of Adaptation. that I could not find the words to analyze effectively in my recent book about the film, Narrative Theory and Adaptation. The direction in which my videographic analysis moves is far from the typical academic account, but it embraces a more ambiguous and playful tone that draws on the film’s own discursive strategies, especially in its final section; my video essay reflects on its own critical practices, leaving viewers uncertain as to what analysis is intended to be “real” versus satirical, much like Adaptation.’s reflexivity about its own filmmaking process and ambiguously pitched final act. Videographic criticism can push such unconventional expressive tones to greater extremes than writing, especially when using the raw materials of a media text that already employs such reflexive themes. Thus, in videos like mine, Talbott’s, and many others, the combination of ideas and tone simply could not be conveyed successfully using the written word, with videographic poetics adding new methodological possibilities to film and media scholarship. Additionally, my own process of generating ideas about Adaptation. would not have been feasible without the experience of analyzing its footage in the context of Adobe Premiere, because I would not have noticed some of the juxtapositions and connections had I just been watching and writing about the film through conventional means.

The possibilities of videographic method can be expanded even further through the combination of these two shifts, merging the computational possibilities of video-editing software with the expressive poetics of sounds and images. The former draws from scientifically derived practices of abstraction that are common to digital humanities: taking coherent cultural objects, such as novels or paintings, and transforming them into quantified datasets or graphs. The latter draws from artistic practices of manipulation and collage: taking coherent cultural objects and transforming them into raw materials to create something more unusual, unexpected, and strange. Videographic criticism can loop the extremes of this spectrum between scientific quantification and artistic poeticization together, creating works that transform films and media into new objects that are both data-driven abstractions and aesthetically expressive. I next outline three such possibilities that I have developed, using case studies of films that I know well and have used in the classroom.

The model of poeticized quantification that I am proposing resembles the vector of literary analysis that Lisa Samuels and Jerome McGann call “deformative criticism.” Such an approach strives to make the original work strange in some unexpected way, deforming it unconventionally to reveal aspects that are conventionally obscured in its normal version and discovering something new from it. Both Stephen Ramsay, in Reading Machines, and Mark Sample extend Samuels and McGann’s model of deformances into the computational realm, considering how algorithms and digital transformations might create both new readings of old cultural objects and new cultural objects out of old materials. This seems like an apt description of what videographic criticism can do: creating new cultural works composed from moving images and sound that reflect on their original source materials. While many types of video essays might be viewed as deformances, I want to explore a strain of videographic practice that emphasizes the algorithmic elements of such deformative work. One hopes that such work can also be inspirational to digital humanists working outside of videographic criticism as well, suggesting how algorithmic deformations, as well as any digitally constituted scholarship, can intertwine the computational and the poetic in unexpected ways.

One way to deform a film algorithmically is through a technique borrowed from conceptual art: the imposition of arbitrary parameters. From Oulipo, the collective of French artists who pioneered “constrained writing,” to proto-videographic artworks like Douglas Gordon’s 24 Hour Psycho or Christian Marclay’s The Clock, to obsessive online novelties of alphabetized remixes of films like ARST ARSW (Star Wars) and Of Oz the Wizard (The Wizard of Oz), artists have used rules and parameters to unleash creativity and generate works that emerge less from aesthetic intent than from unexpected generative outcomes.[7] For example, 24 Hour Psycho (1993) takes Hitchcock’s original film and slows it to the titular twenty-four hours, showing each frame for approximately a half second. The experience deforms the original’s storytelling, pacing, and suspense, but allows viewers to notice elements of shot composition, gesture, and production design, all by permitting us to see what the film always was: a series of still images with only slight differences. Scholars can adopt such an unorthodox approach as well, allowing ourselves to be surprised by what emerges when we process datasets of sounds and images using seemingly arbitrary parameters. One such approach is a concept that Christian Keathley and I devised as part of our NEH workshop: a videographic PechaKucha. This format was inspired by oral PechaKuchas, a form of “lightning talk” consisting of exactly twenty slides lasting exactly twenty seconds, resulting in a strictly parametered presentation. Such parameters force decisions that override critical or creative intent and offer helpful constraints on our worst instincts toward digression or lack of concision.

A videographic PechaKucha adopts the strict timing from its presentational cousin, while focusing its energies on transforming its source material. It consists of precisely ten video clips from the original source, each lasting precisely six seconds, overlaid on a one-minute segment of audio from the original source: the video stands alone without a presenter. There are no mandates for content, for ideas, for analysis—it is only a recipe to transform a film into a one-minute video derivation or deformance. In creating videographic PechaKuchas with our workshop participants and our undergraduate students, we have found that the resulting videos are all quite different in approach and style, despite their uniformly parametered length and rhythm. For instance, Tracy Cox-Stanton deforms the film Belle du Jour into a succession of shots of the main character Séverine vacantly drifting through rooms and her environment, an element of the film that is far from central to the original’s plot and themes, but is clearly embedded in the visual presentation of the character.[8] Or Corey Creekmur compiles images of doors being open and shut in The Magnificent Ambersons to highlight both a visual and thematic motif from the film.[9] In such instances, the highly parametric exercise enables the critic to discover and express something about each film through manipulation and juxtaposition that would be hard to discern via conventional viewing and even harder to convey evocatively via writing. Keathley and I have both used this PechaKucha technique in the classroom, as well as a departure point for our own analytic projects; as a mode of deformative research, it helps both novice students and experienced critics see a film in a radically new light, viewing the sounds and images as source material to be played with, rather than as locked-in components of a finished work.

Such experiments with videographic PechaKuchas follow arbitrary parameters to force a type of creativity and discovery that belies typical academic intent, but they are still motivated by the critic’s insights into the film and aim to express something. A more radically arbitrary deformance removes intent altogether, allowing the parameters to work on the film and removing the critic’s agency. I devised the concept for a videographic PechaKucha randomizer, which would randomly select the ten video clips and assemble them on top of a random minute of audio from the same source film; Mark Sample and Daniel Houghton executed my concept by creating a Python script to generate random PechaKuchas from any source video.[10] The resulting videos feel like the intentionally designed PechaKucha videos that I and others have made, sharing their uniform length and rhythm, but the content is truly arbitrary and random: they include repeated clips, idiosyncratic moments from closing credits, undefined sound effects, and oddly timed clips that include edits from the original film. And yet because they follow the same parameters as intentionally designed PechaKuchas, they are arguably just as much of a distillation of the original film and as such have the possibility to teach us something about the source text or create affective engagement with the deformed derivation.

Just as the algorithmic Twitter bots created by Mark Sample or Darius Kazemi produce a fairly low signal-to-noise ratio, most randomly generated PechaKuchas are less than compelling as stand-alone media objects; however, they can be interesting and instructive paratexts, highlighting elements from the original film or evoking particular resonances via juxtaposition, and prompting unexpectedly provocative misreadings or anomalies.[11] For instance, in a generated PechaKucha from Star Wars: A New Hope, Obi-Wan Kenobi’s voice touts the accuracy of Stormtroopers as the video shows a clip of them missing their target in a blaster fight, randomly resonating with a popular fan commentary on the film.[12] Another randomly generated PechaKucha of Mulholland Drive distills the film down to the love story between Betty and Rita, highlighting the key audio moment of Betty confessing her love, with most clips drawn from scenes between the two characters; the resulting video feels like a (sloppy but dedicated) fannish remix celebrating their relationship. A generated PechaKucha of All the President’s Men is anchored by one of the film’s most iconic lines, while the unrelated images focus our attention on patterns of shot composition and framing, freed by our inattention to narrative. There are nearly infinite possibilities of how algorithmic videos like these might create new deformations that could help teach us something new about the original film or constitute a compelling videographic object on its own merits; such randomly generated videos support Samuels and McGann’s insight that reorganizing a text can reveal something buried within it that would be invisible in its conventional form. Each of these acts of deformative videographic criticism takes approximately two minutes to randomly create itself, generating endless unforeseen critical possibilities.

Videographic PechaKuchas take inspiration from another medium, the oral presentation, but we can also translate modes of film and media scholarship into deformative videographic forms. One of the most interesting examples of parameter-driven deformative criticism is Nicholas Rombes’s “10/40/70” project. In a series of blog posts and a corresponding book, Rombes created screen captures of frames from precisely the 10-, 40-, and 70-minute marks in a film and then wrote an analysis of the film inspired by these three still images. Rombes acknowledged that he was deforming the film by transmuting it into still images, thus disregarding both motion and sound, but he aimed to draw out the historical connections between filmmaking and still photography through this medium shift. The choice of the three time markers was mostly arbitrary, although they roughly mapped onto the beginning, middle, and end of a film. The result was that he could discover aspects of the film that were otherwise obscured by narrative, motion, sound, and the thousands of other still images that surrounded the three he isolated: a clear example of a deformance in Samuels and McGann’s formulation.

What might a videographic 10/40/70 look like? It is technically simple to patch together clips from each of the designated minute markers to create a moving image and sound version of Rombes’s experiment. Although we could use a range of options for the length of each clip, after some experimentation I decided to mimic Rombes’s focus on individual frames by isolating the original shots that include his marked frames, leading to videos with exactly three shots, but with far more variability in length, rhythm, and scope. As with Rombes’s experiment, the arbitrary timing leads to highly idiosyncratic results for any given film. Raiders of the Lost Ark yields a trio of shots without obvious narrative or thematic connection, but in isolation, we can recognize the cinematographic palette that Steven Spielberg uses to create action melodrama: camera movement to capture moments of stillness with an emphasis on off-screen or deep space, contrasted with facial close-ups to highlight character reactions and emotion. Isolating frames in Star Wars: A New Hope also calls attention to movement, with consistent left-to-right staging: first with the droids moving across the desert, then with Luke running to his landspeeder, then with Obi-Wan’s head turning dramatically, which is continuous with the rightward wipe edit that closes out the second shot. Both of these iconic films are driven by plot and action, but these arbitrary shots belie coherent narrative, allowing us to focus more on issues of visual style, composition, and texture.

Depending on the resulting shots, narrative can certainly emerge within these deformations. In Fargo, we start with a shot of used-car salesman Jerry sputtering about negotiating a car sale in the face of an irate customer, which abruptly cuts to Jerry sputtering about negotiating with kidnappers to his father-in-law, Wade, and colleague Stan in a diner, highlighting the consistent sputtering essence of Jerry’s character, underscored by his nearly identical wardrobe across different days in the original film. The middle scene plays out in an unbroken eighty-second static shot, pulling us away from the deformity and placing us back into the original film, as the coherent narrative eclipses the incoherence of the 10/40/70 exercise. But knowing that we are watching a deformation, we wait for the unexpected cut to jump us forward in time, splitting our attention between the film and its anticipated manipulation. The narrative action guides the transition, as Wade impatiently refuses to abide by Jerry’s plan to deliver the ransom and stalks away saying “Dammit!” The resulting arbitrary edit follows the most basic element of narrative, cause and effect: we cut to Wade being shot by one of the kidnappers, punctuated by a musical sting and evoking Stan’s earlier line that they will need “to bite the bullet.” The final jarring effect stems from the final shot being less than three seconds long, a startling contrast to the previous long take, and underscores the contrast between the incongruities of mundanity and brutality, boring stasis and vicious action that are the hallmark of Fargo and much of the work of the Coen brothers. Although it certainly feels like an unusual video, Fargo 10/40/70 also functions as a cultural object in its own right, creating emotional responses and aesthetic engagement in a manner that points to one of the strengths of videographic work: the power of sound and moving images works to captivate viewers, even when conventions of storytelling and structure are arbitrarily deformed.

The distinction between print-based and videographic deformations becomes clear by comparing Rombes’s written results working with stills to a videographic version of the same film. Rombes analyzes three stills from Mildred Pierce, and they point him toward elements of the film that are frequently discussed in any analysis: the contradictions and complexities of Mildred’s character, how she fits into the era’s gender norms, and the blurs between film noir and melodrama.[13] The images launch his analysis, but they do not direct it to unexpected places. The videographic version is more provocative, offering more opportunities for (productive) misunderstanding and incoherence. The first shot finds Mildred’s old associate Wally panicking and discovering her husband Monty’s dead body in a noirish moment of male murder and mayhem, but quickly gives way to a scene of female melodrama between mother Mildred and daughter Veda. Mildred’s first line, “I’m sorry I did that,” suggests a causal link that she is apologizing for murdering Monty. Knowledge of the film makes this implied causality much more complex, as the murder is a future event that sets the stage for the rest of the film being told in flashback; in the frame story, Mildred appears to have murdered Monty, with the flashback slowly revealing the real killer to be Veda. Thus this scene works as a decontextualized confession made to the actual (future) murderer, adding temporal resonance and highlighting how the entire flashback and murder plotline was a genre-spinning element added to the screenplay but not present in the original novel. The third scene picks up the discussion of the restaurant and finances, bringing it back to the conflict between Wally and Monty; if we were to temporally rearrange the shots to correspond to the story chronology, the opening shot of Wally finding Monty’s body would seem to resolve this conflict, creating a closed loop of causality for this deformed version of the film. This brief analysis is no more valid or compelling than Rombes’s discussion, but it is certainly less conventional, triggered by the narrative and affective dimensions cued by the videographic deformation that ultimately seems more suggestive and provocative than the three still images.

The model of the videographic 10/40/70 relies on the single shot as the core unit of a film, following a tendency common to much academic work on moving image media. My third and final type of videographic deformation also highlights the shot, but using a distinctly different approach. As mentioned earlier, one of the most prominent forms of quantitative and computational analysis within film studies is statistical stylistics, especially as shared on the crowdsourced Cinemetrics website. Although there are numerous metrics on the site, the most common and well known is ASL, or average shot length, computed by dividing the time of a full film by its number of discrete shots. The resulting number indicates a film’s overall editing pace, charting a spectrum from quickly cut movies (such as Batman Begins at 2.37 seconds or Beverly Hills Chihuahua at 2.72) to longer-take films (such as An American in Paris at 21 seconds or Belle du Jour at 24).[14] Most films fall between three and eight seconds per shot, with much variability between historical eras, genres, national traditions, and specific filmmakers.

An ASL is in itself a kind of quantitative transformation, a reduction of a film to a single numeric representation. Cinemetrics does allow more detailed quantification and visualization of a film’s editing patterns; for instance, a more granular and graphic elaboration of Mulholland Drive’s ASL of 6.5 is shown in Figure 20.3.[15]

But these numbers, tables, and graphics make the film more distant and remote, leaving me uncertain about what we can learn from such quantification. According to Yuri Tsivian, Cinemetrics’s founder, the insights are quite limited: “ASL is useful if the only thing we need to know is how long this or that average shot is as compared to ASL figures obtained for other films, but it says nothing about each film’s internal dynamics.” Certainly, such comparison is the most useful feature of ASL, because it allows quantitative analysis among a large historical corpus, a general approach that has proven quite productive in digital humanities across a range of fields. But I question Tsivian’s quick dismissal that ASL “says nothing about each film’s internal dynamics.” Does a film with a 2.5-second cutting rate feel and function differently than one with a 15-second ASL? Certainly, and it does not take a quantification to notice those differences. But such a quantification might also point toward a more thorough understanding of editing rates through deformation.

Videographic methods, which use media texts to critique themselves in reflexive ways, allow us to impose a film’s ASL back onto itself. I have created a videographic experiment called an “equalized pulse”: instead of treating ASL as a calculated average abstracted from the film, I force a film to conform to its own average by speeding up or slowing down each shot to last precisely as long as its average shot length.[16] This process forces one filmic element that is highly variable within nearly every film, the length of each shot, to adhere to a constant duration that emerges quantitatively from the original film; yet it offsets this equalizing deformation with another one, making the speed of each shot, which is typically constant, highly variable. Thus in a film with an ASL of 4 seconds, the equalized pulse extends a 1-second shot to 25 percent speed, while an 8-second shot runs at 200 percent speed. If you equalized an entire film to its average pulse, it would have the same running time and the same number of shots, but each shot would be slowed down or sped up to conform to an identical length. Every shot exerts the same temporal weight, but each feels distinct in its tempo and pace. The result is, unsurprisingly, very strange—but productively so, both by revealing editing strategies in the original film and by creating new aesthetic objects out of the film’s raw materials.

What does Mulholland Drive look and feel like when equalized to a pulse of its 6.5-second ASL? Can we learn something more about the “film’s internal dynamics” than its numeric representations on Cinemetrics? Take the film’s opening scene following the credits, with Rita’s car accident on the titular street; in the original, it lasts 4:07 with 49 shots ranging in length between 0.3 and 27 seconds. The deformed version with an equalized pulse of every shot lasting precisely 6.5 seconds runs 5:18, because the original sequence is cut comparatively faster (ASL of 5.04 seconds) than the film as a whole. The effect is quite uncanny, with super slow-motion action sequences bookended by sped-up shots with less on-screen action; the car accident is particularly unsettling, turning a nine-shot, 6-second sequence into a grueling and abstract 58-second ordeal that oddly exaggerates the effect of experiencing a moment of trauma in slow motion. As a whole, the video does convey the sense that a pulse of 6.5 seconds feels quite deliberate and drawn out, although the variability of action obscures the consistency of the editing pulse.

Another scene from Mulholland Drive offers quite different effects, despite the same algorithmic deformation to conform to the same equalized pulse. The memorable scene in Winkies Diner, where two men discuss and confront a dream, is a pivotal moment in the film, signaling an affective impact that transcends any rational comprehension or interpretation. When equalized to a 6.5-second pulse, the scene’s uncanniness is ratcheted up, downplaying the dialogue’s rhythm that emphasizes Dan in the original, in favor of a more even distribution between the two men. The slow-motion close-ups with distorted voices highlight the film’s dreamlike quality, and its overall slower pace increases the sense of foreboding that already pervades the scene. By the time the horrific bum is revealed at the scene’s end, I find myself completely enthralled by the editing pulse and pulled into the affective horror that the scene always produces, suggesting that its impact is not dependent on Lynch’s designed editing rhythms. Extending this equalized pulse to the entire film, each scene and sequence feels quite different, even with a uniform shot length throughout.

Mulholland Drive is a film packed with abundant strangeness, even before its deformation, but equalized pulses make more conventional examples strange as well. Even though Mildred Pierce features the unusual combination of noir crime and family melodrama, it is still a far more straightforward film in keeping with its 1940s era. Its ASL of 10.09 is much slower than films of today, but is fairly typical of its time. Equalizing the pulse of a crucial scene in the family melodrama, with Veda driving a wedge between Mildred and Monty, who finally end their dysfunctional relationship, highlights various character interactions. When Mildred gives Veda a car, the deformation speeds through her thanking her mother, but lingers on her exchange with Monty, underscoring the closeness between stepfather and daughter; in the original, the emphasis is reversed in terms of timing, but equalizing the shots actually better represents Veda’s attitudes. The deformation lingers over shots without dialogue, letting us closely examine facial expressions and material objects, but speeds through lengthy dialogue shots, like an impatient viewer fast-forwarding through the mushy emotional scenes. The final lines exchanged between Mildred and Monty are unreasonably drawn out, milking their mutual contempt for all that it is worth. The scene is still legible, especially emotionally, but it redirects our attention in unpredictable ways: arguably a key goal of an effective deformance.

Equalizing the pulse of an action film on the other end of the pacing spectrum, such as Raiders of the Lost Ark, also points to unseen revelations. The film has an ASL of 4.4 seconds, longer than most contemporary action movies, but still quite brisk, especially for director Steven Spielberg. I deformed the iconic opening sequence, but used the sequence’s faster ASL of 3.66 rather than the whole film’s pacing, because that allows for a direct comparison of the original and equalized versions (see Figure 20.4). The effect is definitely striking, as the deformed version races through the buildup toward action and peril, while lingering painfully on darts flying through the air, near-miss leaps, and other moments of derring-do. In the slowed-down shots, one notices odd details that would be overlooked in the conventional film, like the discoloration of Indy’s teeth, and the sequence conveys a very different momentum. When placed side by side with the original, it highlights how much of the sequence is weighted toward the approach and buildup rather than the action, while the deformed version lingers on moments that regularly flit by.

The editing timeline visualizes these differences, but in a way that is analytically obscure; the videographic form allows us to feel and experience the analysis in ways that computational visualization cannot. What stands out most to me in this deformation is the role of music, as John Williams’s memorable score still manages to hit its key themes and punctuate the action, despite its variable tempo and rhythms.

This experiment in equalizing a film’s pulse points most interestingly toward the different types and functions of rhythm and tempo. In a conventionally edited film, variation of shot length is a main source of rhythmic play, both in creating emotional engagement and guiding our attention. Eliminating that variation by equalization creates other forms of rhythm and tempo, as we notice the relative screen time given to various characters, anticipate the upcoming edits in a steady pulse, and engage with the interplay between image and sound. These equalized deformations highlight how much the analysis of editing and ASL privileges the visual track over the audio: we are not quantifying audio edits or transitions in such metrics, as continuous sounds bridge across shots, slowing or speeding up like an accordion.

Equalized pulse experiments reveal and highlight how visual editing functions in conjunction with music, especially for instances where the musical track is more dominant, as with film musicals or music videos. Experimenting with musical sequences yielded the most exciting examples of equalized pulse, because they highlight the transformation of rhythm and tempo: the musical track stretches and squashes to create unpredictable rhythms and jettisons its standard tempo, allowing the steady beat of the changing visuals to define the speed. For instance, “Can’t Buy Me Love” from The Beatles film A Hard Day’s Night becomes a collage of fast and slow motion when equalized to its sequence ASL of 4.9 seconds, making an already playful and experimental sequence even more unpredictable. Musical sequences combined with dance add another layer of rhythmic play, as with the transformation of Singin’ in the Rain’s “Broadway Melody” into a deformed and almost uncanny work when equalized to its ASL of 14.9 seconds. Musical numbers are often edited at a slower pace than their films as a whole, providing more attention to performance and dance without being pulled away by edits. A rare exception is one of the fastest-cut films listed on the Cinemetrics site, and certainly the fastest-cut musical I know of: Moulin Rouge, with an ASL of 1.9 seconds. The “Roxanne” number, with an even brisker ASL of 1.05 seconds, is the only equalized pulse video I have yet made where the visual tempo becomes noticeably dominant, offering a steady beat of images and sounds whose fast-paced deformations go by so quickly as to often escape notice.

These equalized pulse versions of musical numbers are the most engaging and affective examples of videographic deformations I have made, functioning as compelling cultural objects both on their own and as provocatively deformative paratexts. They also demand further analysis and study, opening up a line of examination concerning the relative uses of edits, music, and dance to create rhythm and tempo. As such, these videographic deformations may not stand as scholarship on their own, but they do function as research, pointing the way to further scholarly explorations. Whether that subsequent scholarship is presented in written, videographic, or multimodal forms is still to be determined, but such work shows how videographic criticism is more than just a form of digital dissemination. Likewise, videographic deformations convey affective and aesthetic dimensions that textual deformances often lack: experiments with the written word can lead to great revelations, but I did not fully understand the uncanny transformative power of deformations until I experienced an equalized pulse video.

Transforming a bound cultural object like a film into a digital archive of sounds and images enables a mode of critical engagement that is impossible to achieve by other methods. Such experimental work is invaluable in the classroom as well; it is useful as a tool to help students notice aspects of visual media that are hard to discern through conventional viewing practices focused on narrative, spectacle, or emotional engagement. As such, videographic criticism can function as a DH research method—both in transforming media objects in unexpected and revealing ways and by enabling a mode of poetic expression to convey ideas ill suited for writing—that is poised to develop the field of film and media studies in unpredictable new ways. Additionally, digital humanists can learn from videographic criticism and deformances as reminders of the aesthetic power of the cultural works that we manipulate, quantify, and transform. A common (and certainly overblown) criticism of the digital humanities is that reducing literary works to data points both destroys aesthetic elements essential to literature and undermines the central role of reading; videographic criticism mines the aesthetic elements of moving images and sounds and redirects the power of viewing toward the act of critical engagement.

Notes

Bibliography