Of Coding and Quality

Let’s Code!

The question whether humanists should learn how to code has been debated for several decades (Ladurie; Hockey). Blog posts by humanities scholars describing their experiences teaching themselves programming, or web pages with helpful resources for the aspiring digital humanities (DH) programmer are plentiful (McDaniel; Sadler; Weingart; Afanador-Llach et al.). For the computational humanities, the question is more or less settled. If you are a computational humanist, you have to know how to code; you have to know how to “hack” and not just how to “yack” (Nowviskie). But what exactly does this mean? To answer that question, we will first define how we understand the different terms often used to describe the activity of writing code: coding, programming, and software engineering.

Generally, when speaking of coding, it refers to writing code—for example, writing a Python script that sorts and renames files. Often coding is associated with writing scripts or simple programs that have one specific goal (e.g., download a dataset from an API or running a specific analysis on a dataset). Programming goes beyond the simple activity of writing code; it encompasses planning, documentation, and an understanding of the bigger picture of the resulting program. Examples would be the generalization of a piece of code to be used by others or to be published as a package as well as the initial development of a web application. Software engineering, in contrast, has an even more holistic view of the process of creating software. It does not only focus on the implementation of a program but also on its design by taking into account issues such as hardware, user, and infrastructure requirements. For instance, getting a web application to run locally on a laptop requires in many cases “just” programming. Adding test cases, setting up a continuous integration and delivery pipeline, making sure it can be deployed to different servers by externalizing configuration, and preparing it to be translated into other languages would require software engineering. We believe that code quality is an important factor to consider no matter if a scholar is “just” coding or if a software engineer is developing an application to serve several research groups at a time. The methods and strategies might just have to be adjusted to fit a particular use case.

Lauren Klein raises this issue: “the real question, however, isn’t whether or not students should learn how to code; it’s what they should learn by doing so.” Learning to code can be a valuable experience for any digital humanist, even if they do not end up using it for their research. However, if the goal is to use coding for computational research, we believe one should aim to learn how to program. Knowing how to write a for-loop in Python is in most cases not enough to develop programs that can be trusted to produce accurate research results. Miriam Posner remarks that bootcamps are a good form of training in the digital humanities. However, she states that “a two-day long bootcamp, and even a week-long training opportunity, is not the ideal. The ideal is long-term collaboration with people you come to know and trust over the years.” While we agree with this statement, Posner might have a different group of people in mind. She suggests that bootcamps should be followed by regular meetings to create a community of like-minded people. While such a community can be very beneficial, it might not include the people who are on the more technical end of the spectrum of computational humanists; the ones that might not even call themselves computational humanists but software engineers who work on computational or digital humanities projects. Such people will likely not attend a bootcamp that teaches humanists how to code or how to develop a web application, but they are the ones who will be able to support scholars in becoming programmers rather than coders or to take the coding work of a scholar and turn it into a reusable and sustainable program.

Here we do not want to further engage in the discussion whether humanities scholars should learn how to code or what specifically they should learn, but we want to briefly introduce an idea from software engineering that adds another dimension to that debate. It is the concept of “leaky abstractions” that was popularized by Joel Spolsky in 2002. To understand what leaky abstractions are, however, we first need to understand what abstractions in programming are. At the core, any program consists of just ones and zeros. You don’t have to write these ones and zeros because there are several layers that abstract from it. Before a Java program can be run, there is another program (the compiler) that translates the Java code into another form of code, called Java bytecode (much less human friendly). That code will then be executed by yet another program that turns it into machine code, which is even less understandable by a human. This process continues until the code is made up of ones and zeros that can be executed on computer hardware. Each step in this process represents an abstraction. Java bytecode abstracts from machine code. Java abstracts from bytecode. Abstractions make programming easier. A Java programmer usually does not have to worry about registers or even memory allocation. However, sometimes the lower levels bleed through, and a programmer has to understand the underlying mechanisms to avoid pitfalls or to fix a bug. This is what Spolsky calls a leaky abstraction.1 A Java programmer, for example, might encounter an “OutOfMemoryError” message. Unless they understand what memory management is and how Java handles it, it is difficult to fix or avoid such a scenario. Other prominent examples involve encoding issues. While most programming languages make it fairly easy to deal with file encodings when processing text files, many programmers do not quite understand what a file encoding actually is. This often leads to copy-pasted solutions from the internet or comments such as “then I had to remove all the question marks that appeared in my text.” The details of how characters are represented in a computer by bits and bytes leak through to the programming layer that abstracts from them and can only be correctly handled if fully understood.

The problem with leaky abstractions is that it is often not obvious how they manifest or that they are even there. Additionally, many programming courses and tutorials promise that a language is easy to learn and emphasize how much can be accomplished in a short time frame. While this is often true, those resources forget to mention that there is a lot more under the hood that might affect the outcomes. Similarly, coding “recipes” that provide solutions to common scenarios, including copy and paste solutions for often-encountered issues, are very popular. While it is very helpful to have these resources available, they often result in people copying code without actually understanding what it is doing. This not only introduces unintended side effects but also has a big potential to introduce leaky abstractions. Spolsky so pointedly observes that “even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.” In 2019, Jayanti Bhandari Neupane and colleagues found that a Python script that was used for certain chemistry calculations returned different results depending on the operating system that was used. The authors point out that this could invalidate the research findings of multiple papers. Although Python abstracts from the underlying operating system, the fact that different file systems handle file sorting differently leaked through and influenced the calculations of the script (Neupane et al.). Leaky abstractions inevitably affect projects in computational humanities. The question is, what can we do about it?

My Code Has Bugs

Every programmer makes mistakes, which means that basically every program contains bugs (Soergel). If anything, self-taught programmers are likely to make more mistakes than fully trained software engineers because they are, among other things, often less prepared to handle leaky abstractions. Depending on the maturity of the code, the general estimation is that 1,000 lines of code contain from 1 to 50 bugs (Harrison). Now consider that 1,000 lines of code is not that much. Depending on the programming language used, this number is reached relatively quickly with just mildly complex code. The Python script linked in the footnote to index documents downloaded from HathiTrust in Elasticsearch is about 200 lines of code.2 The Pymarc Python library to work with Marc21 records consists of about 1,500 lines of code.3 Neither of these examples are very complex, but code grows fast.

In addition to a possibly higher rate of programming bugs compared to industry software, there is the potential issue of a lack of fundamental computer science knowledge among DH programmers. Others have observed “a worrying trend, in that the minimum requirement for scholars working in this field should be that they at least have an understanding of the difference between a markup language like XML, and a more sophisticated programming language such as Python” (O’Sullivan, Jakacki, and Galvin, i145–46). While more research is needed on that topic, personal experiences suggest that there does indeed exist a lack of understanding regarding fundamental computer science concepts, technologies, and skills. Examples include ignorance of the existence of and need for virtual environments in Python, unawareness of programming best practices such as version control and unit testing, and a lack of knowledge about multithreading and parallel processing. By no means does a computer science background guarantee knowledge or experience in these topics, but it is more likely that there exists at least some basic understanding of the form “I know what I don’t know” and that there is more foundational knowledge to understand those leaky abstractions.

There is a debate whether the same quality standards that are used in professional software engineering projects should be applied to computational research. Some argue that scientific code does not have to follow software engineering best practices because the users of the code are programmers with in-depth knowledge of the internal workings of the code and hence don’t require it to be well documented or maintainable (Bull; Anonymous). We disagree. While it is probably true that a lot of code developed for research is only used within the same lab by a small group of people, this does not mean that they would not benefit from good documentation and coding best practices. Things like spaghetti code (long code blocks with no structure or recognizable organization), bad variable naming, or missing tests count as bad practice in software engineering for good reasons. They make it hard to understand, check, and trust code. At the very least, understanding and modifying badly written code takes a lot longer than understanding and modifying well-documented code that follows best coding practices. Worst case, it makes it a lot more likely that the code contains bugs that are hard if not impossible to find. The argument that scientists write code that is free of any bugs that would affect research outcomes because they check carefully that any change they make does not lead to wrong or inaccurate research results is simply not true (see, for example, a case described by Miller). Furthermore, leaky abstractions are a good example why this view is problematic. The Wikipedia community even makes it a point to emphasize that “a particularly problematic cause of software defects” is “the reliance of the software developer on an abstraction’s infallibility” (Wikipedia Contributors). But abstractions are created by developers, who make mistakes, which means abstractions are never flawless. It is hard to believe that researchers would not rely on the trustworthiness of, for example, a programming language or popular library.

In fact, there are many examples of papers that have been retracted because of programming bugs. For example, in 2020, a paper in Current Biology (Letzner, Güntürkün, and Beste) was retracted by the authors because of an error in their MATLAB script. If specific results are used to inform policy decisions, things get especially complicated. A piece of code called covid-sim, developed by Professor Neil Ferguson and his team at Imperial College, London, was used to inform the United Kingdom’s policy on Covid-19, and it got a lot of public attention after it was released on GitHub.4 Reviewers criticized its overall quality and its inability to reproduce the same results given the same starting conditions (Boland and Zolfagharifard). There are quite a few different issues that can and should be discussed regarding this situation, and we do not intend to engage in the United Kingdom’s Covid-19 lockdown discussion, but it is a prime example of how research in general needs to change in order to deal with the complexity of computational approaches. The computational humanities are no exception. So, let us briefly look at some of the main issues of covid-sim and relate them to the computational humanities.

The first issue to note is that the code was not published until April 2020. Reportedly, the code had been worked on for multiple years and was used in other scenarios before it was uploaded to GitHub (see Boland and Zolfagharifard). Unfortunately, it is a quite common scenario that code is not or only later published in a version control system and made public. A survey conducted by the library of the University of St. Andrews found that out of 124 participants who developed programs only thirty-five (28 percent) used a form of version control (McCann). A survey by Yurdagül Ünal and colleagues came to a similar conclusion, stating that “gaps are evidenced” by, for example, “file naming systems and version control” (Ünal et al.). While neither of these two surveys were targeting digital or computational humanities scholars in particular, it is reasonable to assume a similar situation in this field. If code is not published, research relying on that code cannot be reproduced, improved, or applied to other datasets and projects, which means not only can research results not be validated but every project has to start at zero instead of building on previous programming work, slowing down research as a whole.

Another issue of the covid-sim code is that it contains multiple bugs (Boland and Zolfagharifard). It does not adhere to software engineering standards like modular design to make it easier to read and maintain. More important is the possibility that the code does not produce the same results when run under the same conditions as reported in a GitHub issue (U.K. RAMP “RED TEAM”). While this is, according to the discussion thread of the issue, acceptable in some scenarios, it is also suggested that it could be the result of a bug. The issue here is not that the program might produce different results for different executions, since this is the nature of some algorithms, but that it is not clear if this is the case or if the code is flawed.

While programming bugs in general are to an extent unavoidable even under perfect circumstances, it would be a manageable issue if proper review and publication practices for code existed. However, while the awareness grows for code quality, such practices are still not the standard. Additionally, in the computational humanities, most projects will not get the public attention that, for instance, the covid-sim model received. In many cases, even if the code is published, nobody will ever actually look at it, let alone take the time to review and test it. This also means that code developed for one project is unlikely to be reused for another project as it is too hard or too time-consuming to understand and modify. Instead of building off each other’s work, improving it, and eradicating bugs, developers repeat what others have already done, slowing down research.

So, What Can Be Done about It?

Tara McPherson states that “we must have at least a passing familiarity with code languages, operating systems, algorithmic thinking, and systems design. We need database literacies, algorithmic literacies, computational literacies, interface literacies” (McPherson, “Moving beyond Our Boxes”).5 While we absolutely agree with this statement, a “passing familiarity” might in many cases not be enough to achieve high-quality code. If a scholar develops some code to analyze an image or text dataset in C or C++, then they need to understand how memory allocation and management works. Similarly, if a researcher (or research team) creates a web application to allow users to run certain analyses on the web, there needs to be an awareness of the complications of multithreading environments. Depending on the goals of a project, there will be a point when a humanist scholar with some (potentially formal) computer science training simply won’t be enough. While some people are able to master both fields—for instance, a humanities discipline and software engineering—most of us will tend to be an expert in one but not both. As we look toward ways that we can develop, integrate, and support the programming expertise that we are calling for in DH, we offer several models and ideas.

One potential model DH can draw on is the concept of the “research software engineer” (RSE). In the past couple of years, the term RSE has become increasingly popular to describe those who apply their software engineering and other related skills to research problems in a number of disciplines (Vanessasaurus). In 2014, the Research Software Engineer Association was created to provide a “community and raise awareness of the UK’s Research Software Engineers” (UKRSE Association, “Welcome”). The association has now been succeeded by the Society of Research Software Engineering.6 Similar efforts are underway in several other countries, among them the United States, Germany, and the Netherlands.7 The Research Software Engineers Association described RSEs as “people in a variety of roles who understand and care about both good software and good research” (UKRSE Association, “What Is an RSE?”). By now, many institutions employ RSEs that can help scholars with their coding. There are even a few RSE units specializing in DH, such as King’s Digital Lab at King’s College, the Center for Digital Humanities at Princeton University, and the Digital Innovation Group at Arizona State University.8

In the digital humanities, however, a more common model than RSE units is the one in which a single person embedded in a DH project, lab, or department does all the programming work or in which a humanities scholar works with developers from a library. Often these people do not have a formal computer science background but are self-taught and might have a higher degree in a humanities discipline (see the survey results in DHTech). Additionally, they would often not describe themselves as RSEs, although they are doing at least some work typical for an RSE role. This hesitancy toward adopting the term “research software engineer” might partially stem from not having formal training in software engineering and not doing (and not wanting to do) software engineering 100 percent of the time. Another reason could be that titles can convey different meanings and areas of expertise depending on their contexts (the term “lecturer” has very different connotations in U.S. and U.K. systems, for example). Choosing a position as a research software engineer might raise concerns about future competitiveness when applying for more traditional academic jobs.

Scholarship best advances within professional groups. To meet the challenges related to computing in the humanities, professional communities geared toward people doing RSE work in the digital humanities can provide an important platform for networking and exchange.9 Certain types of software bugs, such as the ones originating from leaky abstractions, can best be avoided through continuous learning and support from other like-minded people. A community can provide and advertise training and career development opportunities and connect its members to benefit from each other’s expertise. This does not only mean that more skilled developers offer advice or training for less experienced ones but also that developers with, for example, formal computer science background and no humanities training can learn the intricacies and pitfalls they might encounter in a digital humanities project. For instance, issues like incorporating uncertainty in date representation or text extraction of 200-year-old manuscripts are not typically part of a computer science curriculum. In the interest of brevity, we will hereafter use the term “RSE” to encompass all people doing RSE work that identify with the description of the RSE role even if not with the term itself.

Quinn Dombrowski and colleagues (“Voices from the Server Room,” in this volume) describe their experiences about providing high-performance computing (HPC) support to humanists. They describe challenges when working with users in regard to missing background knowledge. This is another area in which RSEs or people with an RSE-like skill set could be helpful for any computational humanities scholar who does not (yet) possess the skills to work with HPC resources. RSEs could provide support to aid in the communication between HPC support staff and humanities scholars or to support execution of locally developed code on HPC resources.

However, simply adding an RSE to a research group is not the solution to all problems. There might be multiple desired outputs of a research project, such as a script to analyze a specific dataset and a web application to visualize and analyze results. Often all these tasks are simply lumped together as programming work. However, very different skill sets are needed for the different types of work. Many scholars doing computational work may have learned a programming language such as Python or R at some point to be able to automate calculations they know how to do on paper. This often works well, and all that might be needed is some guidance on, for example, proper version control. Good coding practices become important, however, when a scholar aims to allow others to use their code by developing libraries. A 200-line spaghetti code script is not easy to read or maintain, but it is usually doable by the author of the script. If that is turned into a library, however, meant to be used by others and possibly extended, that is a different story. Adding code to messy code keeps increasing its complexity and messiness and eventually makes it almost impossible to understand. It is at that point that we believe that scholars competent in programming but whose passion and interest do not lie with software development should start considering adding RSEs to a project. They can ensure that coding best practices are considered in the development process and make informed decisions about what infrastructure to best use for distribution and what quality assurance measures can be taken. Software development projects that go beyond scripts and libraries, such as projects that plan to develop web or desktop applications, should especially consider adding RSEs because this is what many software engineering programs focus on and it requires a very different skill set than writing short analysis scripts. While it is, of course, possible to acquire expertise and experiences in several different areas and for a scholar to also be a skilled software developer, most people do not start out with these skills. The practical experiences of a software developer have a big influence on the quality of their software. The more experienced a software developer is, the more knowledgeable they usually are with coding best practices, making them better equipped to deal with leaky abstractions. But experiences have to be acquired, which can be a long process full of mistakes and bugs.

Which brings us to our next point. Providing humanists with formal computer science training and adding RSEs to projects to increase code quality is just one piece of the puzzle. There will always be bugs in code, no matter how skilled and knowledgeable the programmer. Finding and eradicating those bugs is aided by measures an individual programmer can take such as writing tests. There are also procedures that can be put in place by teams of developers and the community at large. In particular, we are referring to code reviews (see, for example, Sadowski et al.). During a code review, a programmer other than the author inspects a piece of code with the goal of finding defects and suggesting improvements. Code reviews are generally valued as a measure to improve code quality (Sadowski et al.); however, a 2018 RSE survey suggests that many projects have only one developer (Philippe et al.), which can make team-internal code reviews difficult if not impossible. As Rebecca Sutton Koeser so pointedly observes: “no one will ever read my code. . . . In fact, in practical terms, the better I do my job, the less likely it is that someone will read my code, since the most likely reason for someone to trawl through code is because they have to do so in order to find and fix a bug” (Koeser, “[Re]viewing the Code,” 386).10 And even this is only likely to happen if someone reuses or extends someone else’s code.

A solution to this problem could be to establish a community code review and publication process that allows developers to submit their code to be reviewed by other programmers. There are journals that publish articles about software, but the code review part is limited, for good reasons.11 If done right, a community code review process could not just improve code quality overall but also give scholars more confidence when reusing the programs of other projects. An example of such an effort is the Network for Computational Modeling in the Social and Ecological Sciences (CoMSES Net), which aims to improve computational modeling in the social and ecological sciences (CoMSES; Rollins et al.). The digital and computational humanities community would greatly benefit from such a system, and there are in fact plans for a CoMSES sister project that focuses on digital humanities code as well as a newly created working group that was started at the Association for Computers and the Humanities (ACH) conference in 2021.12

However, implementing a generally accepted code review process faces multiple obstacles. Most importantly, cleaning up code enough to submit it, finding reviewers to review it, and then improving code according to reviewer comments is a time-consuming task. While probably most researchers can see the benefits of it, actually engaging in that process could be hampered by time and money constraints. Funding agencies and publishers are in the position to support and facilitate this process. They could put requirements in place for projects to publish their code in a way that ensures at least a minimum of code quality. Funding agencies could require implementation details, infrastructure, and personnel considerations that take into account required skills and technologies for projects that plan a computational component. It is also not unreasonable to require that an RSE or someone with similar experiences be on any grant that plans to develop software.

That is only part of the problem, though. As mentioned previously, many people doing RSE-related work in the computational humanities do not have a formal computer science education; in fact, many are self-taught. They might hesitate to submit code for review worrying that it is not good enough and that they might be shamed for it. A prominent example of how difficult it can be to ask coding-related questions and expose one’s own work is Stack Overflow.13 As of this writing, this question-and-answer site for programmers has 12 million users with 8,700 questions asked per day (Stack Exchange). While the Stack Overflow site is widely used by programmers, it has often been criticized for its unwelcomeness; coding beginners especially are afraid of judgmental and unfriendly comments on the site (Chipps). Similar observations can be made for the covid-sim project. There are several issues and discussions that lack productive criticism and are simply not helpful to the authors (see, for example, MrHash; pdehaye). Jeff Leek (“Simply Statistics”) describes an experience in which someone reviewing their code commented in a way that was “clearly designed for the sole purpose of embarrassing us (the authors) and discrediting our work.” It is therefore crucial that any code review process for computational humanities provides a safe space that allows for productive feedback, with the goal to improve code quality and coding skills, and it is not meant to be used as a tool to discredit another researcher’s work. Leek calls for a process that encourages researchers to share their code for review, recognizes that code inevitably contains mistakes, and suggests strategies to achieve positive outcomes for the author and the community when bugs are discovered.

This is especially important in the context of the discussion around computational humanities and gender bias (Latex Ninja, “The Computational Humanities and Toxic Masculinity?”). At the time of this writing, the original list that collected names of people interested in a computational humanities community included 24 percent women and 74 percent men.14 The Computational Humanities Research forum that was based on that list has 450 members (at the time of this writing), of which 36 percent identify as women and 55 percent as men.15 DHTech, a community of “technical people” in DH, has 19 percent female members and 80 percent male members.16 These numbers show that the computational humanities have a similar gender imbalance as computer science, which makes it likely that the situation for other minorities is not much different, either. A badly designed code review process could have the potential to make these numbers even worse if, for example, the process would be perceived as intimidating and requiring too much expertise of the reviewee. Guidelines on how to provide feedback must therefore be carefully crafted. A code review platform has to encourage and accept anyone writing code for a humanities project to submit to it, no matter how confident the author is. The community has to embrace the diverse backgrounds of its members. Only then can the overall quality of code in the computational (and digital) humanities increase.

It might also be useful to add other initiatives that go beyond just reviewing code to encourage more people interested in computational humanities. The idea of mentorship programs has entered discussions in the research software engineering community. Such programs could provide more hands-on support for coding beginners, and they could also specifically target certain minorities to make them feel more welcome in the community. Roopika Risam’s chapter in this volume, “Double and Triple Binds,” is a great presentation of the different factors that might keep researchers from learning to code and engage with computational humanities. Her experiences with having a team of people offering support when learning to program is especially relevant here. We need a community to provide this kind of support to any researcher who wants to learn how to code and program.

Whose Work Needs to Be Recognized?

Besides improving code quality, a generally accepted code review process would also have another benefit, which concerns the recognition of coding work. Depending on the project, there are various degrees of programming work involved in the implementation of a project. Oftentimes, however, this work is not being fully credited. In some cases, the computational part of a project might consist of the development of a script to clean and analyze a dataset. Usually, one of the authors is the author of that script and receives credit for their work through the publication of articles. While this does not address the issue of code quality, the author of the programming work is given credit for career advancement. In many cases, however, there are multiple applications or people involved in collecting, cleaning, and managing data, and often the work that is being done is only mentioned as a side note rather than being cited (e.g., in case of software applications or datasets; see Smith, Katz, and Niemeyer; Mooney and Newton) or being attributed academic recognition. Writing a script for data collection and cleaning is in many cases not recognized as an academic endeavor and therefore does not count toward career progression (Sula, Hackney, and Cunningham; D’Ignazio and Klein).

There are multiple issues associated with this situation. First of all, if cleaning up and publishing code does not provide any advantage for the author of the code in terms of career advancement, it is usually not very high on the list of things to do. In better cases, code is at least pushed to a repository. In worst cases, code is kept on the programmer’s hard drive to be lost potentially forever.17 If the goal is to develop a program, get results, and then publish these results, it might not matter to the author that much if the code follows standard best practices, is well tested, documented, readable, and maintainable. A clean, well-documented, and versioned code base optimizes flexibility (especially regarding the cyclical nature of computational research with lots of pivoting and refocusing), but it is difficult for early career scholars to justify spending time on improving something that will not count toward a tenure-track evaluation. Although this is slowly changing, it still presents an issue for many young scholars.18 But even in cases in which code counts for a scholar’s career progression, the question then becomes what does count. It might be easier to decide this question in cases of a web application that gets traffic that can be measured, but what about a Python library for analyzing citations? Does it count how often it has been forked on GitHub? How many projects use it? Or how well has the programmer documented it? Again, a code review and publishing platform might be a solution to at least some of these questions. This is where professional organizations such as the Open Modeling Foundation enter the picture. These are large enough organizations establishing broad standards that can be understood by tenure and promotion and other advancement committees. Similar to a traditional article, publication would allow scholars to show that they produced quality work.

However, this practice might not be appropriate in all cases. While code reviews are realistic for shorter scripts and possibly smaller libraries, it becomes less manageable for bigger web applications, desktop applications, or comprehensive libraries. Nonetheless, there should be a discussion about how such projects can leverage community support through, for example, partial or ongoing code reviews or third-party testing strategies to ensure the quality of the developed software. A code review process also does not account for reuse and application of a program. This problem exists across disciplines, and there is an active working group (see Katz and Chue Hong) that has produced principles and guidelines for citing software and giving credit (see, for example, Katz, Chue Hong, and Fenner; Smith, Katz, and Niemeyer).

Computational humanities, possibly to an even greater extent than digital humanities, requires a shift from the traditional one-person research project that is common in the humanities toward a more team-oriented approach that is typical of science. Even small projects can require several different skill sets and might only be realizable through collaboration. RSE units that exist in some universities can be well-suited collaborators for some projects (see, for example, University of Sheffield, “Research Software Engineering”). They can provide guidance in project planning and development support for projects and can be a way to ensure long-term maintenance of projects. Often these groups also provide training for scholars to, for instance, facilitate good coding practices and make sure researchers who do computational work possess basic computer science knowledge. This, however, is not necessarily a good fit for every project or not even a possibility in places where such a group simply does not exist. In such cases especially, we need more community support and infrastructure to ensure research results that can be trusted. A community code review process could be one important cornerstone of such infrastructure. Furthermore, providing opportunities for RSE-focused roles embedded in computational humanities projects (as opposed to service units) and a clearer career path for such roles could contribute to the development of the necessary expertise to successfully move the field of computational humanities forward.

While in this chapter we focused on programming work, there are many more areas of expertise that may be needed, such as that of a statistician, data analyst, or designer. Hence, for the computational (as well as the digital) humanities, we need to engage in conversations about new career paths and recognition, about training and publication, about community and collaboration. Only if we have these conversations can we achieve the code quality needed to establish trust in the results of our research. To paraphrase Douglas Adams, by doing so, we may not be going where we intended to go, but we believe we will end up where we need to be.

Notes

This chapter was written during the 2020 Covid-19 pandemic with two of the authors having small children and no day care. Some of these sentences were literally written with a two-year-old clinging on the writer’s back. Funding by NSF SBE 1656284 RCN: Mapping Authorities and Ontologies in Computational and Digital HPS and from the Smart Family Foundation is gratefully acknowledged.

1. 1. Spolsky was not the first to put this concept into words, but he popularized the term by defining the law of leaky abstractions that says that “all non-trivial abstractions, to some degree, are leaky.”

2. 2. https://github.com/diging/utilities/blob/master/ht-elastic/ht_index.py.

3. 3. https://github.com/JaimieMurdock/pymarc.

4. 4. https://github.com/mrc-ide/covid-sim.

5. 5. From the section “Moving Beyond Our Boxes” in McPherson, “Why Are the Digital Humanities So White?,” 152.

6. 6. https://society-rse.org/.

7. 7. For the association in the United States, see https://us-rse.org/. The German association for people behind research software is at https://de-rse.org/. The Netherlands group is at https://nl-rse.org/.

8. 8. For the website of King’s Digital Lab at King’s College, see https://kdl.kcl.ac.uk/. The Center for Digital Humanities at Princeton University website can be found at https://cdh.princeton.edu/ and the website of the Digital Innovation Group at Arizona State University is https://diging.asu.edu/.

9. 9. See DHTech, a special interest group of the Alliance of Digital Humanities Organizations (ADHO), https://dh-tech.github.io/; see also Research Software Engineering in den Digital Humanities, https://dh-rse.github.io/.

10. 10. From the section “Reviewing the Code” in Koeser, “Trusting Others to ‘Do the Math.’”

11. 11. See the Journal of Open Research Software (https://openresearchsoftware.metajnl.com/), and the Journal of Open Source Software (https://joss.theoj.org/).

12. 12. See the DHTech Community Code Review Working Group (https://dhcodereview.github.io/). The CoMSES community has, in the meantime, established the Open Modeling Foundation to provide a broader platform for computing and modeling across a number of disciplines (see https://openmodelingfoundation.github.io).

13. 13. https://stackoverflow.com/.

14. 14. The percentages are from https://github.com/cohure/CoHuRe/issues/1. In this case, gendered names and pictures were used to decide gender. Some users could not be identified by using just these two criteria, hence the percentages do not add up to 100 percent.

15. 15. See Computational Humanities Research at https://discourse.computational-humanities-research.org/. Again, the percentages do not add up to 100 because the same counting method as before (gendered names and pictures) was used.

16. 16. https://dh-tech.github.io/.

17. 17. We acknowledge that this practice has become less common with GitHub being increasingly accepted by the DH community, but we believe it is still a not negligible issue. Data collected by Lisa Spiro and Sean Morey Smith (“Evaluating GitHub”) also still show that there is a gap between working on a DH project and having a GitHub account.

18. 18. See, for example, CDRH, “Promotion and Tenure Criteria.”

Bibliography

1. Afanador-Llach, Maria José, James Baker, Adam Crymble, Victor Gayol, Martin Grandjean, Jennifer Isasi, et al. “2019 Programming Historian Deposit Release.” Programming Historian. 2019. November 1, 2019. https://doi.org/10.5281/zenodo.3525082.
2. Anonymous. “Discussion of ‘Why We Can Ignore Reviews of Scientific Code by Commercial Software Developers.’” Hacker News, May 18, 2020. https://news.ycombinator.com/item?id=23221697.
3. Boland, Hannah, and Ellie Zolfagharifard. “Coding That Led to Lockdown Was ‘Totally Unreliable’ and a ‘Buggy Mess,’ Say Experts.” The Telegraph, May 16, 2020. https://www.telegraph.co.uk/technology/2020/05/16/coding-led-lockdown-totally-unreliable-buggy-mess-say-experts/.
4. Bull, Phil. “Why You Can Ignore Reviews of Scientific Code by Commercial Software Developers.” Lumps “n” Bumps, May 10, 2020. https://philbull.wordpress.com/2020/05/10/why-you-can-ignore-reviews-of-scientific-code-by-commercial-software-developers/.
5. CDRH. “Promotion and Tenure Criteria for Assessing Digital Research in the Humanities Evaluating Digital Scholarship.” Center for Digital Research in the Humanities University, University of Nebraska–Lincoln. Accessed June 8, 2020. https://cdrh.unl.edu/articles/promotion.
6. Chipps, Sara. “What a Very Bad Day at Work Taught Me about Building Stack Overflow’s Community.” Stack Overflow Blog, July 18, 2019. https://stackoverflow.blog/2019/07/18/building-community-inclusivity-stack-overflow/.
7. CoMSES. “A Growing Collection of Resources for Computational Model-Based Science.” CoMSES Network. Accessed May 21, 2020. https://www.comses.net/.
8. DHTech. “DH RSE Survey Results.” DHTech. Accessed August 9, 2021. https://dh-tech.github.io/survey-results-2020.
9. D’Ignazio, Catherine, and Lauren F. Klein. “Show Your Work.” In Data Feminism, 173–201. Cambridge, Mass.: MIT Press, 2020.
10. Harrison, W. “The Dangers of End-User Programming.” IEEE Software 21, no. 4 (2004): 5–7.
11. Hockey, Susan. “Workshop on Teaching Computers and the Humanities Courses.” Literary and Linguistic Computing 1 (1986): 228–29.
12. Katz, Daniel S., and Neil P. Chue Hong. “Software Citation in Theory and Practice.” In Mathematical Software—ICMS 2018. Lecture Notes in Computer Science, edited by James H. Davenport, Manuel Kauers, George Labahn, and Joseph Urban, 289–96. Cham: Springer, 2018.
13. Katz, Daniel S., Neil Chue Hong, and Martin Fenner. “FORCE11 Software Citation Implementation Working Group.” GitHub Force11/Force11-Sciwg. Accessed May 27, 2020. https://github.com/force11/force11-sciwg.
14. Klein, Lauren F. “Code.” In Digital Pedagogy in the Humanities, edited by Rebecca Frost Davis. Accessed May 27, 2020. https://digitalpedagogy.hcommons.org/keyword/Code.
15. Koeser, Rebecca Sutton. “Trusting Others to ‘Do the Math.’” Interdisciplinary Science Reviews 40, no. 4 (2015): 376–92. https://doi.org/10.1080/03080188.2016.1165454.
16. Ladurie, Emmanuel Le Roy. “La fin des érudits.” Nouvel Observateur, May 8, 1968.
17. Latex Ninja. “The Computational Humanities and Toxic Masculinity? A (Long) Reflection.” Latex Ninja (blog). April 19, 2020. https://latex-ninja.com/2020/04/19/the-computational-humanities-and-toxic-masculinity-a-long-reflection/.
18. Latex Ninja. “News on the DH and Gender Equality.” Latex Ninja (blog). May 24, 2020. https://latex-ninja.com/2020/05/24/news-on-the-dh-and-gender-equality/.
19. Leek, Jeff. “Simply Statistics: How Could Code Review Discourage Code Disclosure? Reviewers with Motivation.” Simply Statistics, September 26, 2013. https://simplystatistics.org/posts/2013-09-26-how-could-code-review-discourage-code-disclosure-reviewers-with-motivation/.
20. Letzner, Sara, Onur Güntürkün, and Christian Beste. “Retraction Notice to: How Birds Outperform Humans in Multi-Component Behavior.” Current Biology 30 (2020): 754.
21. McCann, Patrick. “Research Software Engineering Inside and Outside the Library.” Presentation at RLUK Conference 2017. Accessed May 19, 2020. https://research-repository.st-andrews.ac.uk/bitstream/handle/10023/10488/rse_in_out_lib.pdf?sequence=1&isAllowed=y.
22. McDaniel, Caleb. “Interview with Jason Heppler.” Digital History @ Rice, March 27, 2013. http://digitalhistory.blogs.rice.edu/2013/03/27/interview-with-jason-heppler/.
23. McPherson, Tara. “Why Are the Digital Humanities So White? Or Thinking the Histories of Race and Computation.” In Debates in the Digital Humanities, edited by Matthew K. Gold, 139–60. Minneapolis: University of Minnesota Press, 2012.
24. Miller, Greg. “A Scientist’s Nightmare: Software Problem Leads to Five Retractions.” Science 314, no. 5807 (2006): 1856–57.
25. Mooney, Hailey, and Mark P. Newton. “The Anatomy of a Data Citation: Discovery, Reuse, and Credit.” Journal of Librarianship and Scholarly Communication 1, no. 1 (2012). https://doi.org/http://dx.doi.org/10.7710/2162-3309.1035.
26. MrHash. “Archive This Project and Start Again.” GitHub Issues for Covid-Sim. May 7, 2020. https://github.com/mrc-ide/covid-sim/issues/174.
27. Mullen, Lincoln A. Computational Historical Thinking: With Applications in R. 2018. Accessed June 19, 2020. http://dh-r.lincolnmullen.com/.
28. Neupane, Jayanti Bhandari, Ram P. Neupane, Yuheng Luo, Wesley Y. Yoshida, Rui Sun, and Philip G. Williams. “Characterization of Leptazolines A−D, Polar Oxazolines from the Cyanobacterium Leptolyngbya Sp., Reveals a Glitch with the ‘Willoughby−Hoye’ Scripts for Calculating NMR Chemical Shifts.” Organic Letters 21 (2019): 8449–53. https://doi.org/10.1021/acs.orglett.9b03216.
29. Nowviskie, Bethany. “On the Origin of ‘Hack’ and ‘Yack.’” In Debates in the Digital Humanities 2016, edited by Matthew K. Gold and Lauren F. Klein. Minneapolis: University of Minnesota Press, 2016. https://dhdebates.gc.cuny.edu/read/untitled/section/a5a2c3f4-65ca-4257-a8bb-6618d635c49f#ch07.
30. O’Sullivan, James, Diane Jakacki, and Mary Galvin. “Programming in the Digital Humanities.” Digital Scholarship in the Humanities 30, supp. 1 (2015): i142–47. https://doi.org/10.1093/llc/fqv042.
31. pdehaye. “Work with the *Actual* Professionals at Opening Up Code.” GitHub Issues for Covid-Sim. May 11, 2020. https://github.com/mrc-ide/covid-sim/issues/209.
32. Philippe, Olivier, Martin Hammitzsch, Stephan Janosch, Anelda van der Walt, Ben van Werkhoven, Simon Hettrick, Daniel S. Katz, et al. “softwaresaved/international-survey: Public Release for 2018 Results (Version 2018-v.1.0.2).” Zenodo. Accessed June 8, 2020. http://doi.org/10.5281/zenodo.2585783.
33. Posner, Miriam. “Here and There: Creating DH Community.” In Debates in the Digital Humanities 2016, edited by Matthew K. Gold and Lauren F. Klein. Minneapolis: University of Minnesota Press, 2016. https://dhdebates.gc.cuny.edu/read/untitled/section/c6b8f952-acfd-48b6-82bb-71580c54cad2.
34. Rollins, Nathan D., C. Michael Barton, Sean Bergin, Marco A. Janssen, and Allen Lee. “A Computational Model Library for Publishing Model Documentation and Code.” Environmental Modelling and Software 61 (2014): 59–64. https://doi.org/10.1016/j.envsoft.2014.06.022.
35. Sadler, Jesse. “New Kinds of Projects: DH 2.0 and Coding.” Jesse Sadler—A Blog about Early Modern History and Digital Humanities, August 16, 2017. https://www.jessesadler.com/post/new-kinds-of-projects/.
36. Sadowski, Caitlin, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. “Modern Code Review: A Case Study at Google.” In ICSE-SEIP ’18: Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice, 181–90. New York: Association for Computing Machinery, 2018. https://doi.org/10.1145/3183519.3183525.
37. Smith, Afron M., Daniel S. Katz, and Kyle E. Niemeyer. “Software Citation Principles.” PeerJ Computer Science 2:e86 (2016): 1–31. https://doi.org/10.7717/peerj-cs.86.
38. Soergel, David A. W. “Rampant Software Errors May Undermine Scientific Results.” F1000Research 3, no. 303 (2015). https://doi.org/10.12688/f1000research.5930.1
39. Spiro, Lisa, and Sean Morey Smith. “Evaluating GitHub as a Platform of Knowledge for the Humanities.” Presentation at Digital Humanities 2016. Accessed June 8, 2020. https://digitalscholarship.files.wordpress.com/2016/07/spirosmithdh2016githubpresentationfinal.pdf.
40. Spolsky, Joel. “The Law of Leaky Abstractions.” Joel on Software, November 11, 2002. https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/.
41. Stack Exchange. “Stack Exchange.” Accessed May 26, 2020. https://stackexchange.com/sites?view=list#users.
42. Sula, Chris A., S. E. Hackney, and Phillip Cunningham. “A Survey of Digital Humanities Programs.” Journal of Interactive Technology and Pedagogy 11 (2017). https://jitp.commons.gc.cuny.edu/a-survey-of-digital-humanities-programs/.
43. UK RAMP “RED TEAM” at Edinburgh University. “Apparent Unexpected Difference in Results When Re-Using a Binary Network File.” GitHub Issues for Covid-Sim. April 21, 2020. https://github.com/mrc-ide/covid-sim/issues/116.
44. UKRSE Association. “Welcome to the UK Research Software Engineer Association.” Accessed May 25, 2020. https://socrse.github.io/legacy-website/.
45. UKRSE Association. “What Is an RSE?” Research Software Engineer Association. Accessed May 25, 2020. https://rse.ac.uk/what-is-an-rse/.
46. Ünal, Yurdagül, Gobinda Chowdhury, Serap Kurbanoğlu, Joumana Boustany, and Geoff Walton. “Research Data Management and Data Sharing Behaviour of University Researchers.” Proceedings of ISIC: The Information Behaviour Conference 24, no. 1 (2019). http://informationr.net/ir/24-1/isic2018/isic1818.html.
47. University of Sheffield. “Research Software Engineering Sheffield.” Accessed May 27, 2020. https://rse.shef.ac.uk/.
48. Vanessasaurus. “The Story of the Research Software Engineer.” Vanessasaurus Fun, July 9, 2019. https://vsoch.github.io//2019/the-research-software-engineer/.
49. Van Zundert, Joris J., and Ronald Haentjens Dekker. “Code, Scholarship, and Criticism: When Is Code Scholarship and When Is It Not?” Digital Scholarship in the Humanities 32, supp. 1 (2017): i121–33. https://doi.org/10.1093/llc/fqx006.
50. Weingart, S. “Teaching Yourself to Code in DH.” The Scottbot Irregular. Accessed January 3, 2020. https://scottbot.net/teaching-yourself-to-code-in-dh/.
51. Wikipedia Contributors. “Leaky Abstraction.” Wikipedia. Last modified September 29, 2023. https://en.wikipedia.org/wiki/Leaky_abstraction.