Jana Remy

As important as it is that professional academic organizations such as the Modern Language Association (MLA) and American Historical Association (AHA) have created guidelines for tenure unit criteria for evaluating digital humanities scholarship, there remains a crucial element of administrative infrastructure to apply these guidelines for tenure and promotion candidates. That element is the institutional platform used for the tenure and promotion process. That faculty affairs staff and the campus IT department are unlikely to prioritize the need for a web-based digital evaluation platform to support DH work makes it unlikely that such scholarship can or will be evaluated according to the MLA/AHA guidelines. Thus scholars working in DH may continue to inhabit the margins, and the “exceptions” of the typical evaluation workflow are unlikely to have their work evaluated in a way that respects the “medium specificity” of their scholarship or guarantees the “procedural clarity and fairness” of the review process.1

In this article I offer my perspective as an alt-ac administrator who has grappled with the development and support of an online tenure and promotion (T&P) platform for nearly a decade. During that time I have not only supported my campus’s platform but have spoken at dozens of academic conferences about my work and from those conversations have gained a solid understanding of the variety of ways universities are providing a T&P platform for their faculty. It is my hope that the lessons I have learned along the way can foster a dialogue between various institutions and their academic units, faculty affairs, and IT departments, to make wise choices about the platform that they adopt for T&P, and especially that they can do so in a way that supports (and encourages) all types of digital scholarly endeavors, including those in the digital humanities. On the basis of my experience I would urge institutions to adopt a platform for web-based portfolios. Here I list my reasons, along with explanations of why a web-based system is the best platform for tenure and promotion files.

My role in the T&P process is to mitigate any technological hurdles candidates might face in the process of uploading, embedding, or linking to the supporting materials in their digital portfolios. Not only do I support the candidates with the creation of their dossier but I also troubleshoot any issues that occur with the evaluators and the administrators who oversee the tenure and promotion process. While in this role I have seen hundreds of T&P dossiers, have examined numerous software packages promising to revolutionize the tenure process, and have seen very little progress toward applying the guidelines from the MLA (and other professional organizations) to evaluate digital scholarship. The reasons for this are varied, but they seem to stem largely from the fact that decisions about software adoptions are largely about the business processes of the university and are not made in consultation with faculty who have a stake in how their content is represented on the platform.

First, I offer a bit of background on why I became involved in the tenure process at my university. Years ago, when our chancellor became interested in a digital portfolio platform, I found myself consulting with campus leaders about their requirements for such a platform. At that time, the motivation for moving to digital was due mainly to the cost of shipping large boxes of private materials to external reviewers across the globe, but it also reflected the concerns about paper dossiers’ inability to represent the creative and scholarly activities of faculty working in the arts. The faculty in our film school and in the college of performing arts were not able to adequately represent their work in the pages of a binder. Moreover, there were logistical challenges to shipping DVDs of films or of performances to each reviewer, and to reproducing high-quality prints of the studio art produced by our faculty. At that time I was aware of the MLA’s then-new guidelines about evaluating the scholarly output of faculty working in digital media, and I added their guidelines to the growing list of requirements for our ideal digital portfolio platform.

That took place in 2010, and as we surveyed the landscape of various vendors who provided ePortfolio software, we found none that met the requirements of being able to stream multimedia content, nor were they easy to use or customizable to adapt to the variety of disciplines found on our campus. Following the precedents of a variety of popular digital humanities projects such as CommentPress, Anthologize, and PressForward, we settled on an in-house installation of WordPress MultiSite to support our dossiers. WordPress seemed a wise direction to move because of its flexibility and ease of use. There was also a robust local developer community that we could draw from, should the need for customizations arise.

In our pilot year, a dozen faculty tried the WordPress platform. The next year we doubled that number. And in the third year the chancellor mandated the platform for all faculty for their critical year reviews and their tenure and promotion files. The reason for the mandate was the high level of satisfaction on the part of the evaluators. No longer did they have to go to our administration building during business hours and pore over reams of materials piled into banker’s boxes. Instead, they could peruse the candidate’s ePortfolio from anywhere with a web browser and an internet connection. Additionally, committees could pull up a candidate’s portfolio on a screen in a conference room and discuss the materials together without having to pass around hard copies of documents. They also could collaboratively author a recommendation letter online as they moved seamlessly back and forth from portfolio to Word. In short, the success of the endeavor was due primarily to the ease of use for the evaluators and the evaluation committees. This was nearly the opposite of what was expected when we first launched the platform, as deans were grumbling about changing the age-old pattern of file review. They quickly became the most enthusiastic champions of our ePortfolios. That the buy-in came from the top down meant that candidates with digital scholarship were on a level playing field with their colleagues who had more traditional forms of research output, rather than having to vie for exceptions and alternatives to the standard process.

On the basis of my experience with selecting and supporting a campus platform for ePortfolios, I have learned the following about the benefits and challenges of using digital portfolios for T&P review:

• Avoid platforms that are web forms with PDF output. Whether building an in-house web form or purchasing a product with web form capabilities, using this style of ePortfolio for digital dossiers will limit faculty in expressing the full range of their creative and scholarly endeavors. Even a well-designed web form will not adapt to the multiplicity of ways that faculty create and publish content. Rather, a web form–based template generally adheres to and reinforces the notion that scholarship occurs in journals and in books rather than opening the possibilities for a variety of scholarly expressions that do not necessarily fit into an author / title / publication date style of format.

  The move toward the PDF output of an ePortfolio file is also problematic for the most obvious reason: it creates a static document that is often over five hundred pages long. Those gigantic PDFs are tedious for evaluators to peruse, and unless care is taken to create hyperlinks between sections and to materials outside of the dossier, it replicates the same problems as paper in that large PDFs do not allow faculty working in a digital realm to feature their work in the platform for which it was intended to be accessed and viewed.

  Adopting a platform that allows faculty to create a hyperlinked, interactive, and dynamic site for their portfolio facilitates an ease of navigation for the evaluator in addition to allowing the embedding and linking of materials.

• Require the platform to stream multimedia. While most ePortfolio platforms allow the upload and attachment of multimedia files to faculty dossiers, few allow the content to be hosted and streamed directly from the platform. Streaming is necessary because if the media files are merely attached, evaluators must download each file before listening to or viewing the media. Given that faculty on our campus produce feature-length films and hours-long musical scores, it is simply not acceptable to put the burden of file download on the evaluators, for reasons of both time and space. In some cases, attaching and downloading the media files to the evaluator’s local computer also might violate copyright or intellectual property rules about that media.
• Consider whether to integrate with other campus platforms. On our campus we wanted scholarship held in our institutional repository, the Digital Commons (DC), to be linked to the candidates’ T&P dossier. Doing so would allow faculty to upload their materials in only one place, to a location with a persistent URL/DOI. However, we encountered two obstacles with that plan: the first was that our DC uses Bepress software that does not stream multimedia (see my previous point about why this is necessary), and the second was that some teaching materials are not appropriate for public access, such as videos of students in the classroom, and our DC defaults to making all materials publicly accessible. Thus, an integration with the DC was useful only for a subset of materials.

  There is an emerging trend in the landscape of digital dossiers to have the web form data fields integrate with an institutional data repository to facilitate the tracking of faculty productivity according to algorithms that count the number of publications correlated with their impact factor. This is a concerning practice, especially within the humanities, because it relies on formulas for measuring and valuing the research and creative output of faculty. In short, it can place a higher value on quantity over quality and might diminish the visibility of publications in “nontraditional venues such as new open access journals, blogs or articles written and posted on a personal website or institutional repository, or non-narrative digital projects.”2 I would encourage any campus that is exploring this integration to be cautious about how such metrics would be used to represent faculty productivity.

  Other on-campus platforms that one could consider integrating with the ePortfolios include the Learning Management System, course evaluation system, and faculty websites. In each case, the integrations allow for faculty to link to their work on those platforms rather than needing to upload or re-represent that work separately in their T&P dossier.

• Be wary of platforms that mandate disciplinary homogeneity. In the last decade I have seen hundreds of ePortfolios, ranging the gamut from pharmacy faculty to poetry faculty. I have yet to see any that were identical in tone, structure, or form. Even though we use the same fairly vanilla WordPress theme for each dossier, faculty have a remarkable amount of latitude about how they present their materials. Some like outlines with bullet points, others prefer long narratives broken up with page breaks, and yet others feature visualizations more than text. This wide variety allows faculty to reflect their unique career trajectory in the manner that most suits their sense of aesthetics and discipline-specific expression.

  Moreover, it allows for candidates to present materials in a multitude of formats, which is as important in their discussion of their research as it is in their teaching folio. For example, in their teaching materials section a faculty member can include a written statement on their teaching philosophy, images showing them in their classroom, videos of students doing in-class presentations, and PDFs of syllabi and course assignments. Together, all those items offer a much deeper view into the candidates’ classrooms than any of those alone would provide, and they feature a look at the many ways that faculty engage with their students rather than offering only a static repository of syllabi and course evaluations. That candidates can order, emphasize, and feature different aspects of their teaching practice, as well as link to student materials hosted on the web, allows them to tell the story of their teaching as dynamic, evolving, and multimodal.

• Understand the good, bad, and the ugly of the user experience. For the most part, creating an online portfolio ought to be easy for a candidate who has basic digital fluency. If she can attach a document to an email and buy a book on Amazon, she is likely to have the skills necessary to create an ePortfolio. For some candidates the work of attaching and uploading files can feel tedious, especially when they are locating, renaming, and uploading more than a decade’s worth of teaching evaluations or a similar number of syllabi. Additionally, there is a small subset of users who struggle with making the leap from print to digital, and this hurdle impacts their confidence in an already stressful process. With a web-based platform the greatest challenge for faculty is often that web pages are dynamic and responsive to the device and platform where they are accessed. This means that the line breaks and images do not fall in exactly the same spot as they would in a Word document and that they can vary significantly from desktop to tablet. For the faculty member who is used to the static nature of a PDF or a word document, this can be frustrating and can contribute to that candidate’s feeling a lack of control over the process of creating their dossier. Although training can help to ameliorate this issue, it remains frustrating to many candidates that the text and layout of their dossier varies from one computer to another. On my campus I held quarterly workshops to train faculty on using the ePortfolio platform and also scheduled one-on-one consultations upon request. This level of support tended to assure candidates that their materials were well represented in their digital dossiers.

  But these small user issues aside, using WordPress—a platform with millions of users and a responsive development community—provides a fairly straightforward interface for our faculty. Most have used WordPress at least once before, so it feels intuitive to use it for their ePortfolio. For most it feels open and customizable: they can alter the size, color, and style of the text; they can embed a wide variety of media including tables and images; and they can hyperlink to other sections of their ePortfolio as well as to content on the web.

  Hands down, the user group that is most enthusiastic about using the web-based ePortfolio platform are the evaluators. The ease of login and viewing the materials far outweighs the logistical challenges of working with hard-copy or PDF dossiers.

• Resolve concerns about archiving the materials. Sustainability is a concern for all DH scholarship, with the long-term access and function of online resources in question for all born-digital materials. The concerns about long-term preservation of ePortfolios are no different. On my campus, for example, there is a mandate that tenure dossiers must be kept for the entire career of that faculty member. That is certainly a tall order for junior faculty who may be employed for three, or even four, decades at our institution. To address this problem we store periodic backup snapshots of our WordPress server, and we use a web crawler to copy the HTML of each portfolio at the close of each review cycle, which is then added to the digital file kept on each faculty member at our institution.

In conclusion, the most significant lesson emerging from my years of supporting ePortfolios is that a web-based platform for creating and evaluating content allows all faculty, not just DH practitioners, to build customized media-rich dossiers that feature both print-based and digital content. I have observed that having a web-based platform ensures that evaluators are familiar with evaluating digital materials, which has been a concern for those working in DH who have struggled to represent their work in a process that used to be almost exclusively on paper. Using online ePortfolios minimizes the leap from traditional forms of scholarship to those of the digital. Moreover, I have seen that even the most traditional scholars have an increasing amount of born-digital material for their dossiers, which might include ePubs, online data sets, webinars, podcasts, or online-only journal articles. Thus for all T&P candidates, having a web-based ePortfolio platform on which digital content can be easily linked to and embedded features the growing range of ways that they now teach and work, and of course it familiarizes evaluators with the wide variety of materials that are created by twenty-first-century faculty.

There is some irony in my advocacy for a web-based platform for ePortfolios, because at my institution we will soon conduct an evaluation of our platform to determine whether we can continue to support it. In the past few years our IT department has moved away from in-house development and has adopted more out-of-the-box and cloud-based services. The reason is that it is too costly to maintain and support systems that are unique to our campus. Simultaneously, several for-profit IT companies are promoting ePortfolio software packages for T&P that feed higher ed’s growing desire for data metrics about faculty productivity and that may well come at a cost for faculty whose work is not easily measured according to those standard metrics. While I am sensitive to my campus’s needs to conserve resources and make wise decisions about IT services, it is with some concern about the future that I close this article. I suspect that decisions about my campus’s T&P platform will not be made with the needs of digital humanities faculty in mind, nor even with an eye to the benefits that all faculty enjoy with our current platform. I fear that a move to a standard software solution may introduce more roadblocks into the tenure and promotion process for digital humanities faculty. Thus I would encourage the MLA and other similar organizations to advocate for a web-based platform for candidates, a platform that respects medium specificity, allows for the embedding and streaming of multimedia, supports hyperlinks to online materials, and fosters the creative expression of all candidates. Because it is not enough to have guidelines for evaluation, there also needs to be advocacy for the infrastructure necessary to support those guidelines for evaluating faculty with digital scholarship.