Planned Obsolescence

traditional peer review and its defenses

0 David Shatz notes in the introduction to his 2004 volume on peer review that his text is not only “the first book-length study of peer review that utilizes methods and resources of contemporary philosophy,” but also “the first wide-ranging treatment of the subject by a scholar in the humanities,” a fact that becomes all the more surprising when he points out:

0 Besides its ethical aspect, the topic also has dimensions of epistemological significance, since it implicates such concepts as truth, bias, relativism, conservatism, consensus, and standards of good argument. Philosophers and other humanities scholars have produced a voluminous literature on these subjects. Yet they have not applied their approaches to these topics to peer review itself, that is, to the very procedures and practices that produced much of the voluminous literature in ethics, epistemology, and so many other fields. (Shatz 4)

0 Shatz indicates a number of reasons why this may be so, including that the more nebulous (or, rather, problematized) understanding of “truth” in the humanities precludes such scholars from being able to “show that a peer review was wrong” (Shatz 6), and that a critical study of peer review might require empirical work of a sort for which humanists are neither trained nor rewarded. Beyond these factors, however, I’d argue that a critical study of the epistemological practices of peer review requires a form of self-analysis that, as Donald Hall has argued in The Academic Self, many of us resist. Such resistance might suggest an underlying anxiety about the outcome of the analysis, a concern that the time-honored procedures and standards of the humanities might be shown to be flawed – and thus that the work that has developed through those procedures and according to those standards might be even further marginalized within the academy’s mission of knowledge-production. However, as Hall argues, genuinely “owning” our careers and the ways in which we conduct them requires taking the risk of applying our critical skills to an examination of “the textuality of our own profession, its scripts, values, biases, and behavioral norms” (Hall xiv). Too often, such examinations and proposals for change are met with stern reminders that We Have Never Done It That Way Before. The apparently intractable nature of the way things have always been done is precisely the kind of signal that, in other institutions, impels scholars to critical analysis; a resistance to turning the same critical eye on our own seemingly naturalized assumptions may create (or deepen) an atmosphere of intellectual oppression and stultification, as we allow systems in which we do not genuinely have faith to dictate our engagements with the world, and with one another. Opening up the basis of those engagements through a thorough reconsideration of peer review may be precisely what we need in order to allow our work to help shape the ways of knowing of the contemporary world.

0 Resistance to considering the merits of a more open mode of publishing often runs something like that expressed – in, I assume, an intentionally hyperbolic fashion – by Shatz:

2 It is hard to say who would have the biggest nightmare were open review implemented: readers who have to trek through enormous amounts of junk before finding articles they find rewarding; serious scholars who have to live with the depressing knowledge that flat earth theories now can be said to enjoy ‘scholarly support’; or a public that finds the medical literature flooded with voodoo and quackery. Let us not forget, either, that editors and sponsoring universities would lose power and prestige even while their workload as judges would be eliminated. (16)

2 The vehemence of such resistance often reveals something about the nervousness of those who express it, and as in much psychotherapeutic discourse, it’s only after some initial projection and displacement that the real source of that anxiety comes out: the loss of “power and prestige.”[1.9] However, in responding to those earlier displacements of anxiety, one can provide certain kinds of reassurance. The computer technologies that make open review possible also make possible the implementation of analytical tools that can help filter “rewarding” articles from any “junk” they may be mired in, whether those tools employ the results of the open review system themselves or use other modes of sophisticated textual analysis and recommendation. Further, serious scholars depressed by the apparent anything-goes nature of open publishing can see to it, by participating in the review system, that “flat earth” theories obtain the reception that they deserve. In fact, the public is already flooded with voodoo and quackery, a state easily revealed through the most cursory look at the relationship between the pharmaceutical industry and the bulk of publicly available medical information; post-publication review might actually help readers know how to interpret the material that’s out there.[1.10] But finally, if the loss of power and prestige are our primary concerns in clinging to closed review, we would be best served by admitting this to ourselves up front. If we enjoy the privileges that obtain from upholding a closed system of discourse sufficiently that we’re unwilling to subject it to critical scrutiny, we may also need to accept the fact that the mainstream of public intellectual life will continue, in the main, to ignore our work. This can, of course, be rationalized as the inevitable, unenviable fate of genius in a world of mediocrity.