A First Stab at Some General Principles
Kathleen Fitzpatrick - March 30th, 2007One of the things we attempted to do in the course of our meeting this week was to draw up a cluster of principles that should guide the kinds of work published through MediaCommons, as well as a set of principles guiding the new mode of open peer-to-peer review that we hope the network will instantiate. The lists that follow are drawn from my notes, taken in the course of the meeting, and then elaborated upon in order to complete certain thoughts and draw out certain conclusions. I hope that the other folks who were at the meeting will chime in and correct any misapprehensions or misinterpretations that I may put forward here, and fill in any gaps in my memory. I likewise hope that many other users of this network-in-process will respond as well, and spread the word about this discussion, helping us to create the best possible system for the production of networked scholarly discourse.
Principles Guiding MediaCommons Projects:
1. MediaCommons projects ought to take advantage of or be enabled by the digital environment — that is, they should be multi-mediated, or networked, or community-built, or they should foreground the process of creation and revision. In any case, projects (and proposals for projects) should be clear about how they are using the networked publishing environment as a part of their practice or methodology.
2. MediaCommons particularly desires projects that demonstrate a concern for accessibility to multiple audiences, particularly in terms of writing style. This is not to say that all projects must be equally readable by all audiences, or that difficulty is not valued; rather, it is to say that as the network is open, we believe our work should be open, and valuable, to everyone within that network.
3. Any work published on MediaCommons should provide some transparency of process that persists beyond the finished document, a revision history that a reader can drill into, for instance, as in a wiki.
4. Projects should be published on MediaCommons under the Creative Commons license of the author’s choosing, and may be republished in other forms beyond MediaCommons, but the authors must commit to the project’s persistence within the network. This persistence does not preclude proliferation, though any further publications must acknowledge MediaCommons.
Principles Guiding the MediaCommons System of Peer-to-Peer Review
1. Peer review within MediaCommons is designed less to serve as a system of “gatekeeping” than to provide a new mode of collegial support to scholarly authors, helping them in the development and revision of their work, as well as assisting readers in finding and engaging with high-quality material in which they have an interest.
2. Such a shift from gatekeeping to collegial support of necessity requires a divorce between the interests of peer review — facilitating the production of quality scholarly discourse amongst peers — and the institutional credentialing functions that peer review has come to serve. In other words, the simple fact of a project’s having been published with MediaCommons cannot be taken as evidence for purposes of promotion and tenure review.
3. For that reason, MediaCommons must provide the means for scholars to improve their work via conversation and revision, while also providing a set of metrics that those scholars can present, along with their work, to promotion and tenure committees and other individuals involved in such review processes. Such metrics, however, can never be assumed to stand on their own; institutions must take it upon themselves to find ways to review a candidate’s work for quality, rather than relying upon publishing outlets to make such important personnel decisions for them.
4. Each project published within MediaCommons will have a profile attached to it, tracking such web-native metrics as inbound links, citations, page views, comments, and the like. The network’s editors will also generate, upon request, a statement contextualizing this data, providing both a close reading of the project, via the comments generated during its open review process, and a “distant reading” (see Franco Moretti’s Graphs, Maps, and Trees) exploring the project’s profile within the network.
5. Participation in the open peer-to-peer review process being developed within the network is of primary importance to the life of the community, and thus while not all “peers” (or commenters) must be authors within the site, all authors must be peers, actively involved in the review of projects within their sphere of interest.
6. To ensure such active and engaged participation, MediaCommons will develop a system by which an author’s “reputation” within the network is determined not simply through review of their own projects but also through evaluation, by the community, of their participation within the community. Authors and peers will thus be given some means of rating comments as well as projects.
7. Further, because the open review process demands honesty and accountability in order to work, peers should commit to using their real names, rather than screen names, within the network. Member identity will be verified via the submission and authentication of an institutional email address. Any comments made within the network under a name other than that given by the member and verified during the registration process will be flagged as pseudonymous; such comments may be subject to filtering or some manner of ratings penalty.
8. Just as authors have the ability to select which Creative Commons license their work is published under, reserving to themselves those rights that are important to them, so authors should have the ability to decide what sort of evaluation their work should receive. The review options presented to authors should include the ability to name the particular communities to which the project is addressed, allowing the author to request review from the audiences best able to assess the quality of the work.
Great summation of our conversations, Kathleen! To add a few other points from my notes:
MediaCommons will offer a community space for projects in any stage of development. Any member of MediaCommons can get their own page to serve as a “greenhouse” to develop ideas and build a community of people engaged in the process of germinating projects. Through the use of technologies like tagging & trackbacks, and through the interpersonal networks facilitated by the editorial board, MediaCommons projects can be disseminated, promoted, and workshopped at any stage of the development process, not just in a “ready for publication” stage.
MediaCommons is committed to make the review process timely as well as transparent. Thus editors will work to solicit reviews quickly and encourage members to respond to new submissions in a timeframe more in keeping with the digital world, not the print model.
And to clarify point 8 above: essentially, authors can designate what peer communities a project is trying to reach, and editors will invite review from those communities. This request is open and transparent, so readers (including P&T committees) can see who authors are seeking to address, and how those communities respond.
Thanks Kathleen and Jason for these wonderful summaries.
To further clarify point #8, while authors can select the peer groups and types of feedback/review they would like for their work, any member of MediaCommons (membership in the community being a different designation from peer) can respond to any work available on the site. The peer selection mechanism is intended to help steer certain members toward a work and would also function as a filter (possibly by color coding comments from peers and members differently) to help the author determine how to engage with different respondants (i.e., if a piece receive substantial commenting, the author could request only to see comments from select peer groups to help manage the work flow. As a default, however, all comments from all members would appear upon first coming to the work.
Also, while the editorial board would use the author’s peer selections as a gage for soliciting particular reviewers, peer identification would also be part of the membership process for MediaCommons. In other words, upon joining the community, members would select the peer groups to which they wish to belong (peer groups might include professional credentialing and/or interest groups; being able to select “academic” might be tied to providing a university e-mail account or some such mechanism, but being able to identify as having an interest in “fan fiction” might be open to anyone regardless of professional credentials). In this manner, when an author selects the peer groups they would like feedback from, an e-mail would automatically be generated to all members who selected that particular peer grouping alerting them to the project.
Finally, while all members would have access to a “greenhouse” for germinating ideas pre-”ready for publication”, and these spaces would be networked through trackback and tagging options to facilitate dialogue between community members, these work spaces would also be filtered to the MediaCommons home page based on levels of activity, community interest, scholarly, pedagogical, or outreach value (again based on community voting) and by editorial board stewardship seeking to promote particular projects that have not gotten sufficient attention. In other words, while the Greenhouse is tucked away in the backyard, there will be mechanisms for continually providing members with snapshots of the polination process.
As a member of the general MC community (i.e., not on the board), I’m intrigued and greatly impressed by these General Principles. So it wasn’t all Wii Sports, then! Excellent work.
This idea of metrics is most intriguing, and I’m sure it was discussed a great deal in NJ, and will be discussed a great deal for a while to come. The key factor with all of it is transparency, so that whatever the modes and principles are, they will be made clear to all, at all times. This is critically important if one of the goals of the entire MC project is to change how scholarship is presented and assessed.
Accordingly, I think there needs to be a balance between various factors. The peer review process shouldn’t only function on one “channel” (as Jason and Avi discuss), but similarly it shouldn’t be too complex as well. It needs to be relatively easy not only to do, but to understand when presented to non MC-ers (cf. Kathleen’s pt. #4). While racking up “publications” isn’t the primary goal of MC, I think that tenure track scholars should certainly use their work here and elsewhere in their portfolios, and that senior faculty, deans, and T&P committees should understand how it all functions.
We’ve had a bit of this discussion at the Meadows School here at SMU already, and while our dean is actively engaged in incorporating new technologies and new modes of assessment, successful MC (or MC-ish) endeavors out there will help speed these modes to wider acceptance, here and everywhere.
Just want to express my enthusiasm and admiration for the work accomplished at the recent MC gathering.
The hand-wringing about “gate-keepers” that I see elsewhere in the discussion frankly makes no sense to me. It’s the kind of self-flagellation that makes academics unappealing. As long as we live in a fallen world of limited attention spans and finite human resources there will always be gate-keepers. To think the internet–cyberspace, the Web–makes it otherwise is not only naive, but reckless.
That doesn’t mean one can’t intervene in existing practices, and do so with greater or lesser efficacy. That doesn’t mean ideas can’t be improved. But at some point you have to start intervening. The above principles point the way forward.
Exactly - I agree.
Also - it might not be possible or practical not to do any gatekeeping considering our goals.