Agenda-setting, next phase
Kathleen Fitzpatrick - March 20th, 2007So as I wrote in my comment on Avi’s summary of the SCMS ed board meeting, I have two battling impulses for our meeting next week, one of which is to linger in the “what” questions — What do we want MediaCommons to become? What kinds of projects should we take on? What kinds of community can we imagine? — and the other of which is to press on through, perhaps a bit precipitously, to the “how” questions. These “how” questions are the things that I most want to get to, in no small part so that we have a concrete direction for getting started. They include the questions that Avi closed his post with –
How do we establish credentialing/legitimacy without replicating existing peer review gate keeping models?
How do we move from peer review to open peer-to-peer review?
How do we review multi-mediated scholarship?
How do we attract both community and contributors to MediaCommons?
– and extend to even more day-to-day questions: How will the editorial board function in the development of the network? How will the network’s backend be structured? How will we keep up with changing technological demands? How will we negotiate the balance between volunteer labor and work that must be compensated?
And so forth. What I’d like to ask for, at this point, is your sense of the questions that we haven’t asked yet, both the whats that we shouldn’t overlook in our rush to details, and the hows that I hope we’ll begin developing concrete answers to next week…
I agree with you, Kathleen. We need to be answering the “how” questions, if this project is ever going to significantly move forward. Having operating procedures in place will make MediaCommons both more managable and more innovative, since we will be able to confidently take on exciting projects knowing how to proceed in developing, credentialing, marketing, and “publishing” them.
I also think that how and what questions are deeply intertwined and should be addressed relationally. We cannot figure out what projects to develop until we know how MediaCommons operates, but we also cannot know how we operate until we know what our “products” and “processes” are.
I do think Tara’s suggestion of developing a set of protocols for recognizing and credentialing on-line scholarship is key to establishing legitimacy within the academy and also securing our fair use rights. I think how we do this is through a public wiki, where our editorial board members work together with other members of the community to write the document. Perhaps the first few bullet points will need to be decided in NJ to get the ball rolling.
Part of me wonders if what we need to do is just target a challenging project (either an individual or group one) that puts our editorial board to work in deciding how we evaluate, shape, guide, and legitimate the work. Something tangible to bite into might make these processes more apparent… perhaps this is putting the cart in front of the horse, but I feel like our horses have been dumped in the middle of a field without a map or compass to guide them.
I in turn absolutely agree (little shock, I suppose) with you here, including with your assessment of Tara’s statement of the need to establish legitimacy within the academy, but there is one point that I want to resist a bit. This comes less from your comment here than from your original post recounting the SCMS meeting: Tara is reported there to have asked whether we really wanted to publish bad writing, and to have pointed to academics’ (understandable) nervousness about the effects (on careers, on relationships) that commenting on such bad writing could produce. My resistance mostly comes in the form of pointing to this entry in the Nature peer-review debate, in which the editors of Atmospheric Chemistry and Physics present evidence that strongly suggests that, in fact, open peer-review results in a significant increase in the quality of submissions, precisely because “public peer review and interactive discussion deter authors from submitting low-quality manuscripts, and thus relieve editors and reviewers from spending too much time on deficient submissions.” I find this compelling, and in fact suspect that we’ll find the same thing — that blind peer-review often encourages writers to submit work that’s not really ready, and that a two-stage system of “publishing” followed by “refereeing” might result in better work…
Just to address the question of quality tied to open peer review, I agree with Kathleen’s argument here - making something public should work as a stimulus to generate quality, especially once the tenor of the comments establish a precedent of being serious, critical, and constructive, not just silent response or “nice job” style encouragement. So modeling behavior, both in encouraging productive submissions & offering commentary, should lead to increases in quality.
I think for many scholars there is an incentive to submitting to blind review journals to prove continued productivity to review committees & administrators - rejections are an ok by-product of a promiscuous submission process. But by making submissions & review public, the shame factor should outweigh CV padding.
Finally, I think it would be appropriate for a “first review” to be conducted by editors, as often happens with other journals - a work that is “not ready for primetime” can be returned for revisions until it meets a minimum threshold for posting online. If none of our editorial board is willing to stand by a submission, it should not be opened for public review - that’s also the way to ensure that our site doesn’t get bogged down by the equivalent of scholarly spam…
hmmm “shame factor” - this assumes that the public is a homogenous whole?
anyway - I look fwd to discussion tomorrow and day after
:)
r