Peer Review on the Web: Show me the evidence

dinaanselmi's picture

In the old days, circa the 1980’s, with the birth of Writing Across the Curriculum, peer review was seen as a mechanism to ratchet up the stakes for students, as they were working on their writing assignments. In this approach, faculty members could be assured that at the very least, they were not the first and only person to ever have read a student’s paper.  Even though Jason Jones thinks that may be optimistic, I am still holding firm to that hope.  Moreover, there was the belief, or at least the hope that by providing students with a different audience — their peers — they would see their work differently and take the process of writing and rewriting more seriously.  No longer would writing be seen as a purely independent process “owned” by the student to be “given” to the professor. When I would discuss the peer review process with some of my colleagues, they often expressed skepticism about whether this process could really improve student writing or whether it was a waste of time. My intuitions told me that student final drafts were better and my students often told me that the process helped them reformulate their ideas. Chris Hager’s students seemed to believe that.  But the question still loomed — is there “real” evidence of improvement? The good news for my doubting colleagues and also for myself is that there is now a significant body of research on the positive effects of the peer review process. And even better, many of these studies involve “real” experiments, where a researcher actually compares papers from one group, for example those having received peer reviews with a second group whose papers did not receive peer review.  Results from these types of studies provide solid evidence of student learning that move the discussion beyond the interesting but basically impressionistic accounts of individual teachers.

Fast forward to 2013. We are now taking the concept of peer review to a new level. With the advent of new technologies such as wikis and Google Docs, students can not only comment on each other's writing, but can easily provide feedback, review comments of others, and even collaborate with each other in writing drafts of assignments.  All of this has at once changed the landscape for our work as teachers but we have to again ask ourselves if these new technologies are leading to improvements in student writing and thinking.  Luckily, researchers are now providing us with some answers, using the same scientific method of comparison and control groups. One study by Ina Blau and Avmer Caspi (2009) illustrates this trend. They compared student writing in several different conditions to see if sharing work versus collaborating on work (by suggesting improvements or editing) would lead to more psychological ownership, responsibility and perceived quality of writing outcomes.  While they did find support for some of their hypotheses, most importantly that students in the collaborating groups had the highest perception of quality writing outcomes, the key point to emphasize is that their work demonstrates how one can begin to systematically research questions to help us know which technologies are most effective for different learning outcomes.  Clearly more research will be needed. We need to know, for example, if all students (e.g., novice vs. seasoned writers) will benefit equally from the same teaching technologies or how certain types of online writing might help or hinder different aspects of writing (e.g., grammar, organization, argumentation).  Knowing the answers to these types of questions might help us encourage even our most resistant colleagues into reimagining how they might teach with some of these new technologies. If we pay more attention to evidence-based research, it would make us better teachers and also help to enrich our students' learning.

What kind of evidence would persuade you to consider web-based peer review in your teaching? Share your comments here and also on our book-in-progress, Web Writing: Why & How for Liberal Arts Teaching & Learning.

Work cited:
Blau, I., & Caspi, A. (2009). Sharing and collaborating with Google Docs: The influence of psychological ownership, responsibility, and student's attitudes on outcome quality. In Proceedings of the E-Learn 2009 World Conference on E-Learning in Corporate, Government, Healthcare, & Higher Education (pp. 3329-3335). Vancouver, Canada. Chesapeake: AACE.

Image on front page by Nedral and available on Flickr. 


Jamie Henthorn's picture

In Praise of Googledocs

I am a huge fan of googledocs in the classroom, both as an instructor and as a student. I think that it gives students the chance to work on documents in a way they couldn't before. One of my favorite things about review and student use of googledocs is the ability for students to control what professors see to show their work, but they can also use the chat feature to have a conversation the professor  will not see. In January, Megan Mize wrote about the use of googledocs in a hybrid learning environment. She discusses a hybrid graduate program, but her article shows some of the strengths of review and collaboration in online writing. These simple chat features allow students to create the slightly social collaborative groups that many scholars use and to get used to writing collaboratively early on.