Undergraduate Essay Writing : Online and Face-to-Face Peer Reviews

P eer assessment or peer review can be a powerful pedagogical tool that develops students’ critical and creative thinking skills. Since it involves the joint construction of knowledge through discourse, it could be argued that peer assessment has its philosophical foundations in active learning (Piaget, 1971) and social construction (Vygotsky, 1962). Many students have reported that their learning has improved after reviewing and assessing each others’ work. Topping (1998) conducted a review of the literature and identified that peer assessment yields cognitive benefits in multiple ways: constructive reflection, increased time on task, attention to crucial elements of quality work, and greater sense of responsibility. Interestingly, these benefits were found for both the students who conducted the reviews and the students who received peer-reviewed comments. Other reviews have provided support for the close resemblance of peer assessments to instructor assessments when judgement criteria are well-articulated and understood by the assessors (Falchikov & Goldfinch, 2000). 12


Introduction
P eer assessment or peer review can be a powerful pedagogical tool that develops students' critical and creative thinking skills.Since it involves the joint construction of knowledge through discourse, it could be argued that peer assessment has its philosophical foundations in active learning (Piaget, 1971) and social construction (Vygotsky, 1962).Many students have reported that their learning has improved after reviewing and assessing each others' work.Topping (1998) conducted a review of the literature and identified that peer assessment yields cognitive benefits in multiple ways: constructive reflection, increased time on task, attention to crucial elements of quality work, and greater sense of responsibility.Interestingly, these benefits were found for both the students who conducted the reviews and the students who received peer-reviewed comments.Other reviews have provided support for the close resemblance of peer assessments to instructor assessments when judgement criteria are well-articulated and understood by the assessors (Falchikov & Goldfinch, 2000).

12
Undergraduate Essay Writing: Online and Face-to-Face Peer Reviews We implemented two different approaches of using peer review to support undergraduate essay assignments for students taking large second-year courses in life sciences and biology: a web-based online peer review (OPR) approach and a more traditional face-to-face peer review (FPR) approach that was conducted in tutorial settings.The essays consisted of a review of current literature to discuss the molecular involvement of cancer development or stem-cell growth.Following implementation of the peer reviews, we conducted a preliminary analysis of the pros and cons of using the two methods.Student and instructor feedback suggested that the activity of peer review was generally perceived as valuable regardless of which approach was used.OPR was convenient and saved time and resources relative to FPR, but the technical drawbacks using the OPR approach made it challenging for some students to use.A subsequent investigation using alternative OPR programs that offer additional functionality is planned.
With the size of university classes growing, peer review in a face-to-face setting becomes a larger challenge to facilitate, fund, and coordinate.As such, online peer review (OPR) is emerging as a popular alternative to the classic format, face-to-face peer review (FPR).OPR has been well-established as an effective language learning tool in English as a second language (ESL) classes (DiGiovanni & Nagaswami, 2001;Moloudi, 2011) and poses some advantages over the conventional FPR format.For example, OPR allows the implementation of peer review sessions without physical and time constraints (Rollinson, 2005), and longer peer review sessions may lead to higher quality peer reviews.Less explored is the application of OPR to university-level essay writing for science courses despite OPR becoming more prevalent in university writing classes (Liu & Sadler, 2003).
The purpose of this essay is to share how OPR and FPR methods were used in large secondyear science classes and the preliminary results about the pros and cons of using either approach.The two approaches included the Calibrated Peer Review TM (CPR, 2001) a program freely available from the University of California at Los Angelos (UCLA), and a traditional FPR approach conducted in tutorial settings, facilitated by teaching assistants (TAs) and coordinated by an instructional assistant.Unlike most previous studies which have focused on using OPR and FPR to develop rudimentary language skills in ESL classes, we hope to elucidate whether the application of these two modes of peer review can be extended to the university level to enhance essay writing, specifically in the sciences.This report is intended to be a preliminary comparison of OPR and FPR that precedes a more direct comparison in the future.

Implementation of Face-to-Face and Calibrated Peer Review
Just over 500 students participated in FPR as a component of their science essays for a second-year research methodologies course in the fall of 2010.
Students were asked to write a literature review on the molecular environment that supports stem cell growth.Peer review was facilitated by TAs during two regularly scheduled tutorials.These peer review tutorials were integrated into the essay assignment as writing "checkpoints" to monitor student progress at two crucial stages in the development of the essay: 1) a preliminary abstract; and 2) a rough draft of the essay.
For the first checkpoint, we wanted students to comment on the clarity of their peer's writing opposed to critique the content of the essay, so we prefaced the peer review session with an hour-long writing workshop.The remaining hour was devoted to the actual peer review activity.Students were assigned to groups of three by the TA, and each student reviewed two hard-copy abstracts that were between 150 and 300 words in length.Provided with a checklist to guide them through each peer review, students were asked to read, review, edit, and make comments on each abstract.After each review, peer dyads engaged in a 5-minute follow-up discussion to clarify any unclear comments.Finally, students were asked to fill out evaluation forms for both peer reviewers to ensure that students contributed constructive feedback.
The second checkpoint required students to have completed a rough draft of their essay with a maximum of 2000 words.The format of the second checkpoint was essentially the same as the first checkpoint except that in lieu of the initial workshop hour, more time was allocated for reading and reviewing the longer drafts.
The web-based Calibrated Peer Review TM (CPR, 2001) allowed us to try a double-blind OPR approach with a second-year cell biology essay assignment in the fall of 2009.Students were asked to write a literature review on the molecular involvement of cancer development.Three hundred and fortyeight students participated in both checkpoints for this assignment.The first two checkpoints required students to submit short early drafts: a 350 word outline for the first checkpoint and a 500 word draft for the second checkpoint.After submitting their own drafts, students then evaluated low, mediocre, and excellent exemplar essays that the instructor included.Once this was completely satisfactorily, the program randomly and blindly distributed the written submissions such that each student received three of his or her peers' draft essays to review.A checklist and guidelines were provided to assist the peer reviewers.Subsequently, students self-evaluated their own submission using the same criteria that were used for the exemplars and their peers' submissions.Once the deadline passed for this review period, the program revealed the results of the reviews along with all of the comments that were provided by the reviewers.

Discussion and Future Avenues of Research
Overall, the use of peer review to improve science essay writing was positively perceived by most students, regardless of the format.This study was not meant to be a formal evaluation and comparison of the two approaches, so in this discussion, only anecdotal results are reported.When the classes completed anonymous course evaluations, over 90% of FPR students said that peer reviews were useful for the construction of their final essays.Likewise, the large majority of OPR students said that receiving feedback was helpful in improving their essay writing.Students were asked during lectures about their experiences using the OPR and through a show of hands, the online anonymous survey results replicated general trends that were reported by students in lecture.It seemed that more OPR students believed that it was useful to evaluate other peers' work than FPR students.About half of the students participating in either OPR or FPR agreed that the perceived optimal number of peer review cycle was two or three sessions.Fewer suggested that one peer review cycle was optimal, while some requested a fourth optional review of their final draft before submitting their essay.
Although our student feedback suggests that FPR was beneficial, it was also time-consuming.Time constraints may be the biggest barrier in implementing FPR, especially in large courses (Rollinson, 2005).In our own FPR tutorials, TAs expended four hours of tutorial time and further time grading the reviewer evaluations.Marks for reviewer evaluations had to be manually entered into an excel file.OPR appears to be an attractive alternative to implementing peer reviewed assignments when time or space is limited.OPR was conducted outside of scheduled course hours and four TA hours were saved per TA.Additionally, peer reviewer marks were calculated automatically by the Calibrated Peer Review TM program, which reduced the total marking time to the duration of the final grade entry.
OPR also permitted longer peer review sessions than FPR.For each checkpoint, FPR students had 20 and 40 minutes for the first and second peer reviews respectively (including time for follow-up), whereas OPR students had a span of five days to complete their peer reviews with no time limit on how long they spent completing the review.Some peer review studies found that the number of comments made in OPR were more constructive and detailed than its FPR counterpart (Liu & Sadler, 2003;Warschauer, 1996;Xu, 2007), which could be attributed to OPR's longer duration.Whether this is because people are more inclined to comment or because they have more time to comment in the OPR format remains unresolved.
One factor that may contribute to students' willingness to comment is the double-blind aspect of the OPR approach.We believe that the anonymity of the peer reviewer may permit more honest peer reviews and thus perhaps more constructive feedback.If the peer reviewer knew that his or her identity was known to the peer, as in FPR, peer reviewers may be more reluctant to voice criticisms even when they may be warranted.Comments from DiGiovanni and Nagaswami's (2001) ESL peer review study echo this notion: "sometimes we are not "truth[ful] with our partners in person and with online [peer review] you can say more things" (p.268).Double-blindness is feasibly implemented in OPR but not in FPR, and therefore, anonymity may be an important advantage of OPR.
Despite the aforementioned benefits of OPR, the Calibrated Peer Review TM program was not without its limitations.The text entry stage of this particular program was not user-friendly and required students to encode their submissions using HTML coding.Even though students were provided with an HTML template, some students complained that they had technical problems and consequently submitted their outlines late.This issue could be easily resolved if the text entry stage had a userfriendly interface.Some students also indicated that they had difficulties logging into the website, which required them to remember their assigned username and chosen password.Notwithstanding, only a few (less than 20%) students felt that the Calibrated Peer Review TM program was difficult to use, implying that these were minor setbacks.From the instructor standpoint, a large, initial time investment was involved in designing and setting up the online program.Instructors had to write the assignment instructions, find and input a low, mediocre, and excellent exemplar, design the reviewer evaluation form (forced-choice, rating, open questions), score the exemplars according to the evaluation, and set the dates for each phase.
Another limitation of the Calibrated Peer Review TM program was that there was no messaging system to enable computerized conversation among peer dyads.Without a messaging or chat system, peer reviews were limited to single exchanges, consisting of the peer reviewer commenting on a student's written submission with no reciprocity from the peer thereafter.Other OPR programs, such as Norton Textra Connect (Raschio, 1997), have one-to-one messaging systems that allow studentstudent discussion.Thus, this is a limitation of the system we used and not generally to OPR.Without two-way dialogue, the writer does not have an opportunity to defend his or her grammatical and syntactical choices, and likewise, the reviewer does not have an opportunity to justify his own comments and suggestions.Two-way collaborative dialogue is inherent to FPR, and students freely engage in self-explanation and elaboration.Perhaps less OPR students than FPR students felt that the feedback they received was useful because the Calibrated Peer Review TM program did not include an opportunity for conversation and clarification.Wooley (2007) corroborates this reasoning as he claims that students benefit cognitively by explaining themselves to their peers.Accordingly, OPR students would not experience the same cognitive benefits as FPR students.Furthermore, many peer review studies confirm that having dialogues about writing and exchanging comments about one's writing is perceived as a constructive activity (Nystrand & Brandt, 1989;Schultz, 2000;Spear, 1988;Zhu, 1994).
Based on our preliminary analysis, we suspect that dialogue provides more constructive feedback during peer review for university-level essay writing.We feel that it would be worthwhile to contrast the effects of two-way conversation with one-way feedback on peer review feedback.To make a fair comparison of monological and dialogical discussion, we should eliminate any confounding variables by maintaining the two peer review formats as similar as possible, leaving the type of discourse as the lone variable.For example, using Norton Textra Connect (Raschio, 1997), which has a chat system, and the Calibrated Peer Review TM program (which does not have a chat system) for our comparison could be appropriate.Despite the notion that feedback quality may be improved through two-way conversation, we must also consider that peer reviews would have to be conducted synchronously with peer review and peer reviewer online simultaneously, possibly raising new logistical issues for large classes.Thus, even if the dialogical and conversational approach generates better feedback, we wonder whether it would warrant devoting additional time and resources.Or would an inexpensive, flexible system conferring satisfactory peer review feedback be sufficient?Ultimately, it is a dilemma of conversation or convenience.

Conclusion
This preliminary report provides an anecdotal comparison of OPR and FPR for use in undergraduate science essay writing.While a slightly greater proportion of students found peer review useful in the FPR format than the OPR format, we cannot conclude that one is better than the other.For one, we gathered feedback on the two approaches from two different courses that had similar but distinct essay assignments.In the OPR format, students may provide more honest criticisms because they are safeguarded by a veil of anonymity, but feedback may be limited by a lack of dialogue.Regardless of the peer review format, students reported that the peer review process was valuable and wanted it to be included in other essay assignments.They suggested that two or three peer reviews cycles were optimal to constructing their essays.In our experience, OPR represents a convenient and effective alternative to peer review despite some minor technical issues within the program.Future avenues of research may include conducting a controlled comparison of FPR and OPR (e.g., same students enrolled in one course exposed to both types of peer review) and exploring the facets of monological versus dialogical discussion.