Going Blended with a Triple-Entry Activity : Students ’ Online Discussions of Assigned Readings using Marginalia

In this paper, we describe and investigate small group discussions of assigned readings in an online version of a “triple-entry activity” in a blended course used an annotation tool, Marginalia. We wondered if students would interact in this structured, critical, reflective reading activity as effectively online as they had when the activity was undertaken on paper in face-to-face classes. We investigated what happened, why, and if successful, and how these findings might inform the use of annotated discussions in the future. We found 30% of comments acknowledged the value of ideas expressed in a group member’s response to a reading, 30% extended those ideas, 11% connected the reading to personal experience, 9% were questions, and 6% answers. Approximately 60% of the interactions were between one group member and the author of the response; 40% involved comments that were connected to each other as well as the author’s response to the reading. Students felt using Marginalia to comment on classmates’ responses and having classmates comment on their responses facilitated their learning from assigned readings. The instructor agreed and felt the online discussions also contributed to the development of a community of learners between face-to-face classes. In addition, reading students’ responses and discussions before each class informed the instructor’s preparation for in-class activities.


Introduction
fter teaching small Education courses (maximum 24 students) face-to-face for 20 years with consistent positive evaluations, the first author, Dr. Kanevsky, wanted to offer one of them in a blended format, i.e., "through a combination of online and face-to-face experiences" (Means, Toyama, Murphy, & Baki, 2013, p.6).Although the face-to-face course filled well and the evaluations were strong, she wanted to make the course more accessible to potential students who lived too far from the university campus to arrive in time for evening classes that began at 4:30 pm.She resisted shifting to a blended format until online tools could be found that enabled her students to have a core course experience, the "triple-entry activity", and its benefits (Kooy & Kanevsky, 1996).
Here we describe the triple-entry activity, first offline and as it was adapted for use in a blended course, how it was implemented, and its outcomes.
Here the word "entry" means a written contribution to a conversation about an assigned reading.In the face-to-face version of the triple-entry activity, the first "entry" was a student's written response to an assigned reading.The second "entry" was in the form of comments written in the margins by a few classmates when they exchanged and read each other's responses in class.Rich, lively small group and whole class discussions of their readings ensued.Students' conversations began in the comments they made in the margins and went deeper when they spoke.Students were creating zones of proximal development (Vygtosky, 1978), social spaces in which they surfaced, examined, deconstructed and reconstructed their knowledge.
Students submitted their marked-up responses to the instructor for her feedback at the end of the class session.Her feedback was the third "entry".This process not only engaged students critically and reflectively with the course content, it played a fundamental role in developing the learning community.These interactions were what Dr. Kanevsky wanted to preserve in an online discussion format when she "went blended."

Going Blended
As previously mentioned, a major reason for shifting the delivery format from face-to-face to blended was to increase access for students who would not be able to get to campus to attend a weekly evening class.It would also enable students to control when and where they learned between class meetings so it accommodated students' varied backgrounds and busy lifestyles.This flexibility was particularly important for students for whom English was an additional language as it gave them additional time to make sense of the readings and find words to express their thinking.
Interacting with classmates was an essential feature of the instructor's social constructivist views of learning and the course.The asynchronous nature of the online component of discussions of readings in the triple-entry activity removed the time pressures that constrained students' reflection and discourse in synchronous settings (Garrison & Vaughan, 2008).While face-to-face in class discussion is fast, spontaneous, and requires confidence, an online discussion is qualitatively different.Online discussions can increase opportunities for reflection.Due to their asynchronous nature, reflection is an intrinsic element of an online discussion.There is no expectation of a quick reply; there is time to think carefully about the response, to edit and proofread before publishing it.Using a blended format that allows for an online social exchange enables all students to actively participate in the academic discourse.In this course, the exchanges began online in the triple-entry activity and were extended in the class meetings.
To implement the triple-entry activity online, Dr. Kanevsky needed to find a suitable technology that would enable students to comment on each other's writing online as comfortably as they had faceto-face.As it turned out, the solution was readily available.A web-based annotation tool, Marginalia, had already been designed and developed by colleagues (including the second author) at Simon Fraser University.It enables students to make comments in the margin beside ideas of interest in text posted in a discussion forum.In fact, the comments in the margins are the actual meaning of the word marginalia.This practice is an ancient tradition having been practiced by readers for hundreds of years (Howard, 2005).It is also a familiar study habit for most students.With this technology and support in place, Dr. Kanevsky felt she could move the triple-entry activity online.
In the following sections we describe the nature and process of the triple-entry activity and provide an account of our online implementation of it using Marginalia.We then report the results of a preliminary investigation of our implementation, and Dr. Kanevsky's reflections on her experience and observations of student learning.

The Triple-Entry Activity
The triple-entry activity is a five-step process that engages individual students with the content of assigned readings prior to collaborative exploration and extension of their meaning and implications in class.In face-to-face courses, before each weekly class, students read an article and prepare a structured written response to it.Their three to four-page responses began with a four to 10 sentence objective summary of the reading.This was followed by six student-selected quotes from the article, each with a subjective reflection describing its personal or professional significance.Requiring six quotes enabled students to address a range of topics in each reading and kept the workload manageable.The guidelines for the activity specified each reflection should describe why and how the quote provoked them (see Kooy & Kanevsky, 1996 for a detailed description).The response concluded with a lingering question of personal or professional interest that was related to the reading, also with reflection, explaining its significance.
The second step took place in class when students exchanged responses in groups of three or four.As they read other's responses, they were required to make substantive comments in the margins, asking for clarification, or describing why a passage had caught their attention.In the third step, students talked, continuing the conversation that had already begun in the margins of their responses.This fed into an instructor-designed follow-up activity (Step 4) to draw closure to the discussion prior to submitting their response to the instructor.She read students' marked up entries, adding her comments and feedback to those already in the margins (Step 5), and returned them to the students the following week.
This process introduced students to new material and surfaced their initial understandings and misunderstandings of course content, confusions, and questions (Hughes, Kooy & Kanevsky, 1997;Kooy & Kanevsky, 1996).Students talked more and the instructor lectured less.They learned "actively, constructively and interactively" (Chi, 2009), coconstructing clearer, more complex understandings of concepts, theories, issues, and implications.They reported it was a demanding, yet fun process.Dr. Kanevsky heard and pursued their interests with them, aligning them with the learning outcomes of the course.A strong sense of community developed during the semester as students exchanged responses with different classmates each week.
Dr. Kanevsky had been using and refining this activity since she was first introduced to it by a colleague, Dr. Mary Kooy, in the early 1990s when she was searching for a way to increase students' active, collaborative engagement with course readings.Both instructors were committed to developing pedagogies based on sociocultural and constructivist principles of learning.Implementing the triple-entry activity, with Dr. Kooy's support, transformed the course, her role and her students' learning in powerful ways that enhanced their theoretical integrity.Dr. Kanevsky was thrilled, and they documented the process and outcomes in a traditional course format in their 1996 publication (Kooy & Kanevsky, 1996).

Marginalia and the Online Implementation of the Triple-Entry Activity
Marginalia is a Web-based, open source annotation software application specifically designed to encourage active reading and provide feedback (Glass, 2005; see webmarginalia.net).The application is integrated into the Moodle discussion forum.Users can easily select and highlight text in a forum post and comment on it, i.e., annotate it.These brief notes or comments of no more than 250 characters appear in the right margin of the forum page.
An earlier study of Marginalia showed that forum participants used it to record thoughts, acknowledge each other's contributions, and most often to discuss ideas beside text highlighted in forum posts (Xin, Glass, Feenberg, Bures & Abrami, 2011).This enabled them to contrast ideas, develop common ground, and create "weaving messages" that connected thoughts, thematized contributions, and advanced discussions.
With Marginalia in place, Dr. Kanevsky was ready to move the triple-entry activity online in a first attempt to "blend" a small Education course addressing the "nature and nurture" (psychology and education) of gifted individuals.The blended version met face-to-face four times (one six-hour Saturday session in each of the four months in the semester).Between the class meetings, students engaged in an online asynchronous version of the triple-entry activity which "primed the pump" for further clarification, exploration, extension, application and critique during the face-to-face classes.
In the first face-to-face class meeting of the semester, the process of preparing, posting and discussing responses to assigned readings in the Moodle discussion forum was introduced and practiced in an abbreviated form with a brief, two-

Figure 1
An excerpt from an online triple-entry activity using the Marginalia annotation tool page reading.This activity gave students an opportunity to become familiar with the course website, the discussion forum, and Marginalia.After debriefing this process, extensive structured guidelines for the content and format of a full response and online discussion were distributed.These guidelines are available from the first author upon request.Each student was expected to post a response with all of the required elements (summary, six quotes, and a lingering question with reflections) to each assigned reading in an online forum for discussion by members of their group (three or four students per group).Students repeated the tripleentry activity for each of the six readings (in total) assigned between the four monthly, full-day class meetings.
Figure 1 provides a screenshot of an excerpt from a students' response to a reading in which a few key elements of a response are labeled.The excerpt begins with a quote chosen by the student and is followed by her reflection on it in italics.The shaded text, which appeared highlighted in yellow on the screen, was selected by the students who posted comments (annotations) that appear in the margin to the right of the quotes and reflections.The comments were made using Marginalia.
Unlike the face-to-face version, the online version of the triple-entry activity enabled Dr. Kanevsky to read students' discussions (responses and comments) prior to class rather than after.She offered feedback on their format and completeness but did not participate in the conversation.
A lesson learned in the first year of going online with the triple-entry activity was related to the time needed to discuss a response in an asynchronous context.If students posted their response on the Friday before a Saturday class, members of their group had little time to comment.This resulted in moving the deadline for students to post their response to Wednesdays to allow more time for the discussion, i.e., for group members to comment.

Our Investigation
As instructors, educational psychologists, and for Dr. Xin, as one of the designers of Marginalia, these authors had many questions about student's learning and experience with the triple-entry activity online.Was it successful?
Did it enhance students' understandings, and teaching practice, and strengthen the learning community between class meetings as had been found using the face-to-face version?What worked well and what needed to be revised?We wanted evidence-based answers to these questions and more.In the upcoming pages we report quantitative and qualitative data that addresses the following questions: 1. Did the number and nature of highlights selected and students' comments indicate students were discussing the content of the readings and engaging with each other in valuable ways? 2. What types of comments did students make with Marginalia? 3. What were students' perceptions of the triple-entry activity using Marginalia? 4. Overall, did Dr. Kanevsky feel the tripleentry activity online was successful?
All 17 students enrolled in the course (16 females, one male) were teachers who differed in age, academic background and professional experience (from 0 to 25 years).Fifteen had participated in online discussions in previous coursework and seven had worked with Moodle before.All but one felt they could independently solve computer-related (technical) problems.A majority reported being online at least 10 hours per week for academic tasks and other reasons.
Our Results

Number and nature of highlights and comments
To measure the volume of participation in the forum discussions, we counted the number of highlights (words, phrases or passages selected in a peer's response to a reading) and comments made using the Marginalia tool.Over the course of the semester, students selected a total of 1209 highlights and made an equal number of comments corresponding to the highlights.Due to limited resources, we focused our analysis on the triple-entry activities students completed for the first, third and fifth (out of a total six) of readings.They were assigned at the beginning, middle, and end of the semester.In them, students respectively selected 202, 188, and 175 highlights and made comments for each highlight.In total, 565 highlights/comments pairs were made.
Most of the interactions involved only two students, the author of the response and the individual making the comment (see Figure 2).The We also examined how students interacted in the margins.We found two common patterns, which we named "hub-and-spoke" and "chain."As shown in Figure 3, in a hub-and-spoke interaction, two students selected the same chunk of text and each made a comment on it that was directed only to the author of the response; there was no connection made between the ideas in the comments.This was just a note to the author, not a real conversation because it did not evoke a response.In a chain interaction, comments were made by two or more group members and ideas in students' comments were connected with each other.In Figure 4, this is represented by the vertical arrow between Students E and F and was more of a conversation.
As can be seen in Table 1, the majority of the interactions were "hub and spoke," i.e., comments were directed solely to the author of the response.It Figure 3 Hub-and-spoke interaction appears group members were simply annotating the author's response without connecting to others so they were not actually discussing it, just commenting.
Approximately 22-32% of the interactions involved chains of comments that represented more active engagement with peers in true discussions.The lower frequency was likely due to the instructions in the guidelines, i.e., they did not direct students to extend interactions initiated by others.
To engage students in collaborative discourse in an online forum, Xin and Feenberg (2007) suggested instructors moderate discussions in specific ways (e.g., prompting, referring, recognition, weaving, delegating).Similarly, Brookfield and Preskill (2005) also encourage assigning participants specific roles (e.g., questioner, scrounger, appreciator, theme spotter, devil's advocate) to achieve sustained and meaningful discussion.The use of moderating functions and role assignment are techniques we are considering using in future implementations of the blended versions of triple-entry activities.

Types of comments made with Marginalia
Each highlight and comment was coded according to its role in the interaction.Our coding scheme was based on the 12 types of "conversational moves" described by Brookfield and Preskill (2005, p. 99-100).They originally intended the conversational moves to prompt different types of contributions in face-to-face discussions.We adapted them and

Table 1
Percentages of each type of interaction (hub-and-spoke or chain) and the number of students who made comments.

Type # of students commenting
Reading created additional codes to capture the nature of the "moves" our students made in the online discussions of the assigned readings (see Appendix).Many highlights and comments received more than one code as they were lengthy and served more than one purpose.The third author coded the data and occasionally consulted with the other two authors to clarify the meaning of the codes.We achieved 85.4% initial inter-rater agreement between the codes assigned by the first and third authors on data from two discussions randomly selected from each of the differences between codes.
A total of 957 conversation move codes were assigned to the comments made by students in their triple-entry discussions of the three readings.As can be seen in Figure 5, the greatest proportion of students' comments acknowledged the value of their classmates' ideas (30.9%) and extended them by adding new information to that provided in the reading (30.5%).A further 11% involved making personal connections by relating the reading to their professional and/or personal experiences.Students' efforts to help each other clarify the meaning of a

Figure 5
Percentages of each type of conversation move in students' comments in the discussions of the assigned readings.
response and/or the content of the reading appeared as they questioned (9.3%) and answered (6%) each other.
These findings are consistent with those of Xin et al's.( 2011) investigation of the ways students created shared understanding of each other's position on a topic by making comments in the margin.Xin et al. (2011) called this "common ground."By finding common ground, students enhanced each other's understanding of the topic.It was evident that students actively participated in a sense-making process by actively choosing quotes to present to classmates, specifying the personal or professional significance of each, by annotating and reading their classmates annotations of their own work.Marginalia provided students with the opportunity to do so in a blended course format.This process is consistent with a constructivist theoretical framework in which the learner has to actively participate in the learning activity (Sfard, 1998).

Students' perceptions of the tripleentry activity using Marginalia
Six of the 16 students responded to a survey of their perceptions of the online triple-entry activity using Marginalia.All felt using Marginalia to comment on classmates' responses and having classmates comment on their responses facilitated their learning from the assigned readings, although they thought making comments was more helpful than having others comment on their responses.All but one of the six provided further evidence they had enjoyed the process and felt it played a significant positive role in their learning.Benefits they mentioned included being able to ask other students clarifying questions about the reading and discuss ideas with them, that it enabled them to notice connections among ideas mentioned in classmates responses and their own, and that the process of commenting slowed them down so they focused more attention on what they were reading.It seems they appreciated the opportunity to establish the common ground mentioned by Xin et al. (2011).
Students' concerns focused on the work required to prepare their response to each reading.Three respondents indicated it was "a lot of work" and/or "time consuming."Two disliked the highly structured format and length limits for responses and comments, however, they came to understand this was necessary to manage the amount of time required by group members to read and comment on each response.One had mentioned she preferred face-toface courses numerous times throughout the term.This was a helpful reminder that online learning is not appealing to all students.
When asked how the process might be improved, students' suggestions focused only on the content of the response, not other aspects of the triple-entry activity (e.g., the discussion).They mentioned reducing the number of quotations (with reflections) and requiring summaries only for responses when there was a choice of readings.All of the students' feedback was considered when revising the activity for subsequent cohorts.Because only six of the 16 students filled the end of the term survey, their responses cannot be considered representative of the whole class.They do however indicate aspects of the triple-entry activity that impacted future efforts to improve students' experiences and their learning.

Was the online triple-entry activity successful?
Like the students, Dr. Kanevsky was pleased with the outcomes of taking the triple-entry activity online using Marginalia, and, of course, she also saw room for improvement.Her reflections on students' use of Marginalia, participation, interactions, learning community, management, the blended format, and plans for the future are summarized below.

Students' use of Marginalia
In conversations throughout the term, many students mentioned the Marginalia annotation tool made online discussions easier, more effective and fun because it allowed them to select and highlight the specific passage of interest to them in a peer's post and their comment appeared in close proximity to it in the margin.This was a contrast to the frustration they experienced when using a traditional "reply" format in which their comment appeared beneath their classmate's entire response.They found Marginalia friendly and familiar because it was similar to the comment function in their word processors.As others have found, these students made it clear they remembered, clarified, and shared their understandings while using Marginalia (Ovsiannikov, Arbib & McNeil, 1999).Two students recommended it to instructors in other courses and two others began using it with their students before the end of the semester.Dr. Kanevsky felt students' use of Marginalia surpassed her expectations and she looked forward to continuing to use it in this and other courses.

Nature and level of students' participation
Most students consistently fulfilled the requirements of the triple-entry activity online (posting their response and comments).Occasionally, as with the face-to-face version, a few needed prompting to provide a missing quote, reflection or comment.The content and format of their responses to the readings were very similar to those prepared by students in face-to-face course offerings.Students made fewer comments in the online version of the triple-entry, likely because the guidelines for the activity specified they were to make "three or more comments on each classmate's response."We suspect that students often made the minimum number of comments because they were preoccupied with the professional turmoil they were experiencing due to a province-wide labour dispute and strike.Although their online comments may have been fewer in number, they were noticeably longer and richer.Marginalia limits a comment to 250 characters and some students made comments that spanned two comment windows.Dr. Kanevsky had never seen such lengthy comments on printed triple-entry responses.This may be because Marginalia made it easy-it adjusted the spacing of the comments around the text to accommodate multiple comments.This was not possible when responses were shared on paper.

Interactions in the margins
Dr. Kanevsky was slightly disappointed by the reduced intensity of students' interactions online, as there seemed to be fewer chained interactions in the margins than when students' hand-wrote comments on their classmates' printed responses.As was mentioned earlier, the large majority of interactions were hub-and-spoke validations or extensions of a classmate's ideas.She has addressed this directly in a subsequent course offering by encouraging students to attempt to engage in "table talk" in the margins.That prompting increased the proportion of true conversations online.
During in-class follow-up activities, it was apparent the outcomes of students' online interactions between classes had prepared them well because they were able to apply, critique, and extend their understandings beyond the content of the article, co-constructing new knowledge, and often taking a critical stance regarding its implications for their practice.

Development of a learning community
A key feature of the face-to-face course with the tripleentry activity that Dr. Kanevsky was determined to preserve was the strength of the learning community.She knew the weekly exchanges and conversations it inspired played a significant role in its strength and she feared it might suffer over the weeks between class meetings when the course was offered in a blended format.Her fear gradually dissipated with each class meeting as she saw students approach each other before beginning each face-to-face session, eager to continue the conversation they had started online.Clearly, they had connected in ways that established and maintained the learning community.
The interdependence of members of the community was essential to their learning and all students committed to it.They honoured the deadlines for posting their response to a reading knowing that if they ignored the deadline, it reduced or eliminated the time members of their group would have to read and comment on it.And if a group member neglected to comment on a response, the opportunity to learn from that group member's perspective was lost.Fortunately, these situations seldom arose as this interdependence was made explicit in the guidelines and reinforced in each class.
Students' online interactions also contributed to the development of community between class meetings.As mentioned earlier, more than 60% of the conversation moves in our data involved two behaviours that demonstrate two fundamental characteristics of learners in a community: acknowledging the value of others' ideas and extending the knowledge base by sharing what they knew.As the class was small and the composition of the groups changed across activities, students gradually came to know all members of the class.

Student control
In addition to reducing the students' trips to campus, the blended format also gave them greater control over when they would learn as previous offerings required weekly class attendance at times that often clashed with after-school staff meetings and personal and family commitments.

Assessment for learning
Unlike the triple-entry activity in print (offline), the online version served as a rich pre-assessment because Dr. Kanevsky was able to watch and hear students' thinking with and about course content prior to a class meeting.This had not been possible in the offline version because in that version she was only able to read the responses and comments after the class in which they were exchanged and discussed.Of all of the benefits of taking the triple-entry activity online, this was the greatest for her, as it enabled her to plan in-class and future activities that responded to the baseline evidence provided by her students in their responses and comments.

Making thinking visible
The online triple-entry process continued to provide Dr. Kanevsky with invaluable insights on students' beliefs, knowledge, interests, concerns, and experiences.All informed her efforts to prepare relevant follow-up in class activities in which misconceptions and questions surfaced and were addressed collaboratively in the process of coconstructing more sophisticated understandings.This strategy is what Simkins and Maier (2010) called "just-in-time teaching."For example, while reading the triple-entry activities online before a class, Dr. Kanevsky saw potential connections among key concepts and their teaching practice that the students had not, and planned in-class activities that enabled students to also see them.

Frequency of meetings
One of the major challenges in the transition to blended teaching is obvious-meeting less frequently.There are a number of informal communications that were reduced, such as clarifying assignments.Instructors and students need to find ways to continue the informal communications that can take place before and after face-to-face sessions, and during breaks.Initially, this issue resulted in numerous, similar email conversations with different students.This was addressed by creating a forum for questions about the course assignments and resources so all students would have access to the same information and might find answers to their questions before asking.

Future considerations
Based on Dr. Kanevsky's experiences and feedback from the students, a number of changes have been made to the guidelines to make the instructions and expectations clearer and reduce the workload for students.As previously mentioned, students are now explicitly encouraged to chain their comments by connecting them to those made by other students.And the number of highlights and reflections has been reduced from six to four.Every year, students appear to be more comfortable with the technology and interacting online, which has reduced the perceived workload for many students.

Implications
Due to the small sample size and other limitations, we intend this work to be descriptive and not to be generalized.We realize the voluntary survey data is vulnerable to common threats to the validity of selfreport data (e.g., self-selection bias, social desirability).We look forward to addressing questions regarding the impact of increased instructor participation in the online discussions in subsequent investigations and course offerings.How will this change the type, nature, and intensity of the interactions or the formation of the learning community?We also wonder if the topic or nature of the reading impacts the conversations in the margins.

Conclusion
We are encouraged by the successful implementation of the triple-entry activity online and its role in the blended version of the course.This preliminary study demonstrated Marginalia enabled students to effectively engage in that activity in meaningful ways that enhanced their understandings of the readings and the strength of their learning community.In addition, the online format afforded the instructor with insights regarding students' baseline understandings that informed her plans for in-class extension activities.We look forward to improving and continuing to investigate the impact of annotations on learning in the online triple-entry process with larger groups and greater instructor participation in future course offerings.
Many of our lingering questions address not only the Marginalia tool, but also the role and place of the triple-entry activity in students' learning throughout the course.If we seek ways to increase the intensity of interaction, how will this affect students' learning or enjoyment?Perhaps this will depend on students' familiarity or comfort with the topic of the reading.Perhaps the nature of the topic and style of the reading (e.g., textbook chapter, provocative opinion piece, research study, etc.) influence the nature and intensity of the online discussion.The instructor's involvement, prompting continued conversation, may help or interfere.The nature and timing of this encouragement are also likely to be factors.Follow-up studies are underway to address these considerations.
The legacies of marginalia and the triple-entry activity continue in students' online interactions with texts and classmates.Clearly, further research is needed to determine if, when, and how the benefits of Marginalia and other annotation systems can be achieved.We look forward to contributing to this legacy.

Figure 2
Figure 2Percentages of interactions involving two, three or four students Figure 4Chain interaction