Making Students ’ Metacognitive Knowledge Visible through Reflective Writing in a Mathematics-for-Teachers Course

Metacognition directly contributes to learning, performance, and beliefs about the self as a learner. This paper describes the rationale, implementation, and assessment of a weekly online reflection activity based on instructor prompts designed for post-secondary students who aspire to be elementary school teachers. Our study defines four categories of metacognitive knowledge that speak to the specific goals of the course and the characteristics of the students. Using these categories, 71 students’ written responses to four reflection prompts from three course offerings were coded, and their effects were examined in terms of types of metacognitive knowledge demonstrated. Our results not only confirm that students were engaged in metacognition through the reflection activity but also show that students exhibited different categories of metacognitive knowledge in relation to the varying emphases of the prompts.


Introduction
etacognition has been shown to be a powerful predictor of learning (Bransford, Brown, & Cocking, 2000;Veenman & Spaans, 2005;Veenman, Wilhelm, & Beishuizen, 2004;Wang, Haertel, & Walberg, 1990).Flavell coined the term in the late 1970s to mean one's knowledge about and regulation of one's cognitive processes (Flavell, 1979).Determining when, where, and how ideas apply helps students to deal with increasingly complex problems (Erickson, Peters, & Strommer, 2009).However, college and university teachers are continually surprised at the number of students who arrive with very little metacognitive knowledge, knowledge about different strategies or cognitive tasks, and in particular, accurate knowledge about themselves (Pintrich & Schunk, 2002;Tanner, 2012).
When it comes to learning mathematics, it is our experience that many students who aspire to become school teachers not only demonstrate very little knowledge about how they learn or how they should learn, but also experience a phobia toward mathematics.In addition, they often exhibit low selfesteem about their ability to learn mathematics and have a simplistic, and often incorrect, view of what doing and thinking mathematics is about.Our experience has been echoed by others (Ball, Hill, & Bass, 2005;Ma & Xu, 2004;Pape & Smith, 2002).Teachers' habits, confidences, and beliefs influence their classroom practices, which in turn impacts their own students' learning (Beilock, Gunderson, Ramirez, & Levine, 2010;Clarke & Hollingsworth, 2002).It is important for teacher preparation programs to address these issues early, consistently, and persistently.
In spite of the challenges, research has shown explicit instruction has a positive effect on improving metacognitive capability for a broad range of learners (Veenman, Elshout, & Busato, 1994) and particularly for weak students (Pressley & Gaskins, 2006).With this in mind, we have purposefully designed, implemented, and assessed a weekly reflective writing activity using prompts for a mathematics-for-teachers course.The activity engages students in thinking about their learning and themselves as learners.The ethics review board of our university approved the study, as it is considered quality improvement.
This paper is organized into six sections.We begin with a brief literature review on metacognition and metacognitive knowledge, the need for explicit instruction, and the assessment of metacognitive knowledge.Then, we provide an overview of the course and design of the reflection activity.Next, we present our research questions and methodology, which is followed by the data analysis.Specifically, we detail our analysis pertaining to the four categories of metacognitive knowledge that we have identified.We conclude by discussing our findings and how our study contributes to current research and teaching.

Metacognition and Metacognitive Knowledge
Metacognition is an active and growing field of research.Extensive research has been done on the subject, particularly in education, psychology, social science, and neuroscience.An indication of the extent of the activity is the proliferation of terms and concepts created to express what metacognition might entail.Such expressions include metacognitive beliefs, metacognitive awareness, metacognitive knowledge, feeling of knowing, judgment of learning, metamemory, metacognitive skills and strategies, and self-regulation, among others (Veenman, Hout-Wolters, & Afflerbach, 2006).Metacognition has become an umbrella term.Researchers have expanded Flavell's original definition "to include an emphasis on planning, monitoring, and evaluating one's own learning processes" (Tanner, 2012, p. 114).Biggs (1987) highlights the importance of this kind of awareness: "To be properly metacognitive, then, students have to be realistically aware of their own cognitive resources in relation to the task demands, and then to plan, monitor, and control those resources" (p.75).He refers to this awareness of one's own learning processes and one's control over them as meta-learning, a subcomponent of metacognition.Zohar and David (2009) introduce "meta-strategic knowledge (MSK) [that is] general knowledge about higher order thinking strategies" (p.179), and define it as "the awareness of the type of thinking strategies being used in specific instances" (p.179, emphasis in original).

Need for Explicit Instruction
Metacognitive knowledge is not easily acquired.
It is one thing for us as teachers to articulate the kinds of thinking we are seeking to promote; it is another for students to develop a greater awareness of the significant role that thinking plays in cultivating their own understanding.(Ritchhart, Church, & Morrison, 2011, p. 15).
In order to be aware of one's own thinking and opinions, Ritchhart and his colleagues stress the importance of making them visible, so that one can deliberately engage with one's own thought processes and opinions about learning, as well as the subject matter itself.Through explicit instruction and the use of classroom routines geared towards raising metacognitive awareness, they call for teachers to create a classroom culture that promotes thinking, learning, and metacognition.Many of the researchers' classroom routines engage students in thinking about thinking, and in developing understanding through reflective writing, both individually and collaboratively.Indeed, research has unequivocally linked reflection and metacognition (Kaplan, Silver, LaVaque-Manty, & Meizlish, 2013).Structured reflection has become a way to enhance understanding and problem solving (Eyler & Giles, 1999).One way to structure reflection is through the use of prompts.Schoenfeld (1992) describes the strategy of the instructor circulating the classroom and asking students "What (exactly) are you doing?(Can you describe it precisely?)Why are you doing it?(How does it fit into the solution?)How does it help you?(What will you do with the outcome when you obtain it?)"(p.356).Our reflection activity incorporates these types of questions through the use of instructor prompts.
Besides specific teaching strategies, the literature also presents three fundamental principles for successful metacognitive instruction: "a) embedding metacognitive instruction in the content matter to ensure connectivity, b) informing learners about the usefulness of metacognitive activities to encourage them to make the initial extra effort, and c) prolonged training to guarantee the smooth and sustained application of metacognitive activity" (Veenman et al., 2006, p. 9).Veenman (1998) discusses how to incorporate these principles and refers to these principles as the WWW&H rule (What to do, When, Why, and How).Based on these principles and drawing on the above-cited literature, we designed our term-long, weekly reflection activity using prompts to explicitly engage students in writing about their own learning, their opinions about mathematics, and themselves as learners.

Assessment of Metacognitive Knowledge
In order to evaluate the effectiveness of instructional intervention and to gauge metacognitive processes, much effort has been expended to define and to assess metacognitive knowledge and its components.Assessment methods are richly varied, ranging from product-based (e.g., learning outcomes) to process-based (e.g., pretest-posttest design), from quantitative measures (e.g., questionnaires) to qualitative measures (e.g., interviews, thinking-aloud protocols, and text-based analysis), and from off-line methods (e.g., data collection conducted before or after task performance) to on-line methods (e.g., data collection conducted during task performance) (Veenman et al., 2006).
All methods have pros and cons.For example, questionnaires are relatively easy to administer to a large number of people; on the other hand, interviews and think-aloud protocols can provide deeper insight into the students' perceptions, experience, and their thought processes.Productbased assessments allow researchers to measure the success of interventions, whereas process-based assessments are likely to lead to insights into how metacognitive tasks are carried out and what factors, whether situational or general, might contribute to performance.
Furthermore, some assessment methods may be more intrusive than others.For example, using data generated/collected as a natural part of the learning process (e.g., assignment data, observational data) is less intrusive than using data collected specifically for research purposes.Fundamentally, researchers believe that methods need to align with the metacognitive knowledge or the skill component to be assessed (Veenman et al., 2006).Equally important, methods need to be context-sensitive and attuned to the particularity of a course, its learners, instructional goals, and intended learning outcomes (Zohar & David, 2009).
The design, implementation, and assessment of our reflection activity account for these considerations and are consistent with findings from the literature.

About the Course and Design of the Reflection Activity
The mathematical content course is a gateway course designed for students who aspire to become grades K-7 teachers at a Western Canadian research university.
It is delivered both online and face-to-face (F-to-F), altogether five times a year.The number of students ranges from 40 to 100 in the F-to-F version and 25 to 40 in the online version.Typically, half of the students in this course are weak in mathematics and/or anxious about their learning of mathematics.For this reason, a chief goal of the course is to enhance students' awareness of their own learning and themselves as learners in order to reduce anxiety about learning mathematics.
The reflection activity comprises 12 weekly reflection prompts, a special midterm exam reflection that focuses on deeper insights into a particular mathematical concept and the student's own learning, and a self-evaluation at the end of the course.The entire activity is mandatory but graded only by the self-evaluation, which requires that the students' responses be used as evidence.Using a learning management system (LMS), the entire reflection activity is administered online, which allows the instructor to respond to all students in both course types.The students are provided with the following definition and clarification, both verbally by the instructor and in writing through the LMS: Definition: Reflection is the process of internally examining and pondering -carefully and criticallyan experience and its meaning to the self.The result is a change in conceptual perspective, which influences future behaviour.
Clarification: In this course, you are asked to do a reflection, when 'an experience' is alluded to in the reflection prompt.Beyond stating some facts about your experience regarding a particular topic or learning, you are asked to delve deeper into how your thinking as a result of the prompt is affecting and changing you.While you are asked to reflect as directed per topic that is covered, you are also expected to continue to reflect without my guidance in order to grow as a learner.
The reflective activity is sequential: a reflection prompt is supplied by the instructor, a response is given by the students, and feedback is provided by the instructor.Typically, the reflection prompt is either related to a classroom activity such as "What made an impact in your learning during the Centimetre and Isometric Dot Paper exploration?" or a general question regarding a mathematical concept such as "What does place value mean to you now as opposed to before?".Students are requested to analyze how the concept/exploration is affecting and changing their mathematical perspective along with their knowledge, thinking, and learning, which in turn shapes the students' responses.Lastly, in order to deal with the volume of responses, the instructor replies to student responses from approximately only every second prompt throughout the term by providing feedback that supports the metacognitive analysis.The feedback is pre-constructed based on the first author's finding as an instructor that nearly all students' responses fall into certain categories (e.g., expressing confidence, frustration, language barrier, or formula-orientation).A general feedback is established as well in order to take care of the occasional student responses that do not fall into any of the determined categories.This so-called feedback design along with its mechanism, rationale, and implementation is expounded in the work of Menz, Xin, and Li (2015).

Research Questions
Our study focuses on examining students' written responses to the weekly reflection prompts in order to investigate students' metacognitive knowledge as they engage with mathematical concepts or solve mathematical problems.Specifically, we seek to answer the following two research questions about metacognitive knowledge:

Methods
Our methods are non-intrusive, process-oriented, quantitative, and qualitative.Based on the works of Mason and Scrivani (2004), Schoenfeld (1992), Tanner (2012), Veenman et al. (2006), and Zohar and David (2009), we have identified four categories of metacognitive knowledge (MK) that speak specifically to the purpose of the reflection activity, which is to enhance students' awareness of their own learning and themselves as learners with the goal to reduce anxiety about learning mathematics, in order to assess students' written responses.To aid our analysis of student responses, we have identified indicators for each category in the form of verbs.All responses are segmented into clauses.Based on the indicator, the clause is assigned to the category (see below) it belongs to, if at all.Clauses that are not relevant to the prompt, reference previous responses, provide an example, or are factual in nature are not coded.To find out the robustness of our categories, we calculated the inter-coder reliability of our code assignments based on percentage agreement.Nine students' responses were chosen at random and their coding compared between the research assistant and first author, which resulted in 100, 80, 80, 80, 77, 77, 74, 74, and 67 percent consensus respectively.Therefore, a percentage agreement of the inter-coder reliability was obtained at 79.The four categories are provided here along with a description and examples of their respective indicators: MK I concerns the planning and monitoring of heuristics and strategies.Indicators are task-oriented and specify or provide details about a heuristic or strategy that usually involves an action verb, such as "draw", "visualize", "write", "read", "show", "discuss" or "solve".Example indicators are "reread", "rewrite", "familiarize", "apply" or "break down".
MK II concerns the evaluating of heuristics, strategies, and self.Indicators are both task-and self-oriented, and involve a verb that is of an evaluative nature about a heuristic, strategy or self.Example indicators are "struggle with", "tend to", "appears easier" or "does poorly".
MK III concerns the awareness of self-possessed knowledge.Indicators are self-oriented and involve a verb that signals awareness.Example indicators are "able to", "notice", "understand" or "happens to me".
MK IV concerns feelings about self with regard to learning mathematics.Indicators are self-oriented and involve a verb that expresses feelings.Example indicators are "like", "dislike", "discouraged" or "ashamed of".
We analyzed seventy-one (n=71) student responses to four weekly reflection prompts in three course offerings that were delivered during 2014 and 2015.Although all enrolled students participated in the study, only those students who responded to all four above-mentioned prompts, as well as the selfintroduction, were counted (see Table 1).The selfintroduction was analyzed for statements of confidence and beliefs described in the next section.The four of the thirteen prompts in our analysis were selected because they were well distributed throughout the term, with more or less equal intervals between the times these prompts were assigned (see Table 2).In the week of the first midterm examination, namely week 6, a special midterm reflection activity was assigned to the students, which explains the mismatched "week count" and "prompt number" in Table 2. Furthermore, the four prompts vary in their emphases from mathematical concepts to problem-solving strategies, and themselves indicate a variety of MK I through IV (see Table 2).For each prompt, individual student responses were coded for the four categories of metacognitive knowledge.The coding results were then tallied and compared across prompts and course offerings.

Data Analysis and Results
Metacognitive Knowledge: Research Question 1

What types of metacognitive knowledge do students demonstrate in responding to the reflection prompts? Do the responses vary in the types of metacognitive knowledge demonstrated from prompt to prompt? Do they vary from course offering to course offering?
Consistently across all four prompts, students' responses demonstrate each of the four MK categories (see column labelled "Total" in Table 3).However, the counts for MK II and MK III (704 and 582 respectively) are much higher than those for MK I and MK IV (241 and 158 respectively).
Comparing across the four prompts, we see that students responded well in volume to each of the four reflection prompts when combining all four MK categories; however, the total counts of MK for prompt 12 are noticeably higher than those of the other three prompts (see row labelled "MK Total" in Table 3).The frequency distributions in percent of responses over the four MK categories also vary from prompt to prompt (see Figure 1), with Prompts 1 and 5 leaning toward MK II, and Prompts 7 & 12 leaning toward MK III.Responses to Prompt 12 are the only ones, comparatively speaking, that demonstrate MK IV reasonably well.Overall, the four prompts seem to favour MKs II and III more than MKs I and IV.One reason for the dominance in metacognitive Table 2 The four reflection prompts selected, for which student responses were analyzed.The emphasis/emphases of each prompt is denoted by MK I through to MK IV

Week Count Prompt Number
Prompt What do you struggle with the most during problem-solving activities?(MK II) When do problems appear easier to manage?(MK II) 5 5 How important are diagrams for you in problem-solving?(MK II) Do you try to produce them yourself?(MK I) 8 7 When do you ever use ratios?(MK I) Are ratios useful in day-to-day math?(MK II) 13 12 The course started with an intense problem-solving activity and this activity was weaved into group assignments, online assignments, and readings from the textbook/extracts throughout all topics of the course.Reflect back how you dealt with (MK I) and felt about problem-solving at the very beginning of the term (MK IV), and share what -if anything -has changed for you (MK III), and explain why.knowledge concerning evaluation and awareness could be that the earlier-stated definition and clarification direct the students to be "critical" (which relates to evaluation) and to "delve deeper" in order to document "change" (which relates to awareness).Furthermore, the entire course, regardless of being offered online or F-to-F, continuously demands that students evaluate and think about their learning regarding the material and delivery that is being offered.While feelings are addressed, they are less dominant throughout the course.
Figure 1 MK percentage distribution by reflection prompt across student responses, combining data from three course offerings (n=71) Comparing across the three offerings (see Figure 2), we see that the two 2015 offerings, one online and one F-to-F, show a very similar distribution of MK categories across prompts while the distribution of the 2014 online offering appears somewhat different.However, taking a closer look at the distributions, we can see that the difference exists mainly in the responses to Prompt 1 and mainly in MK I. Prompt 1 seems to have invited significantly more reflection in planning and monitoring heuristics and strategies among the students of the 2014 offering than it did in the two 2015 offerings (see the first bar of Prompt 1 in each graph of Figure 2).Because this variation is fairly minor considering the whole picture, we claim that the prompts are consistent across course offerings in engaging students practicing metacognitive knowledge with varying emphases.Furthermore, in all three graphs in Figure 2, each MK category exhibits similar patterns of increase and decrease across the four prompts.For example, MK II is dominant in Prompts 1 and 5 and then tapers off in Prompts 7 and 12, whereas MK III is first weak in Prompts 1 and 5, and then continuously increases in Prompts 7 and 12.These behaviours in increase and decrease are mostly aligned with the dominant MK classification of each prompt.
As an example, we take a closer look at Prompt 12, which is itself identified as prompting MK I, MK III and MK IV (see Table 2).Significantly, only Prompt 12 is identified as prompting MK IV among the four prompts, and for the first time, its responses show substantial occurrences of MK IV.Equally important, Prompt 12 is also the only one identified as prompting MK III, and its responses demonstrate the highest occurrences of MK III.Also in Prompt 12 both MK III and MK IV dominate over MK I, which is weak in response occurrences.However, further investigation using data from future course offerings is needed to continue verifying these initial findings.
As a global answer to Research Question 1, the results indicate that the emphasis/emphases on the categories varies from prompt to prompt, even though all four prompts invite students to use all four categories of metacognitive knowledge.This finding is consistent across course offerings, whether online or F-to-F and from term to term.This gives us confidence in the robustness of these prompts for assessing the types of metacognitive knowledge that students demonstrate.These results also suggest that instructors can purposefully design their reflection prompts to engage students in practicing particular types of metacognitive knowledge because the emphasis/emphases on the categories from each prompt also resulted in an emphasis/emphases on those particular categories in the reflection responses.
Metacognitive Knowledge: Research Question 2 Does students' awareness of their own learning grow over the course of the term, so that they can employ what they learn about their own learning of mathematics to gain greater competencies in mathematics?
Originally, the instructor (first author) was hoping that students' awareness of their own learning and feelings towards their learning would grow over the course of the term.That is, they would demonstrate more MK III and MK IV toward the end of the term, and indeed, our distribution graphs show this is the case (see Figure 2).However, this could be attributed to the wording of the reflection prompts that favour the use of these two MKs over the other two.Our study is inconclusive at this point about whether the reflection activity has led students to gain greater awareness of self-possessed knowledge as well as of their own feelings about learning over the duration of the term.This is the area for which further research is required using additional assessment methods such as a pretest-posttest design.However, the analysis of the responses does indicate that students are active in metacognition, and as Figure 1 indicates, MK II and MK III regarding evaluation and awareness of knowledge are dominant across the four prompts.
Overall, the analysis confirms that the different emphases in metacognitive knowledge of the reflection prompts appear to direct students to reflect using specific types of metacognitive knowledge.This result is significant in that it would allow an instructor to guide students to reflect within a specific area of metacognitive knowledge.The need for explicit instruction is well-documented and justified in the literature (e.g., Eyler & Giles, 1999;Ritchhart et al., 2011;Schoenfeld, 1992).Increasingly, researchers and teachers are designing, implementing, and sharing strategies to directly support student's development of metacognitive awareness and knowledge (e.g., Bransford et al., 2000;Mair, 2012;Zohar & David, 2008).Further investigation using data from future offerings of our course and continuous refinement of our prompts will contribute to the shared knowledge and practice.

Conclusion and Discussion
Metacognition is an important and yet complex exercise to engage in teaching and learning.Our study focused on metacognitive knowledge, a subcomponent of metacognition, and examined the effectiveness of using reflective writing activity to engage students in acquiring and applying metacognitive knowledge and making these visible for themselves as well as for the instructor.Specifically, we have defined four categories of MK to help us assess students' metacognitive thought processes.Making these categories explicit has helped us to better understand what we hope to achieve through our teaching, namely to enhance students' awareness of their own mathematical learning and themselves as learners in order to reduce anxiety about the subject of mathematics, and why we do what we do as teachers when cultivating students' metacognitive awareness.Such realization will help us to be more intentional about our teaching.At the same time, it helps us consider how students learn to think and think to learn.The reflective writing exercise allowed our students "to externalize the processes of thought so that they can get a better handle on them" (Perkins, 2011, p. xv).
Our study has demonstrated that instructors can purposefully target various aspects of MK when providing students with reflective writing opportunities.Verified by the literature and demonstrated in our own study, we see the importance of prolonged engagement of students in metacognitive exercises.Through these exercises we help students to form a new habit of thinking, namely thinking about their thinking and learning, so that they can successfully implement metacognitive knowledge for the learning of mathematics -or any other subject for that matter, such as planning learning tasks, monitoring comprehension, evaluating progress, accessing self-possessed knowledge, and dealing with the feelings of the self specifically during mathematical learning.

Figure 2
Figure 2 Comparing percentage distribution of each MK in the responses to the four prompts for A. online Fall 2014, B. online Spring 2015, and C. F-to-F Spring 2015 courses

Table 1
Enrolment, study participants, and mode of delivery of the three course offerings

Table 3
MK counts by reflection prompt and category combining student numbers for three course offerings (n=71)