Student Motivation in Response to Problem-Based Learning

Problem-based learning (PBL) is a self-directed learning strategy where students work collaboratively in small groups to investigate open-ended relatable case scenarios. Students develop transferable skills that can be applied across disciplines, such as collaboration, problem-solving, and critical thinking. Despite extensive research on problem-based learning, a thorough examination of student engagement in relation to PBL is lacking (Savin-Baden, 2014; 2016). We studied student motivation through self-reflection exercises, surveys, and peer-evaluations in a third-year undergraduate anthropology course (N = 49) with students of differing levels of course material experience. We hypothesized that throughout the course student motivation would be highest in students with the most subject matter experience. The results show that student motivation was higher in students with more subject matter experience at the beginning of the course, however during the course their motivation decreased. By the end of the course the majority of students had high motivation toward PBL, however, we were surprised that forensic specialists with the most subject matter experience had low motivation at the end of the course. This research is important to demonstrate the challenges of implementing PBL in a traditional curriculum, and to provide suggestions for engaging diverse student populations in PBL.

tudies in the health sciences have found that problem-based learning (PBL) increases student motivation and engagement in course material over traditional lecture formats (Albanese & Mitchell, 1993;Murray & Summerlee, 2007;Prosser & Sze, 2014).PBL follows constructivist theory by allowing students to control their own learning process in small group collaborations where they work out practical case-based scenarios (Douglass & Morris, 2014).Instructors act as facilitators to promote studentcentered learning by asking relevant open-ended questions that enable students to develop their own paths to achieve the learning outcomes of the course.Students gain transferable skills such as critical thinking, collaboration, and problem-solving that they can apply across disciplines (Klegeris & Hurren, 2011).Problem-based learning began as a teaching method to foster problem-solving skills in medical students (Barrows, 1996;Barrows & Tamblyn, 1980).It has subsequently been successful in other professional science-based programs (Masek, Yamin, & Ridzuan, 2011;Prosser, 2004;Vardi & Ciccarelli, 2008); however, the extension of PBL to the wider undergraduate curriculum has primarily involved modified (i.e., hybridized) versions that integrate mini-lectures or content driven sessions as scaffolding to accommodate larger class sizes and greater student diversity (Allen, Donham, & Bernhardt, 2011;Fukuzawa & Boyd, 2016;Savin-Baden, 2014).

Intrinsic Motivation
Even though student engagement is a key component of PBL (Allen et al., 2011;Savin-Baden, 2014, 2016), there has been little to no examination of student engagement in PBL studies.Furthermore, the current definitions of student engagement may be misleading because they are focused on institutional learning outcomes such as content retention (Savin-Baden, 2014), rather than including "student connection with the learning context, discipline, peers, and tutors that enable transition and voicefulness in learning" (Savin-Baden, 2016, p. 3).Self-regulation demonstrates engagement as students identify goals, compose strategies to plan and manage resources, and monitor their progress towards their intrinsic goals (Zimmerman, 2002).Self-regulation requires intrinsic motivation (Sungur & Tekkaya, 2006).Students must be intrinsically motivated to successfully engage in self-directed learning (Hung 2011;Savin-Baden, 2014).According to Self-Determination Theory, extrinsic motivation is guided by external factors such as grades and approval, while intrinsic motivation is based on autonomy, competence, and relatedness (Douglass & Morris, 2014;Masek et al., 2011).Intrinsically motivated students use their own interest and sense of satisfaction to challenge themselves with a particular task (Raiyn & Tilchin, 2015).PBL has been suggested to increase student intrinsic motivation through these mechanisms because students are responsible for their collaborative process to investigate a relatable problem.The premise is that the more control students have over their own learning process, the greater their intrinsic motivation will be as demonstrated by their self-regulatory engagement in the course material (Douglass & Morris, 2014;Radovan & Makovec, 2015).

Assessment of Intrinsic Motivation
Assessing intrinsic motivation during PBL can be challenging and it is critical that appropriate assessment strategies are used to measure student achievements (Murray & Summerlee, 2007;Raiyn & Tilchin, 2015).For example, traditional content driven assessments (e.g., multiple choice tests) have been negatively associated with PBL because they encourage students to focus on content learning rather than higher order thinking (Murray & Summerlee, 2007;Raiyn & Tilchin, 2015).In contrast, PBL assessments should align with the teaching approach by emphasizing the learning process through collaboration, problem-solving, and critical thinking (Murray & Summerlee, 2007).Critical self-reflection is a method where students integrate their learning experience with learning outcomes.An effective self-reflection encourages students to think differently about their learning and guides them from superficial interpretations of complex problems toward critical thinking (e.g., the Describe, Evaluate, Assess Learning (D.E.A.L.) method (Ash & Clayton, 2004;Ash, Clayton & Atkinson, 2005;Ash, Clayton & Moses, 2007).Students express their motivation in the critical reflections through their perceived ability to: complete a task (i.e., competence); collaborate with their peers (i.e., relatedness); and see benefits from successfully completing the task (i.e., autonomy) (Belland, Kim, & Hannafin, 2013).

Subject Matter Knowledge
Prior subject matter knowledge has also been discussed as a key determinant for successful PBL (Jonassen, 2011;Schmidt & Gijselaers, 1990;Sockalingam & Schmidt, 2013).Studies have found that students with more subject matter experience have improved group dynamics, subject interest, and academic achievement; however, problem familiarity does not influence critical reasoning of a particular problem (Schmidt & Gijselaers, 1990;Sockalingam & Schmidt, 2013).Thus, subject matter experience likely plays a large role in the success of PBL implementation and learning outcomes.
We focused on student motivation during the implementation of PBL as part of a larger ongoing project investigating different active learning initiatives in a technologically innovative classroom at the University of Toronto Mississauga.Our student population (N = 49) had a diverse subject matter experience and no previous exposure to active learning.By focusing on student experience during the implementation of PBL, we hope to contribute to the broader discussion of what variables influence the effectiveness of PBL (Allen et al., 2011;Loyens, Jones, Mikkers, & van Gog, 2015).We expect the results to show that students with greater subject matter experience will be more intrinsically motivated throughout PBL implementation.Ultimately, we hope to produce suggestions on successful implementation of PBL for student populations with diverse subject matter experience.

Course Structure
The University of Toronto Mississauga (UTM) is part of a large tri-campus research intensive university.UTM has an undergraduate population of approximately 14,000 students and 600 graduate students.
Human Osteology is a third-year undergraduate course that is a core requirement for the forensic anthropology program and a recommended course for the biological anthropology specialist program.In the 2015 iteration of the course, groups of 2-3 students were assigned one adult human skeleton for the term, and 2 of these groups were combined to form each PBL group (i.e., 4-6 students studying 2 skeletons).Two PBL cases relating to 1) sex determination of their adult skeletons and 2) age determination of their adult skeletons were conducted through the course.We composed the PBL groups based on students' course material experience as indicated in pre-course critical self-reflections and their program affiliation at the university.Forensic science specialists (N = 9) have been accepted into the forensic science program based on science related pre-requisites (minimum mark 75% in each course) and a minimum grade point average of 3.0.Biological anthropology specialists (N = 15) have previous experience in skeletal anatomy, and biological anthropology majors (N = 19) are students with an introduction to skeletal anatomy.Biological anthropology minors (N = 6) have no experience in skeletal anatomy but they have completed an introductory course in biological anthropology.Each PBL group included at least one forensic specialist, or one anthropology specialist, along with one anthropology major and one anthropology minor.Weekly PBL sessions were held in a technologically enhanced classroom to improve the active learning experience (Neo & Neo, 2005).
Each PBL group had direct access to online resources (e.g., internet resources, course references, electronic whiteboard & discussion board).Two sessions with interactive group exercises were given at the beginning of the course to encourage a cooperative atmosphere between group members; to emphasize the relevance of PBL principles to the field of anthropology; to familiarize students with the technology in the active learning classroom; and to provide a foundation of basic course terminology (Robinson, Harris, & Burton, 2015).During PBL sessions students could conduct online research simultaneously and collaborate on a discussion board that was projected at each PBL group table for the facilitator to monitor.This technology allowed two facilitators for 12 PBL groups.Google documents were used by the students to track their contributions to the PBL assignment.Students had access to a number of online resources on writing a scientific paper and their textbooks gave them a good overview of the literature as a place to start deciding what methods they would employ.The instructor and teaching assistant received PBL training from an experienced educational developer who oversaw the study.They acted as facilitators by asking openended questions to encourage productive group discussions.Each week the groups also met in the laboratory where they studied their assigned skeletons and learned different parts of the skeletal anatomy.
We followed the traditional closed-loop model of PBL (Murray & Summerlee, 2007;Savin-Baden, 2014;Schmidt, 1983).Groups of 4-6 students were given a practical problem to investigate as a group in a 5-phase process.During the problem phase groups brainstormed the key issues of the question to identify the information that they needed to investigate a solution.They used available resources to hypothesize possible solutions and then assigned specific tasks to each group member.During the self-directed learning phase, students worked individually on their tasks to investigate the problem.The PBL groups then reconvened for the reporting phase where they shared their individual findings and decided as a group on the methods that they would use to investigate the problem.They then moved to the laboratory, where they performed the application phase and actually tested out their chosen methods on the specimens.Lastly, they finished the PBL process by producing a scientifically written group report.

Student Assessments
Student assessments and grading were based on the critical self-reflections and peer evaluations within groups that were submitted with each written PBL group report (Murray & Summerlee, 2007).Students wrote critical reflections using the D.E.A.L. method at the beginning of the course and after the submission of each PBL report to reflect on their motivation for the course; their previous experience with course material; the learning outcomes of the course; their understanding of problem-based learning; how they felt about group work; and an assessment of their strengths and weaknesses.In their reflection on motivation, students ranked motivation as low, medium, or high and then described why they chose that level.For each student group (forensic specialist, anthropology specialist, anthropology major, and anthropology minor), student motivation at the beginning, middle, and end of the course was compared using a Friedman's ANOVA (nonparametric analysis of variance for repeated measures).Subsequently, for each student group post hoc analysis with Wilcoxon signed-rank tests were conducted to test for significant differences in student motivation between the beginning and end of the course.
The written PBL reports were assigned both a group grade and an individual grade based on the peer evaluation and the contribution of each group member (as indicated on the Google document).It is important to remember that in PBL there is not a single correct solution.Students worked with the instructor to determine the grading rubric for the PBL reports at the beginning of the course.This criterion-based rubric was produced to give students a clear understanding of the learning outcomes and to help students to focus on the learning process (Vardi & Ciccarelli, 2008).Students indicated in the reflections that they used the feedback on the rubric from their first PBL report to improve the writing of the second PBL report.Student reports were graded on their ability to: appropriately research the literature; justify the methods that they chose; make inferences from the literature; execute the methods; and create a comprehensive discussion on the methods' accuracy and validity.
Students also responded to PBL surveys at the end of the course where they used a Likert scale (1, strongly disagree to 5, strongly agree) along with a written section to evaluate student motivation in the course according to relatedness (participation goals), competence (learning of course material and learning outcomes), autonomy (the application of PBL outside of the course), and the use of technology in the application of problem-based learning.This study fell within section 2.5 of the University of Toronto Tri-Council Policy Statement on Ethics, and it was approved by the University of Toronto Mississauga ethics officer as a program evaluation review within normal educational requirements.

Motivation: Forensic and Anthropology Specialists
At the beginning of the course, all the biological anthropology specialists (N = 15) and 80% of the forensic specialists (N = 9) stated that they were highly motivated for the course material and groupbased problem solving (Figure 1).In self-reflections, all of the specialists mentioned that a primary motivator in the course was either their grade point average or a graduate school application.Most mentioned that the course was a program requirement and 70% of the specialists stated that previous knowledge of the course material and their ability to memorize a lot of material very quickly were their primary learning strengths.In the mid-course critical reflections, the high motivation of the specialists had substantially decreased (Figure 1).These forensic specialists stated time management as their greatest weakness in relation to the course.They felt that they did not have enough instructor guidance to properly "solve" the problem, but they did recognize that PBL groups forced them to collaborate with peers and take more responsibility for their own learning.Similarly, anthropology specialists felt that the PBL projects involved a significantly greater workload than traditional courses.
At the completion of the course, most anthropology specialists (84%) had high motivation while 88% of forensic specialists had low motivation (Figure 1).Forensic specialists felt that the workload outside of the classroom was unmanageable.They felt that they were not given enough direction toward the solutions to the problems and they did not see the applicability/practicality of PBL skills in future research.The Friedman ANOVA of nonparametric variance showed a significant difference in motivation throughout the course in forensic specialists (X 2 (2) = 7.600, p = 0.022) and anthropology specialists (X 2 (2) = 17.688, p = 0.001).Post hoc analysis with Wilcoxon signed-rank tests displayed a significant difference in motivation in forensic specialists between the beginning and the end of the course (Z = -2.428,p = 0.015), but no significant differences in anthropology specialists between the beginning and end of the course (Z = -1.633,p = 0.102).Ordinal values were assigned as 1 for low motivation, 2 for medium motivation, and 3 for high motivation (Figure 1).

Motivation: Anthropology Majors and Minors
Anthropology majors (N = 19) and minors (N = 6) had lower levels of motivation at the beginning of the course (Figure 1).Many of the minors stated that they were taking the course to get a science course requirement for their degree in the social sciences.In contrast to the specialists, majors and minors were more focused on their weaknesses.The majority discussed time management and remembering a lot of facts as their primary concerns in doing well.Majors and minors increased their motivation toward the end of the course (Figure 1).In their selfreflections, these students appreciated the peer learning process, and they were overcoming their apprehension toward group work.They were learning from their peers; thinking differently about the course material; learning how to critique the literature and write scientific reports.Both anthropology majors (X 2 (2) = 17.077, p = 0.001) and anthropology minors (X 2 (2) = 8.588, p = 0.014) displayed statistically significant changes in motivation throughout the course, as well as a significant difference in motivation between the beginning and the end of the course for anthropology majors (Z = -3.051,p = 0.002), and for anthropology minors (Z = -2.000,p = 0.046) (Figure 1).

Figure 1
Changes in student motivation during the beginning, middle and end of the course 1

Post course PBL surveys
The post-course PBL surveys showed that 95% of all the students felt that they had achieved the course learning outcomes.
The majority of students recognized that problem-based skills could be used across disciplines, and felt that although the course required more participation than other courses they had learned to work collaboratively in a group setting.They felt that PBL taught them how to investigate the literature to solve a problem but they did not feel that the process helped them think more critically about 1 A Friedman ANOVA test was used to determine significant differences in motivation for each student group (forensic specialists, anthropology specialists, anthropology majors, and anthropology minors) throughout the course (p = 0.001).Low motivation (ordinal variable = 1), medium motivation (ordinal variable = 2), and high motivation (ordinal variable = 3).* p ≤ 0.05.** p ≤ 0.01.*** p ≤ 0.001.the material.Less than a third of the students felt that PBL benefited their learning process more than the traditional lecture format, and more than half of the students stated that they would choose a traditional lecture-based course over a PBL course (Table 1).There were no significant differences found between the final grades of the different student categories as determined by a one-way ANOVA (F (3,46) = 0.489, p = 0.691) (minors 68.2%, majors 71.9%, anthropology specialists 73.5%, forensic specialists 71.7%).

Discussion
The purpose of this study was to examine the effect of subject matter experience on student motivation in PBL.We expected motivation to increase as the course progressed in keeping with other studies that have emphasized the importance of student engagement for PBL success (Masek et al., 2011;Raiyn & Tilchin, 2015).We also expected students with the most subject matter knowledge to the highest motivation as previous studies have suggested that prior subject matter knowledge is a key determinant for successful PBL (Jonassen, 2011;Schmidt & Gijselaers, 1990;Sockalingam & Schmidt, 2013).As we expected, there was an increase in intrinsic motivation for most students in this PBL course.In general, these students recognized the applicability of the skills learned in PBL (relatedness, autonomy, competence) to other disciplines and future endeavors.Contrary to our expectations, when the students were divided by their subject matter experience, the forensic specialists with the greatest course material experience and high motivation at the beginning of the course had the lowest motivation at the end of the course.We expected forensic specialists to have high autonomy with the problems and collaborative process as these are integral parts of a forensic profession.In our study, PBL did not motivate forensic students to engage in self-directed learning.We will discuss possible explanations for the challenges to motivate these students.

The Educational Environment
Similar to other studies (Hung, 2011;Vardi & Ciccarelli, 2008) we found that encouraging students to develop intrinsic motivation through PBL is a challenge in an environment of traditional lectures and exams where students have been rewarded for their extrinsic motivation.Hung (2011) argues, that in order for study habits to transition from traditional to constructivist self-directed learning, students need to change their self-regulatory goals.Since the PBL research is largely conducted in medical and other professional settings it may not be applicable to general undergraduate courses in the social sciences because many medical programs have adopted PBL throughout the curriculum.In addition, they may be a more homogeneous student population where students have more common learning goals, and problems are developed systematically (Hung, 2011;Masek et al., 2011).In our study, forensic specialists were fixated on content learning because this course was a core pre-requisite for fourth-year specialist courses that are being taught in the traditional lecture format.They were concerned that the self-directed learning of PBL meant that they were not learning enough of the course material to succeed in their intensive program.

Student Learning Behaviors
For students to be motivated in PBL they must understand and engage in PBL learning outcomes.Problem-based learning outcomes emphasize a selfdirected learning process (collaboration, research, and problem solving) and differ from traditional learning outcomes that emphasize content retention (Prosser & Sze, 2014;Savin-Baden, 2016).In our study's precourse self-reflections, students expressed their strengths and weaknesses in terms of subject matter knowledge and rote learning skills because they did not have any experience in self-directed learning.We learned at the end of the course through the critical reflections and surveys that 95% of the students, including the forensic students with low motivation, felt that they had successfully achieved all the PBL learning outcomes.Student motivation, therefore, is not necessarily reflected in learning outcome assessment.Hung (2011) points out that when PBL studies focus only on learning outcomes they are missing out on variables (such as student motivation) that can lead to the improvement of PBL practices.
Students with low intrinsic motivation engage in "ritual behaviors" where they appear to be active in the learning process but they continue to be content driven and fail to demonstrate higher order thinking, such as being able to make inferences between the literature and the problem (Dolmans, Wolfhagen, van der Vleuten, & Wijnen, 2001, p. 885).Forensic specialists expressed frustrations over the lack of guidance and they attempted to draw answers from the facilitators.They continued to emphasize traditional learning outcomes such as the importance of their course grade as a primary motivator to learn course material.Time management was another common contributor to low motivation.Students found that researching and critiquing the literature required more time outside of the classroom than traditional courses.Forensic specialists, in particular, felt that the time requirement was unmanageable.Minors and majors who increased in motivation through the course recognized their weaknesses in their critical reflections at the beginning of the course and perhaps this made them more open to the concept of PBL as an investment for future learning (Bate, Hommes, Duvivier, & Taylor, 2014).Forensic specialists may have resisted the transition to intrinsic motivation because they were successful in the traditional model with extrinsic motivation; however, the other student groups could have been equally successful in the traditional lecturing format in other disciplines.

Strategies for the Transition to Problem-Based Learning
Based on our findings and other published work, we have the following recommendations to improve the PBL experience for diverse student populations.When individual PBL courses are introduced in a traditional curriculum, instructors must impart to students the direct benefits of PBL.In order to invest in the transition from content driven to process driven learning, students must view PBL in a positive light (Bate et al., 2014).Students have to understand the pay-off for the extra effort and time as they take control of their own learning.They must believe that PBL will give them lifelong skills that are transferable in a continually changing world (Savin-Baden, 2016).Our introductory group exercises allowed students to engage in PBL in a relaxed group environment with no grade attached.Students took turns leading group discussions on a number of short, and practical open-ended problems.We stressed the application of PBL skills to the everyday experience of the students so that they would see the importance of self-directed learning.The students positively evaluated these exercises in their critical self-reflections.
Many post-secondary institutions are focused on improving student engagement.This provides a good opportunity for student selfreflection in the curriculum (Douglass & Morris, 2014).The greatest obstacle in critical student reflections is getting students to think more critically about their learning experience.Structured critical reflections such as the D.E.A.L. method guide students to think about their learning in different ways (Ash et al., 2005).
Critical reflections throughout the PBL process allowed us to monitor students' insight into their motivation in active learning.We recommend that instructors allow for some flexibility in the course structure based on the critical reflections.

Instructional Support
Problem-based learning has been modified into a diversity of forms that vary in their problem types, facilitation, form of group interaction, and assessment (see Savin-Baden, 2014).These hybridized versions of PBL demonstrate its flexibility for different learning contexts but make it difficult and often futile to compare student motivation between different PBL implementations (Savin-Baden, 2014).Hybridized forms of PBL can be used to introduce students to active learning because they can be modified for the context and student population (Fukuzawa & Boyd, 2016).We recommend short weekly problems in introductory PBL courses.This strategy gives students short-term reinforcements to encourage their motivation for the PBL process, and it gives them time to become familiar with the tasks that are required in active learning environments.
Hybridized PBL courses use the principles of PBL but often support the instructional design with added instructor interventions.This structural scaffolding may be in the form of short lectures, class discussions, or added resources to guide the students through the process (Allen et al., 2011;Belland et al., 2013).Scaffolding is important to prevent student frustration and unease when they do not have the confidence to work through missteps or conceptual impasses (Allen et al., 2011;Belland et 2013;Masek et al., 2011).Savin-Baden (2014) however, suggests that scaffolding can also hinder self-directed learning because false starts and uncertainty through collaboration are crucial components of self-directed problem solving.Scaffolding may prevent students from crossing what Savin-Baden (2016) calls a "transdisciplinary threshold", which involves the evolution of thinking to a higher order that transforms the way students view concepts (Savin-Baden, 2016, p. 2).Belland et al. (2013), however, argue that the correct scaffolding design can inspire student motivation in PBL.We recommend scaffolding in introductory PBL courses as a way to encourage the students' confidence in their ability to engage in active learning.This is particularly important when PBL is introduced in a traditional lecture-based curriculum.Students are enrolled in other courses that emphasize extrinsic motivation, and so they may need instructor support to encourage investment in self-directed learning.Scaffolding can be reduced as students' intrinsic motivation increases.

Limitations of this Study
It is important to recognize that there may be unique circumstances in this one course that are not applicable to all undergraduate social science courses.The small sample size could overemphasize individual eccentricities.Also, forensic specialists may have adopted more elements of intrinsic motivation than they recognized in their self-reflections.There may have been variables other than subject matter experience (e.g., sex, demographics of the students) that resulted in a difference in motivation in forensic specialists.It may be that this particular program attracts a certain type of learner.This was not investigated in this study.

Conclusion
Our study found that students with more subject matter experience did not necessarily have greater intrinsic motivation toward PBL in a third-year undergraduate anthropology course consisting of students of differing subject matter experience and no exposure to active learning.Problem-based learning implementation in a traditional curriculum requires more instructor support to encourage the students to invest in the transformation of their learning.We suggest that instructor supports should include introductory group exercises that emphasize the benefits of collaborative PBL without a grade attached.Assessments should include self-reflection exercises for students and instructors to monitor motivation throughout the PBL process, and adjust the implementation of the problems as the course progresses.We also found that the students responded positively to their participation in the creation of criterion-based assessments that clearly involved learning outcomes related to PBL (i.e., participation [relatedness] in the research process [competence] and then the application of their findings [autonomy]).
Short weekly problems instead of prolonged projects may be more effective in introductory courses to give students ongoing feedback and support.Scaffolding in the form of class discussions or short content-driven lectures gives students confidence to feel comfortable engaging in the PBL process.Even if scaffolding hinders the development of intrinsic motivation we feel that it is beneficial at the beginning of the process and can be reduced once the students start to feel more confident in their problem-solving abilities.We are implementing all of these suggestions in a future iteration of this course.

Table 1 I can see how PBL principles can be applied across academic disciplines