Perceptions of Challenges and Enablers of Effective Teaching in a Large Research-Intensive University : Preliminary Findings

In October 2014, the University of British Columbia Vancouver campus (UBCV) ran a campuswide survey to establish baseline information on teaching practices and attitudes among faculty, to measure the impact of existing teaching and learning initiatives and to identify the conditions leading to change in practices and attitudes around teaching. Over 1000 participants with teaching responsibilities across 11 Faculties responded to the survey. This paper explores the main challenges and enablers of effective teaching practice at this large research-intensive institution, and presents preliminary findings centered on emergent themes of class size, workload, balancing activities and content, perceptions of the institutional value of teaching, and the current support for teaching available at UBCV.

Despite the prevalence of information intended to inform scholarly teaching practice, faculty may still be reluctant to move from lecture as the primary mode of instruction and remain skeptical about the efficacy of other methods (Henderson, Beach, & Finkelstein, 2011;Miller, Martineau, & Clark, 2000).Experience as a student remains the single most influential factor impacting how faculty teach (Stigler & Hiebert, 2000).Hence, faculty tend to "teach how they were taught, using largely passive lectures" (Blumberg, 2011, p. 27) and base teaching decision on "what content to deliver and not how to maximize student learning" (Blumberg, 2011, p. 27).
Furthermore, many institutions may be unaware of what teaching practices are employed in the classroom.Classroom practices are rarely shared and many faculty "close the classroom door and experience pedagogical solitude" (Shulman, 1993, p. 6).In an exploration of why evidence based practices were not employed in mathematics, Reys (2013) found this was difficult to determine since institutions rarely collect information on faculty teaching practices.
For many years, the University of British Columbia Vancouver (UBCV) was no exception.A large, research-based institution, UBCV is supportive of faculty autonomy and had no centralized reporting process to gather information regarding teaching practices.As a part of the Lasting Education Achieved and Demonstrated (LEAD) initiative, UBC ran its first faculty teaching practices survey at both the Vancouver and Okanagan campuses in 2008 (see Lin, 2008).This survey was intended to gather baseline information to measure the impact of programs across UBC aimed at improving the effectiveness and efficiency of teaching and learning, such as the Carl Wieman Science Education Initiative (CWSEI, Faculty of Science, launched in 2007), as well as to explore the overall teaching climate at UBC.In 2014, UBCV ran an updated version of the survey as a baseline measure for the new Flexible Learning Initiative (launched in 2013), and as a post-initiative measure of CWSEI impact.In addition, the 2014 survey was intended to provide data to inform strategic planning and decision-making, both campus-wide and within Faculties.

Methods
The development of the 2014 UBCV survey was a collaborative effort between the Science Centre for Learning and Teaching (SCLT) and the Centre for Teaching, Learning and Technology (CTLT).Because the 2008 LEAD survey had since been modified and implemented by various institutions in the Bay View Alliance (BVA, http://bayviewalliance.org/) as well as the Association of American Universities (AAU, https://www.aau.edu/),we started by conducting a comprehensive review of all versions of the survey.We also reviewed other validated survey tools such as the Classroom Survey of Student Engagement (The Trustees of Indiana University, 2015) and the Higher Education Research Institute Faculty Survey (HERI, 2014).In order to be able to make comparisons to the 2008 data, we decided to modify the LEAD survey by adding select components modeled after the BVA and AAU versions.
The result was an instrument consisting of two sections and 30 questions, primarily multiple choice or Likert scales, about teaching practices and perceptions.The first section of the survey included a series of questions related to instructors' classroom practices and expectations for students.The second section contained questions around perceptions of specific teaching practices, access to and knowledge of support or professional development resources, and perceptions of institutional support for teaching.The survey also included the following open-ended questions with no prompt or further instructions: 1. Briefly describe what you consider to be the biggest challenge to your teaching.2. What changes could be made at UBC to help you teach more effectively?3. Briefly describe one factor that has improved your teaching.
We then sought feedback on the revised survey from both faculty across disciplines and staff members from UBCV teaching and learning support centres.
Based on this feedback, we made a number of minor and major revisions including adding a question set about Teaching Assistants and, based on concerns from faculty about how the data might be used, asking faculty to respond to questions in the first section of the survey based on one course and not their entire teaching load.Once these revisions were incorporated, the survey was presented to an ad hoc committee including representatives from the Provost Office and UBCV Associate Deans with teaching and learning responsibilities.This committee suggested a number of additional refinements such as changes in question wording (e.g., misconceptions was replaced by preconceptions; the term flipped classroom was deemed too ambiguous and removed from the survey) and modifications to make the instrument more applicable to courses delivered fully online.The committee also decided to focus the first section of the survey on high enrolment courses.Once the suggestions from the ad hoc committee were integrated, the final version of the survey was validated by faculty and key stakeholders.Institutional ethics review and approval was sought to conform to research standards of ethics and integrity (BREB # H14-01879).
In order to allow for variations in context and to accommodate variations in timing across UBCV's 11 Faculties, we ran unique instances of the survey for each Faculty.These instances included minor modifications such as individual Faculty names, list of departments within each Faculty, name of local support units, and Faculty specific teaching and learning grants.Surveys were administered using an online survey tool licensed by UBC and paper copies were made available to participants who requested them.All but one Faculty were surveyed at the end of October 2014, although exact dates varied slightly by Faculty.Using the online survey tool, we sent an email invitation to participate on behalf of the Faculty's Associate Dean three to four weeks prior to the survey close date with one follow up reminder email two weeks prior to the end date.No incentive for participation was offered.
We received a total of 1177 valid, consenting responses from across UBCV's 11 Faculties.This number represents an average institutional response rate of approximately 24%, with individual Faculty response rates ranging from 14% to 68%.The variability in response rates can be attributed to the different inclusion criteria that were purposefully established and used by individual Faculties.When compared to Vancouver numbers reported in UBC's 2013 Equity Employment Report on faculty track and rank (UBC, 2013), we confirmed that the sample population in the 2014 UBCV survey is representative of the wider UBCV faculty population (Figure 1).
Prior to analysis, the data from each of the 11 survey instances was anonymized and combined into one dataset.Per our ethics approval, in order to preserve the anonymity of participants, we did not conduct analysis on any question or subcategory with fewer than 10 responses.We generated descriptive statistics for all questions at both the institutional and individual Faculty level, as well as ran additional analysis on a number of questions identified as high interest.The responses for the open-ended questions were analyzed and coded through a process in which all codes emerged from the data; no pre-identified themes or codes were used to analyze the responses.We then generated reports for each Faculty, containing both Faculty specific data and the UBCV aggregate data.Faculty specific data was only shared with the individual Faculty and the offices of the Provost and President.
In this paper, we present findings related to five "meta-themes" that emerged from the openended questions.
After the three open-ended questions were coded, we noted connections between a number of the emergent themes.We closely reviewed the top 10 themes for each question, along with associated comments, and identified five "metathemes" that spanned each of the questions.Below, as guiding topics for discussion and interpretation of both qualitative and quantitative findings, the five meta-themes are presented: Class Size; Balancing Content & Activities; Workload; Value of Teaching; and Support for Teaching.The quotes included in the next section were selected to illustrate the feedback provided by different participants.

Figure 1
Participant rank and tenure compared to UBCV population, as  And: "Increasing class sizes -we can't run field schools or field labs, or even many discussion groups the way they need to be run for maximum experiential learning…." Despite this, there is a strong belief among participants that it is possible to improve one's teaching even when class size is large.40% of the participants strongly agreed and 36% agreed with the statement that "Even without a smaller class size, I believe it is possible to improve the effectiveness of one's teaching."

Balancing content & activities
"In my area, the more the students *do*, the better they understand; but finding effective activities that can be done in an acceptable amount of time is a challenge." Issues associated with balancing content delivery and active learning opportunities emerged as the fifth biggest challenge to effective teaching (82 comments), while the adoption of active learning techniques was the eighth most frequently referenced factor for teaching improvement.Participants also overwhelmingly agreed (72% strongly agree and 24% agree) with the importance of encouraging "students to be active participants in learning".However, in responses to open-ended questions, participants reported class size, lack of technology to support active learning, theatre style seating in lecture halls, and the amount of content required to be "covered" as barriers to accomplishing active learning: "Classroom spaces!It is incredibly challenging to teach in lecture theatres where there is no room to move or circulate.Students themselves have identified the physical space where class takes place as an impediment to their ability to work in groups and even to be actively involved in exercises during class."Space modifications to allow for more flexible use of classroom space in support of active learning emerged as the eighth most recommended change (53 comments).
Further, participants were significantly more likely to agree that interactive teaching techniques/active learning was a more effective way to promote student learning than lecture (see Figure 2).

Comparison of response patterns to questions about the effectiveness of different teaching methods on student learning
Yet, in their largest enrolment courses, participants report spending significantly more time on lecture (described in the survey as "Instructor or TA presenting content") than any other classroom practice (Figure 3).Encouragingly, the average proportion of time spent on presentation of content is less than 50%, but it should be noted this variable has a much larger standard deviation than the others.Despite repeated references in the open-ended responses to challenges related to class size, we found no correlation between the amount of time spent on lecture and class size.We did, however, find that as the time spent on lecture increased, there were corresponding decreases in all other variables except assessment (which was relatively static) and discussion (which increased).This suggests participants were more likely to use time not spent on lecture for discussion and not more active practices like group

Figure 3
Mean percentage of time spent on classroom activities.Survey participants reported the time, per 12 week term, spent on the most common teachingrelated activities (Figure 4).On average, participants reported spending 65 hours per term preparing for and 34 hours marking work from their highest enrolment course.They reported spending significantly less time engaging with students (either in class or online).
Teaching Assistants (TAs) are often employed to mitigate workload associated with teaching, particularly marking.Despite this, only 61% of survey participants reported having TAs in their highest enrolment course.TAs were most frequently expected to provide support to students beyond class hours, help facilitate instructional activities and mark assignments and exams (Figure 5).
In responses to open-ended questions, participants also recognized the importance of technology and its potential to reduce workload.However, participants also acknowledged that mastering the use of technology can be time

Figure 4
Average time spent on teaching related activities, in hours.Roles TAs play for participants (analysis excludes participants who did not indicate having TAs) consuming and challenging.Despite this, technology was identified as the number five factor that has improved participants' teaching experiences (98 comments); in one participant's words "New media and technology enhancements has improved my teaching and has opened up new teaching opportunities."Technology has the potential of reducing the workload participants face, once the initial challenges associated with adoption have been overcome.

Value of teaching
There was a strong agreement around teaching being a priority for participants.54% of the participants strongly agreed and 32% agreed with the statement "Teaching is a priority for me".Also, survey participants perceived that the value of teaching is reflected in the institutionalized processes.For instance, 14% of them strongly agreed and 31% agreed that effective teaching plays a meaningful role in the annual review and salary decisions.19% of the participants strongly agreed and 41% agreed that effective teaching plays a meaningful role in the promotion and tenure decisions (Figure 6).
However, the open-ended responses show that some of the most prevalent perceived challenges around teaching have to do with the institutional value placed on research at the perceived expense of teaching excellence.The third most mentioned challenge for participants' teaching was the focus on research in the promotion and tenure process (117 comments).The following quotation exemplifies the challenge: "If there is no reward for teaching, why would one divert any of their personal or professional resources away from the areas that are rewarded?" In addition, the number four challenge was the lack of recognition for excellence in teaching (91 comments).This sentiment was captured in the following response to the biggest challenge question: "Lack of recognition, specifically at the highest level of UBC, that teaching is a priority and not Deliver some course instruction on their own.
Provide support to students beyond the scheduled class hours.
I seek feedback from my TAs when making instructional decisions.

Figure 6
Value of teaching as reflected in the institutionalized processes a given.I have never seen evidence that teaching activity can support a university academic's career prospects." When asked about the kinds of changes that could be made to help them teach more effectively, the suggestion of developing a culture or career progression system that values effective teaching as much as research was mentioned repeatedly, representing the second most popular change suggested by participants (85 comments).As one participant shared: "A culture change that appreciates the REAL value of the teaching mission of the university."

Support for teaching
Support for teaching emerged as a theme in all three open-ended questions.Despite the fact that the vast majority of participants either strongly agreed (56%) or agreed (37%) that ongoing improvements in teaching is part of their job, participants also revealed through their comments that the lack of real support for teaching, including adequate resources and materials, was the number six challenge for their teaching (78 comments The support for improving and changing teaching practices was perceived to be slightly higher at the department/local unit level when compared to UBC leadership.27% of the participants strongly agreed and 35% agreed with the statement "my local unit is supportive of faculty improving and changing teaching practices."When local unit is replaced by UBC leadership in the question, 14% of participants strongly agreed and 45% agreed with the statement (Figure 7).

Conclusions
Based on these findings, we believe participants of the 2014 Teaching Practices Survey see teaching as an important component of their position and view improvement to their teaching practice as an ongoing process that benefits from institutional support.However, they identified several issues they perceived to be challenges to the implementation of evidence-based teaching practices.These challenges include, but are not limited to, increasing class sizes, a perceived lack of support for teaching, and limited flexibility in the existing learning spaces.Some participants also indicated reluctance to invest time in teaching due to the perception that other activities, such as research, were more highly valued by leadership.Participants also indicated a number of factors that enabled effective teaching including, but not limited to, informal discussions with colleagues, feedback from students, participation in professional development opportunities, adoption of active learning practices, and use of technology.
To date, our preliminary findings have been shared both across UBC and more broadly.Faculty specific reports have been distributed to Faculties, presentations have been made for the Provost and President's offices, and survey insights are available at http://ctlt.ubc.ca/tps-report.Preliminary findings have also been shared externally with the community at large through a number of presentations at scholarly conferences and workshops.At the institutional level, the data has been used to inform a number of strategic decisions and at a Faculty level to measure the impact of Faculty initiatives such as the CWSEI and to advise incoming faculty.Feedback received from stakeholders across the institution indicates value in having data of this nature, and comparative data from future surveys will be even more useful.Given the value our institution has found in the findings, we recommend other institutions consider implementing similar surveys.

Figure 7
Figure 7Perception of institutional support for faculty improving and changing teaching practices indicated by the 2013 Employment Equity Report