Teaching Culture Perception : Documenting and Transforming Institutional Teaching Cultures

Erika Kustra, University of Windsor Florida Doci, University of Windsor Kaitlyn Gillard, University of Windsor Catharine Dishke Hondzel, University of Western Ontario Lori Goff, McMaster University Danielle Gabay, McMaster University Ken N. Meadows, University of Western Ontario Paola Borin, Ryerson University Peter Wolf, Queen’s University Donna Ellis, University of Waterloo Hoda Eiliat, University of Windsor Jill Grose, Brock University Debra L. Dawson, University of Western Ontario Sandy Hughes, Wilfrid Laurier University


Introduction
n order to build an innovative, sustainable, and high quality post-secondary educational system, we need to consider the extent to which institutional cultures value quality teaching.As Cox, McIntosh, Reason, and Terenzini (2011) note, a culture with improved teaching quality is likely to lead to improved student engagement and learning.
Institutional culture can be defined as the embedded patterns, behaviours, shared values, beliefs, and ideologies of an educational institution (Kezar & Eckel, 2002), and it matters because the culture of an educational institution helps to shape the experience for both educators and students."Quality teaching" is a dynamic, contextual term, which may hold a I variety of meanings for different stakeholders (Harvey, Burrows, & Green, 1992;Harvey & Stensaker 2008;Hau, 1996;Scott, 1998).Hénard and Roseveare (2012) suggest that at its simplest level, quality teaching is "the use of pedagogical techniques to produce learning outcomes for students" (p.7).Institutional culture and quality teaching should be considered together, as there is an important relationship between institutional culture and teaching (Stein, 1997).The main focus of this component of an ongoing research program was to determine faculty, graduate and undergraduate students' perceptions of the teaching culture at their institution and identify indicators of that teaching culture.
Institutional teaching culture plays a major role in defining ways of perceiving, thinking, and feeling about the nature and scope of education.For example, in an effective institutional teaching culture the importance of teaching is recognized, teaching is constructively assessed, various stakeholders and resources are engaged, and teacher development is supported (Hénard & Roseveare, 2012;Paulsen & Feldman, 1995).Evidence suggests that culture can positively influence outcomes such as student persistence, learning, and engagement (Berger & Braxton, 1998;Berger & Milem, 1999;Cox et al., 2011;Grayson & Grayson, 2003).
Measuring the quality of an institution's teaching culture is challenging and generally takes the form of proxy measures, called indicators.Indicators can reveal the current state and perceived progress toward a specific objective.To assess the progress or change within an educational institution, four groups of performance indicators are customarily evaluated: input, process, output, and outcome indicators (Borden & Bottrill, 1994;Cave, Hanney, Henkel, & Kogan, 1991;Chalmers, 2008;;Richardson, 1994).
Input and output indicators are generally responsible for the quantitative measurement of an intended result or change; with input (or presage) indicators assessing the resources involved in supporting the institution (Chalmers, 2008) and output indicators measuring what is produced (Bruke, 1998).Process indicators provide an understanding about an institution's current practices and quality of practice, and inform further initiatives and policy decisions, leading to quality enhancement (Kuh, Pace, & Vesper, 1997).
Process indicators are usually qualitative and consider the most practical and appropriate measures of quality teaching and learning within higher education institutions (Chalmers & Thomson, 2008).Outcome indicators examine the quality of an educational program and the abilities of graduates (Warglien & Savoia, 2001).In some analyses, the output and outcome indicators are considered together as product variables (Gibbs, 2010).

Figure 1
Relationship Between Indicators and Teaching Culture Various indicators from any one of these types may be used to assess whether an institution values quality teaching, teaching enhancement, and a teaching culture.If researchers wish to determine whether teaching quality is a priority, it is helpful to consider the level at which quality teaching might be considered a priority within an institution.Information about the teaching culture may be gathered from three inter-dependent levels: university or institution-wide, program or departmental level, and individual level (Chalmers, 2008).For the purposes of this study, information gathered at the individual or departmental level would be used to gain a better understanding of the institutional culture.
Before engaging in a change process, Kezar and Eckel (2002) recommend campuses conduct audits of their institutional cultures, since assessing change requires knowledge of the current position, and future goals.The larger study on teaching culture that this team of eight institutions has commenced aims to document and analyze educational stakeholder perceptions on the importance of quality teaching at a university, and various components that contribute to an institutional culture that values teaching.This project involved a pilot study to develop a perception survey, and identification of additional possible indicators through which one could assess an institutional culture.The hope was to allow institutions to establish a baseline, evaluate change over time as well as the effectiveness and impact that projects have on shifting institutional culture.In addition, institutions could use the survey findings to identify practices and strategies to enhance their teaching culture.
Unfortunately, we currently do not have adequate measures to gauge institutional teaching cultures.Consequently, the main goal of the interinstitutional team leading this project was to develop a survey instrument that assesses the prevailing perceptions regarding the teaching culture among key stakeholders -the Teaching Culture Perception Survey (TCPS), and to identify seperate indicators that could be used to triangulate information.
The selection of indicators to be included in the perception survey was guided by Hénard and Roseveare's (2012) conceptual framework that identifies seven overarching themes, to which they refer as levers that provide concrete ways to foster quality teaching in post-secondary institutions.Their levers are: 1) raising awareness of quality teaching; 2) developing excellent teachers; 3) engaging students; 4) building organization for change and teaching leadership; 5) aligning institutional policies to foster quality teaching; 6) highlighting innovation as a driver of change; and 7) assessing impacts.
Their framework was adapted for the current project to more closely address the Canadian context, to decrease overlap between concepts, and to shorten the survey length.
Eight Ontario universities participated in the project which was funded by a Ministry of Training, College, and Universities' Productivity Innovation Fund grant (Kustra et al., 2014).As part of this project, a series of focus groups were conducted in order to determine 1) perceptions of teaching culture, 2) indicators felt to reflect institutional culture, and 3) perceptions of the teaching culture survey designed by the research team.The focus groups were run with full and parttime faculty members, sessional and contractually limited instructors (referred to by different titles at different institutions).We will refer to this group as 'faculty' for brevity and consistency.Additionally, undergraduate and graduate students participated in the focus groups.For the purposes of the current paper, we will focus on the analysis of the focus group results that identified participant perceptions of the teaching culture at their institution, and the indicators they felt might reflect that institutional culture.

Methods
The

Participants
Each institution held a series of focus groups with undergraduate students, graduate students and faculty members.Each group had a maximum of 10 participants, and the total number of participants at each institution is outlined in Table 1.

Procedure
This study was part of a larger study, including an online survey (Kustra et al., 2014).Participants were recruited from those who completed the Teaching Culture Perception Survey (3869 completed the survey: 729 faculty, 1602 graduate students, 1514 undergraduate students and 24 students with undeclared level).As part of the Teaching Culture Perception Survey conducted online, participants were invited to submit their email addresses if they wanted to participate in future focus groups.Once the survey was closed, potential focus group participants were emailed an invitation to participate in the focus groups.
The focus groups were facilitated by trained research assistants, and lasted 60 minutes.Before discussions began, participants were informed about the purpose of the focus group, and were asked for their verbal permission to be audio recorded.Participants who refused audio recording were invited to leave the focus group.Consenting participants were asked to provide a pseudonym on a name card to increase confidentiality.
Each focus group followed the same script (for full script see Kustra et al., 2014), addressing three types of questions: 1) perception of the teaching culture at their institution, 2) indicators of that culture, and 3) perception of the TCPS.The focus of this article is on the first two types.
The faculty focus groups included questions such as: "What is the teaching culture at [ … ] institution?","What are some components of quality teaching?",Probe: "What are the products of quality teaching?"and "Is the survey missing any questions what would tell more about the culture of teaching quality?"Questions for students included: "How do you know if a university values teaching?","What is done at the university that signals that teaching is a priority?",and "How do you know if a university does not value teaching?" Upon completion participants received a gift certificate of $20 toward Hospitality Services at their respective institutions.All sessions were audiotaped, and transcribed exactly from the audiotape by research assistants.Once transcriptions were complete, research assistants were assigned to re-read the transcripts to increase the accuracy.
Qualitative software (MaxQDA) was used to tag recurring themes for focus group questions related to the quality of the teaching culture and indicators.Themes were examined by research assistants from at least two institutions to ensure consistency of approach.

Results
The main focus of this component of the research program was to determine faculty, graduate and undergraduate students' perception of the teaching culture at their institution and identify indicators of that teaching culture.These questions are addressed in turn below by participant group.For the purposes of anonymity, when a centre for teaching and learning is identified, it will be referred to as the 'centre'.

Faculty Focus Groups
During the focus groups, faculty members from all three institutions identified both positive and negative input and process indicators.The frequency with which these indicators were mentioned varied between focus groups; however, several common themes were identified in the transcripts.Overall, the five most commonly mentioned indicators of teaching culture for faculty were: support for teaching, recognition of teaching, infrastructure, priority given to teaching, and teaching evaluations.

Support for Teaching
Faculty members identified a number of teaching and learning practices that were supported by their institution that demonstrated commitment to teaching.For example, participants specifically identified teaching and learning centres and most indicated that the units offer important resources and provide meaningful and relevant support for teaching.One participant stated:

You can ask to have a review by the [centre] and someone will visit your classroom and give you feedback, and there's also things like [the program]
where you can visit other people's classroom and get feedback from them.
Although there was a considerable amount of positive discussion around the support for teaching, it was not uniformly positive.This might be reflective of different institutions, or of different cultures within the institutions.For example, another participant had a different view: I think that you know we've put a ton of money into [the centre] and most of my colleagues, they don't have a high opinion of that, shall we say.They would much rather see the money spent in you know concrete supports for teaching like more TA support, or better classrooms, more proctors for tests ….
Other faculty mentioned that while centres were present, they wondered if they were adequately resourced: Well for [the director of our centre] is there an adequately resourced teaching support center?That's a pretty important question because they do a lot of work.

Recognition of Teaching
Teaching awards were seen as providing the university with an outward means to demonstrate its commitment to teaching and learning:

Recognition of teaching excellence [is] critically important.
Some faculty members, however, were cynical about the selection process and the value placed on teaching within individual departments.Selection criteria, and factors including gender, merit, and number of awards, were questioned.Another focus group participant felt that teaching awards were not valued: …there are you know teaching awards that people can strive towards, and all sorts of things like that.

But, on the ground, is it valued? … I think my students value what I do, but I don't think the university values it very much.
These excerpts provide indication that while the input indicator of recognition of teaching is generally valued, there is cynicism by some around award value and validity.These conflicting perspectives speak to competing priorities, and relate to process indicators to be discussed shortly, namely the ways in which research is valued over teaching and the perceived lack of a valid measure to evaluate quality teaching.

Infrastructure
Aging teaching spaces and inappropriate infrastructure was frequently linked to the culture of teaching.For example, It was a soulless room with very poor AV facilities, with students not in a space where you could do anything but stand at the front.

Another faculty member indicated that
We talk about being student-centered and focused, you know, and making teaching important and we do everything in the opposite direction... we've just renovated a whole bunch of classrooms, …. and then we walk into room after room after room and all the seats are bolted to the ground.
Infrastructure is an input indicator that was generally discussed as a barrier to effective teaching, impeding the use of best practices to engage students in active and meaningful ways.Infrastructure concerns ranged from the types of seating available to overcrowded classrooms to aging or broken technology.

Research above Teaching
When asked to describe the teaching culture on their campus, many of the faculty members who participated in the focus groups laughed or smiled, and then commented that the university culture was not about teaching; it was about research.Participants from every field spoke about how teaching was seen as a 'load' or a 'burden' that was escapable only if you could bring in enough research funding.Teaching release and sabbaticals were referenced as rewards for well-funded researchers while effective teaching was rewarded with an increased teaching load or larger class sizes.The impact of sessional or contractually limited positions was raised frequently.An example of this perspective is: I'm a limited term faculty member so I have a heavy teaching load, and I constantly see people who are tenured faculty members in our department trying to figure out ways to not have to teach.….We're not considered to be the same level of importance..

. So the message is that what you're doing really isn't important…
Another faculty member indicated that:

Promotions are definitely based on research almost solely because effectively if the letters don't come back from the external reviewers as warm or better there's no chance for promotion no matter how good of a teacher you are. And they actually see very little about your teaching because they get your CV and that's what they do their ranking based on.
A faculty member highlighted that this emphasis is evident as early as the recruitment of new faculty.

I think that part of the issue is that we recruit faculty based on research and we ask people to deliver teaching.
Faculty members outlined that the unequal value of teaching and research was particularly reflected in hiring, tenure, and promotion practices.

Teaching Evaluations
Many participants noted that teaching is difficult to measure, and the methods in place were inadequate.Some believed that student ratings of instruction (also known as student evaluations of teaching or course evaluations) were inaccurate, or were more indicative of popularity, course content, or easiness of a course rather than effective teaching.For example, So if teaching was really valued here, there would be a mechanism for measuring the effectiveness of the teaching that the faculty had confidence in, and that's definitely not a student opinion survey.Then there would be another mechanism that allowed that to be factored into our PT decisions in a measurable, justifiable, accountable way.

Another faculty member said:
We have, I think, a sound course evaluation form because it focuses on how effective [you were]… Now, you could be completely disorganized, and never comb your hair, and speak too loud, or not speak well at all, but still be an effective instructor.
Together, these concerns speak to the value that the focus group participants themselves placed on teaching quality and the commitment they had to providing their students with meaningful learning experiences.Other discussions that took place in the focus groups revolved around the value that faculty members placed on engaging students in meaningful and transformative exercises and discussion, researchinspired teaching, and innovative and engaging teaching methods.A few participants echoed the student perspective that accessibility and face-to-face contact was important, since ever-increasing class sizes made accountability on the part of the faculty member and the student difficult.Though current student ratings of instruction were generally considered an invalid way to measure effective teaching, most faculty members agreed that there would need to be a broader overhaul of the system before an alternative system could be enacted.
Overall, faculty identified input indicators (such as teaching support, infrastructure and recognition of teaching) and process indicators (such as hiring and promotion processes and teaching evaluations), which may be valuable for further study of teaching culture.Additionally, the focus group findings indicated existing teaching cultures included both positive and negative examples of most of the possible indicators.

Graduate Student Focus Groups
Graduate student participants shared many of the same concerns as faculty and undergraduates.Graduate students spoke about supporting innovative pedagogy and research-inspired teaching, support for teaching development and recognition of teaching as noteworthy indicators of a valued teaching culture.

Supporting Innovative Pedagogy
Graduate student participants noted that adequately supported innovative pedagogy is an important indicator of an institution's teaching culture.They specifically focused their comments on the effective use of technology and class time.The following student describes the potential damage to an institutional teaching culture when innovative pedagogy is embraced without sufficient support: I find universities rushing into it [online learning] because they save money.While there is an important need for online learning, when it's entirely online learning without any opportunity for in-course, and no support for teachers to understand technology and run an online course, that's when I see university's not valuing education and students.
The availability of technology would be an input indicator, but appropriate use of the technology and developing innovative methods are primarily process indicators, dealing with the delivery of programs.Again identifying both quantitative and valuable qualitative indicators.

Research-inspired Teaching
A second major theme for graduate students was research-inspired teaching; they viewed a quality teaching culture as one in which professors teach students how to find answers themselves rather than, simply, teaching them the answers.However, they acknowledge that the methods must be implemented well, or the experience is not effective:

Support for Teaching
Similar to the faculty focus groups, students identified the need for support for teaching development for graduate students as represented in this comment: We have so many hours in our TA forms [forms listing expected roles and hours]… and most of them are done sitting in lectures, so why not put those hours toward training a TA and feel you're getting paid for these training hours… "I've already taken this course and it would benefit me a lot more to be trained." Graduate students also felt that professional development for instructors is an indicator of a teaching culture: The biggest indication that the university doesn't really value teaching, is the fact that instructors are not actually given any direction on how to teach people."You're here for research.We're going to put you in front of a classroom, and we don't really care whether you actually know what you're doing there."

Recognition of Teaching
Similar to faculty, graduate students identified promotional incentives for teaching as indicative of a culture that values teaching.The most frequently mentioned motivational incentive was recognition through awards, though like the faculty, problems with the process of identifying and distributing awards were also raised.
I mean if a school values teaching a lot, there would be some awards set up for that instead of just the best scholars of the year, or the best publisher of the year.Maybe they'd have a best teacher of the year.
In addition to support for innovative pedagogy, research-inspired teaching and support for teaching development and recognition of teaching, graduate student participants shared many of the same concerns as undergraduate students.Of the five most frequently reported themes, graduate and undergraduate students agreed that teacher accessibility, coherent evaluation of teaching tools, and supported best practices were reflections of an institution's teaching culture.These themes are further explored below.

Undergraduate Student Focus Groups
The five most cited indicators of a quality teaching culture to undergraduates were use of best teaching practices, specific behaviours associated with effective teaching, teacher accessibility, evaluation of teaching, and implementation of student feedback.

Best Practices
Students from all three institutions commonly reported that professors' use of best teaching practices reflected value in teaching.The most frequent practices centred on collaborative learning, such as group discussion, classroom participation, or problem-based learning.Also of interest were effective use of technology and simplification of complex concepts.

I think professors should let students participate more. They should let students discuss issues or questions themselves, and [professors'] conclusions should come last. …The contemporary education system discourages us to think critically. If they
give us all the conclusions, students are more likely to think less.The use of current and supported best practices is a process indicator because it is a means to deliver effective teaching.A valuable qualitative insight to understand teaching culture.

Undergraduate
students identified specific behaviours that they associated with effective teaching as evidence of a culture that values teaching.These behaviours are not coherent methods to be included in best practices, but rather specific behaviours that students perceive as indicators of respect, expertise and professionalism.Like best practices, they are also process indicators, related to delivery of the program.Some students repeatedly mentioned specific behaviours such as professors who walk around the classroom, write their own textbooks, demonstrate passion for the material, dress in a professional manner, and arrive to class on time.The following comment reveals a student's interpretation: [Professors] don't really have a professional demeanour: showing up later than the students, not really dressing as a person who's supposed to be your superior and who's supposed to be instilling all this information to you.You look up to them to see where I can go.When they don't put the effort into coming on time, it makes it feel like it's just a side thing that they're doing.

Teacher Accessibility
Participants reported that the availability and approachability of professors may demonstrate the value they place on teaching.For example, in reply to the question, "When you think of quality teaching, what comes to mind?" a student indicated: Being accessible outside of the classroom hours and really [communicating] that you want the students to understand, that you're going to spend the energy to help them understand if they don't get it in class.
Teacher accessibility may be classified as an input indicator when related to time professors organize their time around teaching, research, and service as institution-supporting resources, and it may also be a process indicator, related to the delivery of a program.

Teaching Evaluation
Like faculty, both graduate and undergraduate students identified the development of valid measures of teaching effectiveness and opportunities for students to provide feedback to instructors on their teaching as indicators of an institution's teaching culture.Undergraduate, graduate students and faculty agreed upon the major flaws of the current evaluations for teaching.Moreover, undergraduate students extended their concern with teaching evaluations to include unmet expectations for the implementation of students' feedback.Undergraduate participants expressed discontent with current measures of teaching effectiveness and the need to develop more effective assessments.Students suggested the process needed to be more sophisticated -including more questions on the surveys, opportunities for mid-term feedback to professors, and the availability of an independent observer to provide feedback on teaching.One student commented that: I have a professor who will continuously ask for feedback through the semester and not at the end.And I was wondering why the university's not providing the student feedback from previous years to teachers.

Implementation of student feedback
Students mentioned that actually using student feedback and evaluations of instructors were indicators of a quality teaching culture; though many reported a lack of necessary change to address grievances.Ensuring that faculty members utilize student feedback constructively to enhance their teaching has the potential to impact teaching culture by improving the standard of teaching at an institution.This will also empower students by demonstrating that their opinions are valued.Further, including the teaching evaluations in promotion, tenure, and hiring decisions indicates that administrators value teaching.Implementing student feedback would be a process indicator, providing understanding about an institution's current practices leading to quality enhancement.The following comment illustrates the impact when there is not a constructive response: Where does our feedback go? … like, for this situation that we had, when we did report to the acting dean, it felt like there was nothing done and the professor actually retaliated and we were like "Ooooh, so should we have said anything?"

Discussion
Focus groups from three different institutions identified complementary themes from focus groups of faculty, graduate students and undergraduate students.In many cases, faculty members identified input indicators -resources that currently exist such as centres for teaching and learning and teaching awards; however, they also mentioned that, while the indicators were present, they were not sufficiently resourced.Faculty members at each institution also indicated that aging infrastructure was a major barrier to teaching effectiveness, and that the space for teaching needs to be adequately designed to support learning and student-teacher engagement.Both support for teaching and infrastructure are types of input indicators, representing operational variables that exist within the university to support and enhance a culture of teaching quality.These are factors over which an institution has control, and can take steps to address, though they carry resource implications.
Themes that related to process indicators were viewed as more problematic, suggesting a negative campus culture surrounding teaching.The two main themes that emerged as process indicators from faculty, and echoed in the comments of students, suggest that teaching quality is frequently overshadowed by a push for greater research, and that the processes in place to evaluate quality teaching are in need of improvement.Faculty members and students identified an unequal value of teaching and research that was particularly reflected in hiring, tenure, and promotion practices.Hiring practices could be considered either input indicators, or process indicators, and the process of hiring, tenure and promotion are accepted in the literature as important factors in an institutional culture (Cox et al., 2011;Kember, 1997).
Faculty, undergraduate and graduate students equally emphasized the need to improve teaching evaluations.Students identified the need to ensure implementation of the feedback collected from the students' evaluations.Faculty and graduate students also highlighted the need for valid and valuable promotional incentives or recognition for teaching.The themes that emerged, such as that of supporting teaching, recognizing effective teaching, constructively evaluating teaching, engaging stakeholders and dedicating resources; are all consistent with the international literature (Cashmore, Cane, & Cane, 2014;Cox et al., 2011;Hénard & Roseveare, 2012;Paulsen & Feldman, 1995;Percy et al., 2008).
Interestingly, the participants in the focus groups identified only input and process indicators.The literature indicates that outcome indicators, although the least common, are considered more useful than input indicators for bringing about meaningful change (Chalmers, 2008).
The Quality Teaching Culture project is a research program to develop and validate assessments of institutional teaching cultures that are appropriate for the Canadian context.An initial phase of this research program involved conducting focus groups with faculty members as well as undergraduate and graduate students to identify significant and relevant indicators of an institution's teaching culture to pursue.
If the themes identified at the three institutions are common at the provincial or national level, initiatives to target those themes could be extremely powerful.For example, the Scottish higher education sector, supported by the Scottish Higher Education Enhancement Committee (SHEEC), identifies one theme of national importance to focus quality enhancement efforts across the country over the course of three years (Gunn, 2014;Schofield, 2007; Quality Assurance Agency for Higher Education, 2014).Enhancement efforts and themes across Scotland have been successful, likely because the activity is supported by resources, infrastructure, and the structured integration of student voice by intentionally involving students in the process.These enhancement themes have had an impact on the teaching culture at institutions across the country; because this large-scale change is made in a collaborative fashion, with evidence of changing practice, it is integrated into decision-making and strategic planning (Matchell, 2008).
A process to examine teaching culture has the potential to change the way postsecondary institutions in Canada view and value teaching.Raising awareness of teaching and promoting quality enhancement can have a long-lasting effect on the culture of teaching, and, ultimately, on student learning.

Figure 1
Figure 1 illustrates the relationship between indicators and the quality of a teaching culture.Input and process indicators (qualitative and quantitative) together clarify the available resources and infrastructure.Input indicators are often related to quantity of resources, and process indicators are the means and processes (an institution's current practices).Understanding these indicators provides the appropriate information and context to better interpret the output and outcome indicators.Together indicators give a picture of the quality of an institution's teaching culture.Various indicators from any one of these types may be used to assess whether an institution values quality teaching, teaching enhancement, and a teaching culture.If researchers wish to determine whether teaching quality is a priority, it is helpful to consider the level at which quality teaching might be considered a priority within an institution.Information about the teaching culture may be gathered from three inter-dependent levels: university or institution-wide, program or departmental level, and individual level(Chalmers, 2008).For the purposes of this study, information gathered at the individual or departmental level would be used to gain a better understanding of the institutional culture.Before engaging in a change process,Kezar and Eckel (2002) recommend campuses conduct audits of their institutional cultures, since assessing change requires knowledge of the current position, and future goals.The larger study on teaching culture that this team of eight institutions has commenced aims to document and analyze educational stakeholder perceptions on the importance of quality teaching at a university, and various components that contribute to an institutional culture that values teaching.This project involved a pilot study to develop a perception survey, and identification of additional possible indicators through which one could assess an institutional culture.The hope was to allow institutions to establish a baseline, evaluate change over time as well as the effectiveness and impact that projects have on shifting institutional culture.In addition, institutions could use the survey

Table 1
Focus Group Participants