Texas Tech University.
All Things Texas Tech

Assessment of Teaching in Higher Education

Fall 2010

Texas Tech Memorial Circle

 

JoAnn Franklin Klinker
Associate Professor, Educational Leadership

Mary Frances Agnello
Associate Professor, Curriculum and Instruction

Aretha Faye Marbley
Professor, Counselor Education

Roseanna Currey Davidson
Associate Professor, Special Education

 

Although the research is clear that quality instruction positively impacts student learning and that ineffective teaching has long-term negative effects, assessing faculty teaching to improve instruction remains unclear. In short, we do not know how to do it well. This is not a small matter for tenure-track faculty.

Despite the classroom complexities and our lack of agreement on how to assess and for that matter to replicate good teaching that stimulates student growth, institutions continue to rely on formative assessment to help faculty improve and summative assessment to determine continued employment. In this paper the authors explore differences in purpose, process, and complexity of formative and summative evaluation, and offer insight into shifts in mindset, mentoring, and motivation that can have profound influences on assessment. For the professoriate the paper may serve to instill what Bok (1987) called a cast of mind, a way of thinking about the knowledge and skills required to solve problems encountered in practice, in this case assessment of teaching.

Formative and Summative Assessment

It cannot be emphasized enough that formative evaluation and summative evaluation are separate in intent, motivation, and construct. The intent of formative assessment is to improve teaching by the deliberate insertion of the observer's point of view. Conducted primarily through an ethic of care by the observer, the observed professional is usually motivated by an ethic that improvement of one’s teaching equates to being professional. The ethics of a critique and professionalism on the part of the observer are, in general, lesser considerations. Common forms of formative assessment include peer review, administrator’s observations or walk-throughs, feedback from students, self-evaluations, and unannounced observations by individuals variously affiliated with the teacher.

The intent of summative assessment is to make judgments about the results of teaching with the emphasis on objective measures for the purpose of administrative decision-making. Summative assessments help administrators decide to recommend tenure or not to recommend tenuring faculty, typically to boards of regents or trustees. Summative ratings are designed to be objective, reliable, valid, standardized, legally defensible, and impersonal. In most organizations, summative assessments are the duty of administrators as the assessment becomes a part of the hiring and retention process. Summative assessment does not lead to teacher improvement and can actually discourage improvement, especially if the evaluation is punitive. But if conducted primarily from an ethic of professionalism through policy enforcement, summative assessment involves an ethic of critique and the element that minimal expectations must be met. A commitment to justice may also require summative evaluations, for example, during actions leading to the dismissal of teachers for violations of ethics code. The primary intent is to make a judgment of teaching as evidenced through observations, examples of teaching and/or formative assessments, or perhaps a teaching portfolio. Such judgment has consequences in regard to employment primarily because it holds faculty accountable.

Compelling evidence indicates that formative assessment improves student learning if instructors use the assessment evidence to adjust their instruction (Popham 2010). Providing a culture that stimulates faculty to grow and adjust instruction rather than to comply and simply meet requirements of an annual evaluation is no easy matter. Peer review as formative assessment conducted from a learning focus in combination with other initiatives is the most effective. The other initiatives may include mentoring, coaching, establishing a culture of risk, and promoting an understanding of the complexity of mindset, motivation, and values clarification through reflection. As an example of the power of values clarification, note that a tenure-track professor in a university environment that values a quiet classroom as a learning classroom will focus on structure and control to ensure learning rather than relationships, passion, or collaborative partnerships as those tend to lead to more noisy learning environments. Considering that a tenure-track professor is conservatively a million-dollar investment, values clarification in teaching that promotes student learning assumes greater impact.

We are learning that the act of teaching is in actuality an unnatural act of immense complexity that does not always engage the individual brain’s preconceptions about how the world works. Assessing how well a faculty member teaches is even more complex given the lack of agreement in the educational community as to what constitutes good teaching. Even the old adage we know it when we see it is suspect when it comes to teaching because of the human tendency to feel good about what is most like what we as individuals do or prefer to do. Add the dynamic of an ever-changing classroom and the values enacted in balancing the innumerable tensions in that classroom, and something as seemingly simple as giving meaningful feedback that will stimulate teaching improvement becomes immensely intricate.

Formative Assessment Process

The boundaries that distinguish formative and summative evaluations should not be blurred no matter how expedient the desired outcome because their purposes are different (Glickman 2009). Organizations caught in the thrall of efficiency, objectivity, and impersonalization, unfortunately persist in doing just that. One of the more common transgressions occurs when clinical supervision, a process commonly used as a formative assessment tool, is used for summative evaluations. As a result, faculty no longer see formative and summative assessments as distinct from one another. That perception leads to confusion as to intent, and that confusion can hamper growth.

Clinical supervision is a five-step process consisting of (1) inviting someone to observe or to be observed, (2) conducting a pre-observation with both parties in which goals and objectives are discussed as well as the instrument that will be used by the observer to gather data (qualitative or quantitative format), (3) gathering data objectively, (4) interpreting the data, and finally (5) providing feedback to the person observed. While the process is straightforward and objective, at each step there is the possibility that the observed teacher will not benefit from the experience because of soft variables. The latter include, but are not limited to, diagnosis bias, feedback technique and tone, lack of active listening, the intent of the observer, the teacher’s motivation to change, discourse issues, timing, and differing values both on the part of observers and the observed. These softvariables plus others can lead to irrational judgments by the observer. In formative assessments the ethic of care ameliorates this problem, but that is not the case in a summative evaluation. In summative assessments the relationship is not collegial but rather hierarchical and the expectation is compliance, not growth (Glickman 2009).

Humans constantly judge or diagnose the world around them. In fact, it is impossible for us to not make judgments even when we do not realize we are doing so. This psychological undercurrent can result in diagnosis bias. The advantage of clinical supervision is that it forces us to put that bias aside when gathering data but the observer is susceptible to diagnosis errors at other stages of the process. Cultivating awareness of our personal construct theory, the lens or construct through which we view/judge the world can help because it forces us to consider other perspectives and points of view. Not making snap judgments and taking one’s time is a simple technique that avoids bias. Also, waiting too long to share results from a formative evaluation can dilute the impact. One week or less for feedback to the observed faculty member is optimal.

The process of formative assessment, because it involves judgment and decisions about others, must be fair, or at least perceived to be fair. Employees will accept criticisms and negative evaluations that affect their employment if they feel the process was done fairly. Keeping the observed person involved is critical to a perception of fairness. Helping that person understand the decision-making process, perhaps through something as simple as thinking aloud about how the observer arrived at the decision, is one way the observer can do that. The payoff is immense, because research shows that when employees are involved in their assessment, be it formative or summative, they have a sense of fair play about the judgment. Transparency of process and thought, even to the extent of sharing the evaluator’s own emotions, including doubt in formative assessments, should be a key component of one’s actions. For evaluators who work with tenure-track faculty, the concept of fairness in assessment cannot be overemphasized.

Teaching Techniques. One area that is not discussed within most educational units and which should be is clarification of the techniques of teaching. Communication rests on being able to understand one another, and having a basic understanding of techniques is the first step in establishing a common meeting ground in assessment between the observed and the observer. It is a point peer reviewers and administrators often miss. We assume that we all speak the same language of teaching, and the reality is we do not. For example, technique as used here should not to be confused with style that includes presentation, timing, organization of content, and a whole host of other variables that express the preferences of the instructor. Before any assessment initiatives begin, preparing the groundwork in common understanding of the language, terms, and techniques of teaching must be undertaken.

Despite the plethora of different names for variations, research is clear that teaching involves five basic techniques: lecture-based, skills-based, inquiry-based, individual and group, and technology-enhanced. Each of the techniques accomplishes specific goals that the teacher wants to accomplish, so although we are inclined to judge which is best, “Asking which technique is best is analogous to asking which tool is best—a hammer, a screwdriver, a knife, or pliers” (Bransford 2000). Variations within the techniques themselves include three to four broad areas:

Combinations and variations of these techniques are limited only by imagination. Given the complexity and ingenuity of human thought, the variations and combinations are infinite.

One Initiative.Peer assistance and review (PAR) programs are emerging in K-12 systems, but are not common, yet, in higher education. These consulting teachers leave their classrooms to mentor, support, and assess a caseload of new and/or struggling teachers. Focused on formative evaluation, a PAR consultant offers frequent direct assistance and makes unannounced classroom visits for up to three years. At the end of a specific period of time, the peer reviewer writes one summative evaluation, which becomes part of the administrative decision-making process to retain or to let the teacher go. PARs challenge a teacher’s autonomy and egalitarianism, because the consulting teachers are in classrooms more frequently than administrators. They succeed because of the motivation, skill, and design/planning that goes into the program. Although PARs are vulnerable to budget cuts, dispensing with such programs is not fiscally sound thinking.

Neuroscience, Midset, and Motivation

Organizations assume that employees need to be held accountable to be motivated to do their jobs, but emerging research indicates that assumption might be erroneous. Until recently, it was commonly accepted that two main drives powered human behavior: the biological drive (food, water, and sexual gratification) and extrinsic rewards and punishments. But a third drive, joy in the performance of the task, is proving to be just as strong as the other two, although it requires the right environment to survive. We call it intrinsic motivation, and researchers like Csikszentmihalyi (1990), Pink (2009), Pinker (2003), and Dweck (2007) are challenging folklore about motivation, including wide-spread organizational acceptance of McGregor (1960) Theory X, which assumed that people avoid effort, work only for money and security, and need to be controlled. McGregor’s Theory Y, which holds that work is as natural as play, that initiative and creativity are wide spread, and that if people are committed to a goal, they will seek responsibility, is more realistic.

A complex process like teaching has the drive of intrinsic motivation at its roots. Autonomy, the desire to drive our own lives in terms of task, time, team, and technique, is a critical element of formative assessments that work. Allowing tenure-track faculty to choose the teaching areas in which they want to improve is preferable to telling them to improve in specific areas. Allowing the observed teacher to choose the time for the observation, the person doing the observation, and the technique they want evaluated is also critical. The principal of autonomy holds true for mentoring and coaching activities as well.

Cultivating a culture of mastery, which fosters improvement in teaching, requires acceptance of failure and risk-taking in efforts to improve. That culture then helps faculty to renew their energy for improvement and to stay motivated. A key to mastery is changing one’s mindset to an acceptance that teaching is infinitely improvable, that it demands effort and deliberate practice, and that to become a master teacher is impossible as there is always room for improvement in teaching. The third essential element, purpose, is simply identifying teaching as a cause that is greater and more enduring than oneself. It is not something that we do for a living but something that we do in order to live fully, completely, and beyond ourselves. Faculty should be encouraged to take pride in what they do. Formative assessment procedures that enable the observed teacher to focus on drive, mastery, and purpose will improve teaching.

Mentoring and Coaching

Mentoring is a learning partnership, and the role of mentor should be less of an authority figure and more of a facilitator. Mentoring relationships fail when the focus is not on learning goals. Too often mentoring relationships fall into stereotypical roles of subservience on the mentee’s part and information giver on the mentor’s part instead of two-way information sharing and discussion. The latter fits more closely to what research indicates motivates people as it encompasses autonomy, mastery, and purpose. Table 1 (Zachary 2000) illustrates the elements in a learner-centered mentoring program:

Table 1 Elements in a learner-centered mentoring program

Mentoring Element

Changing Paradigm

Adult Learning Principle

Mentee role

From: Passive Receiver
To: Active partner

Adults learn best when they are involved in diagnosing, planning, implementing, and evaluating their own learning

Mentor role

From: Authority
To: Facilitator

The role of the facilitator is to create and maintain a supportive climate that promotes the conditions necessary for learning to take place.

Learning Process

From: Mentor directed and responsible for mentee’s learning
To: Self-directed and mentee responsible for own learning

Adult learners have a need to be self-directing

Length of Relationship

From: Calendar focus
To: Goal determined

Readiness for learning increases when there is a specific need to know.

Mentoring relationship

From: One life = one mentor; one mentor = one mentee
To: Multiple mentors over a lifetime and multiple models for mentoring individual, group, peer models

Life’s reservoir of experience is a primary learning resource; the life experiences of others add enforcement to the learning process.

Setting

From: face to face
To: Multiple and varied venues and opportunities

Adult learners have an inherent need for immediacy of application.

Focus

From: Product oriented knowledge transfer and acquisition
To: Process oriented: Critical reflection and application

Adults respond best to learning when they are internally motivated to learn.

Coaching is sometimes confused with mentoring in the educational literature, but when used as a formative assessment, it seems to be more directive than mentoring and focused on acquisition of skills rather than self-discovery. For example, peer coaching can be used by individual faculty in small groups to coach one another on concepts learned, a process that promotes better learning. In this situation, peer coaching is used as a formative assessment strategy that increases motivation and provides immediate feedback.

Issues of Diversity

Faculty of Color

Since students and faculty of color have been underrepresented in undergraduate, graduate, and professional schools of predominantly white institutions (PWIs) for more than a half-century (Johnson 1996), improving the teaching of faculty of color in those institutions is subsumed under a larger initiative—hiring and retaining faculty of color. From 1989-1997, studies indicate a meager 1 percent increase in the number of faculty of color in the PWIs professoriate despite research that indicates faculty of color serve as role models and mentors to students of color and are crucial to the success of students of color. Recent research indicates that the small number of faculty of color hired and retained in academe is thought to compromise the retention and graduation of students of color.

White faculty and administrators who observe faculty of color in formative and summative assessments should be aware of the following: Faculty of color confront many issues that are detrimental to their academic success, such as professional and social integration, a chilly academic climate, social isolation, and racial bias in the classroom (Jalomo 2000; Hendrix 1998; Rubin 2002; Sotello-Turner and Myers 2000; Dey 1998). By virtue of their color, race, ethnicity, and gender, faculty of color may be subjected to inappropriate outbursts, unwarranted attacks, challenges to their authority in the classroom, subtle attacks on their credibility, and biased and negative evaluations. Factors that impede the success of faculty of color are negative labels by university personnel, institutional commitment, institutional climate, academic support services, faculty attitudes, faculty expectations, and lack of mentoring or poor mentoring (Lee 1999; Sotello-Turner and Myers 2000). In the promotion and tenure process, many institutions continue to deny (or refuse to admit) any impact of race and color on judgments of teaching, despite the testimonials of faculty of color to the contrary and the evidence that White students and colleagues articulate different assessment criteria of teaching for faculty of color than the criteria they use to evaluate White faculty (DiPietro and Fay 2005). Consequently, there is a need to create and discover strategies that address retention, academic achievement, and success of faculty of color and women faculty of color. Formative assessment of teaching can be one such strategy if observers are aware of the challenges that faculty of color face.

Race and Summative Student Assessment. While summative assessment is not meant to improve teaching, it is worth mentioning here in the larger context of retention of faculty of color. Race bias in student evaluation is a relatively new area of quantitative research, although there are numerous qualitative studies. Empirical inquiry on race and the intersection of race and gender as it pertains to peer evaluation of teaching to student evaluations of instructors based on the students’ race and gender is almost non-existent. The limited number of quantitative studies on race and summative student evaluations of teaching showed that women faculty received significantly lower evaluation ratings than male faculty, faculty of color received lower course evaluation ratings than White faculty, and female faculty of color received the lowest course evaluations ratings. This research is also supported by numerous qualitative studies and counter-narratives of faculty of color.

Gender

Gender has been shown to be a variable in summative assessments for the granting of tenure and promotion to female professors within universities (Olsen, Maple, and Stage 1995; Porter 1994; Ezorsky 1977). Upper administration in universities is mostly comprised of men. Thus, gender can play a role in the big picture of formative and summative assessment of females, as a predominantly male hierarchy determines power and status from top to bottom of faculty (Olsen, Maple, and Stage 1995). Feminist research has shown that women are perceived to be more circular in their thinking than they are linear—the type of thinking most common to males, and this bias persists despite research that indicates women can and do think linearly (Martin 1987). Therefore, in a power structure dominated by males, such as in the traditional scientific paradigm, linear thinking has been, and continues to be, rewarded (Martin 1987). Inequitable allocations of general support to female faculty, salaries, and travel money have been attributed to this perceived lack of linear thinking, as well as bias toward linear thinking, along with gender bias against women (Olsen, Maple, and Stage 1995).

Gender and Summative Student Assessment. Gender also comes into play in student assessment of female faculty. Female students tend to evaluate female professors more harshly than male professors (Stone and Boldt 1994). In addition, female faculty who teach classes on diversity report to textbook publishers that they receive lower evaluations than their counterparts who do not teach diversity classes. Yet, universities rely on women and minorities to address issues of diversity in the curriculum (Olsen, Maple, and Stage 1995). If summative evaluations are to be meaningful, they must include more than an average of student evaluations over the career of a female faculty member. If formative evaluations are to be significant for female professors, especially those who teach in male-dominated subject areas, they must embrace the strategies explained in the discussion here.

Conclusion

Faculty impact student lives, but myths, biases, organizational beliefs, mindsets, and motivations about formative and summative assessment can also impact the lives of faculty. We are beginning to understand ways faculty are influenced by the organization around them and it is through assessment and how it is conducted that faculty understand just how much their organization values teaching. That in turn influences whether or not faculty within the system believe in the worth of what they do and how they do it. In higher education, intrinsic motivation, the inspiration behind good teaching and formative assessment, requires a culture that supports and encourages the professional drive to always improve one’s teaching.

Bibliography

Bok, D. 1987. "The Challenge to Schools of Education.” Harvard Magazine 89 (5):47-57.

Bransford, J. 2000. How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academies Press.

Csikszentmihalyi, M. 1990. Flow: The Psychology of Optimal Experience. 1990. Harper and Row, New York.

Thompson, C. J., and Dey, E. L. 1998. "Sources of Stress for African American College and University Faculty.” Journal of Higher Education 69 (3): 324-345.

DiPietro, M., and Fay, A. 2005. "Online Student-Ratings-of-Instruction (SRI) Mechanisms for Maximal Feedback to Instructors." Presented at the 30th Annual Meeting of the Professional and Organizational Development Network. Milwaukee, WI.

Dweck, C. S. 2007. Mindset: The New Psychology of Success: New York, NY: Ballantine Books.

Ezorsky, G. 1977. "Hiring Women Faculty.” Philosophy & Public Affairs 7 (1):82-91.

Glickman, C. D., Gordon, S. P., and Ross-Gordon, J. M., Ed. 2009. The Basic Guide to Supervision and Instructional Leadership. 2nd ed.Boston, MA: Pearson.

Hendrix, K. G. 1998. "Student Perceptions of the Influence of Race on Professor Credibility.” Journal of Black Studies 28 (6):738.

Jalomo, R. 2000. "Assessing Minority Student Performance." In Beyond Access: Methods and Models for Increasing Retention and Learning Among Minority Students, edited by S. R. Aragon, 7-18.San Francisco, CA: Jossey Bass.

Johnson, I. H. 1996. "Access and Retention: Support Programs for Graduate and Professional Students. New Directions for Student Services 1996 (74):53-67.

Lee, W. Y. 1999. "Striving Toward Effective Retention: The Effect of Race on Mentoring African American Students." Peabody Journal of Education 74 (2):27-43.

Martin, J. R. 1987. Reclaiming a Conversation: The Ideal of the Educated Woman: New Haven, CT: Yale University Press.

McGregor, D., and Bennis, W. G. 1960. The Human Side of Enterprise: New York, NY: McGraw Hill.

Olsen, D., Maple, S. A., and Stage, F. K. 1995. "Women and Minority Faculty Job Satisfaction: Professional Role Interests, Professional Satisfactions, and Institutional Fit.” Journal of Higher Education 66 (3):267-293.

Pink, D. H. 2009. Drive: The Surprising Truth About What Motivates Us. New York, NY: Riverhead Books.

Pinker, S. 2003. The Blank Slate: The Modern Denial of Human Nature. New York, NY: Penguin Books.

Popham, W. J. 2010. Everything School Leaders Need to Know About Assessment. Thousand Oaks, CA: Corwin Press.

Porter, N. 1994. "Female Faculty: Realities and Representation.” Women's Studies Quarterly Spring-Summer (22): 35-41.

Rubin, D. L. 2002. "Help! My Professor (or doctor or boss) Doesn’t Talk English. Readings in Intercultural Communication." Experiences and Contexts:127–137.

Sotello-Turner, C., and Myers, S. L. 2000. Faculty of Color in Academe: Bittersweet Success. Needham Heights, MA: Allyn & Bacon.

Stone, L., and Boldt, G. M. 1994. The Education Feminism Reader. New York, NY: Routledge.

Zachary, L. J. 2000. The Mentor's Guide: Facilitating Effective Learning Relationships. San Francisco, CA: Jossey-Bass.

About the Authors

JoAnn Franklin Klinker is an associate professor in Educational Leadership, College of Education, Texas Tech University.

Mary Frances Agnello is an associate professor in Curriculum and Instruction, College of Education, Texas Tech University.

Aretha Faye Marbley is a full professor in Counselor Education, College of Education, Texas Tech University.

Roseanna Currey Davidson is an associate professor in Special Education, College of Education, Texas Tech University.

Download PDF