Foundational, process, and outcome principles are essential components of principal evaluation systems.
By Joseph Murphy, Ellen Goldring, and Andrew Porter
Principal, January/February 2014
Web Resources

Educator evaluation is on the center stage of school reform, pushed there by both the larger accountability movement brought to the forefront by No Child Left Behind mandates and competitive grants such as Race to the Top. Although the spotlight has mostly been on teachers, increasingly, principals are being highlighted as well. To ensure that this particular school reform is well executed, core pillars of principal evaluation systems must be established. Although quality evaluation systems can take multiple shapes and include multiple components, they require two core anchors—robust guiding principles and the right content.

We draw our recommendations from a variety of sources and experiences, including the development of statewide principal evaluation programs in Delaware, Ohio, New York, Kentucky, Mississippi, and a consortium of 40 districts in Illinois, as well as the development of the Vanderbilt Assessment of Leadership in Education (VALED), which is currently in use in approximately 4,500 schools. In building VALED, we conducted reviews and original investigations on the state of principal evaluation in the U.S. and tracked lessons learned by seminal actors in this area.

The Current State of Affairs
Principal evaluation processes have remained largely unaltered over the past 30 years. Here is what we know. First, principal evaluation processes have not been developed from the best understanding of effective leadership or from the body of scholarship on school improvement. If one accepts a core law of measurement— that you get what you measure—this is not good news.

Second, while improving in some places, the process of principal evaluation in much of the country leaves a good deal to be desired. It is often perfunctory. It does not promote meaningful, improvement-based dialogue between principals and their supervisors. It has a purpose— meeting mandates—but it lacks viable goals and vitality. It is often devoid of objective evidence of performance, relying on very limited and often weak measures, such as ad hoc feedback from members of the community. Valid gradations of performance are conspicuous by their absence.

Third, information from the evaluation system is largely inconsequential and is not used to guide important decisions, such as promotions. It is unhinged from outcomes. That is, principal evaluation is generally decoupled from school improvement, professional growth, and personnel actions, such as incentives. In the long run, such evaluations are highly problematic for quality schooling. They too often provide a poor platform for school improvement, which we argue should be the endgame of the process.

The DNA of Productive Evaluation Systems
Here are some important lessons for building productive principal evaluations systems, gathered from our experiences in developing and piloting statewide principal evaluation systems.

Principles. Over the past 30 years, we have learned that there is a set of “essential principles” that are at the heart of school improvement (e.g., consistency, alignment, a relentless focus on challenging outcomes). In Visible Learning, the most comprehensive analysis of research findings ever compiled on educational effectiveness, John Hattie reaches the same conclusion about classroom improvement, writing, “It is less the ‘methods’ per se, than the principles of effective teaching and learning that matter.” Over the past decade, we have confirmed that this conclusion also holds in the field of principal evaluation, where there are three types of guiding principles: foundational, process, and outcome.

Foundational principles deal with the overarching frames of the evaluation— the “what.” An example of a foundational principle, for instance, is that an evaluation system be rooted in national standards for school leaders.

Process principles address the “how” of evaluation—for instance, that evaluations have defined timelines, include multiple measures, etc.

Finally, outcome principles speak to larger goals, such as school improvement. These remind us that evaluation is not an end in itself—it is a means for achieving ends.

The Guiding Principles for Principal Evaluation Systems table provides a list of these essential principles needed to anchor principal evaluation systems (see page 23). Unfortunately, it is rare to find evaluation systems that consciously identify and incorporate these principles in evaluation designs.

Content. If guiding principles form one strand of principal evaluation, knowledge about leadership for excellent schools forms the other. As noted earlier, the system is only as good as the “stuff” it assesses. The good news here is that we know what this content is: rigorous curriculum, quality instruction, and personalized culture, for example. And all of the VALED schools and states that we have developed principal evaluation programs for have noted that this content is captured in the national standards for school leaders as they come to life through professional development. In addition, strong academic programs should address the following:

• Curricular rigor, relevance, and alignment. This includes good teachers, effective teaching, and well employed monitoring and assessment systems.
• Building productive school culture. This includes personalized culture for students, professional culture for teachers, and a culture of engagement for parents and the community.
• Providing a center of gravity. This includes establishing vision, mission, and expectations, and coordinating and aligning activity.

We do not need to conduct a new search for the content of principal evaluation. We need to find ways to tap into that knowledge, and then to craft better ways to measure principal practice as it relates to that information.

Designing Systems
We have learned that it is a mistake to begin the process of building the evaluation system with a focus on finding the “right” components (e.g., goals, value-added test scores, observations). But it is, unfortunately, a common mistake. Successful systems can be built using different components, and, additionally, these components can differ from district to district. What we often find, though, is that “component” identification often trumps attention to appropriate content and almost always ignores critical guiding principles. Components are an important part of the principal evaluation narrative, but the essential work is to ensure that they grow from the two most critical dimensions of the system—content and principles.

Common Pitfalls
We have learned lessons about some of the specific “fashionable” components in principal evaluation systems. Components are categories that allow for targets to be framed, information to be brought to bear, and judgments made.

Unclear Rubrics. Components of evaluation systems can become “crutches” that do not yield important or significant information. We find this often to be the case with rubrics. Many that we see are little more than rigorous checklists. We have seen none that concretely link evidence and judgments, nor any that provide objective cut points to score evidence.

Overreliance on Observations. Recently, another favorite has been added to the principal evaluation design, i.e., direct observations of principals at work. There is no meaningful way to use a few discrete observations of a principal to make valid and reliable judgments. Thus, there is very little justification for including direct observation of a principal on any list of the top ways to gather evidence for assessing leadership performance or effects. Focus groups with teachers would be more valuable. So, too, would focus groups with students and/or parents. Spending time observing “schooling” in a principal’s building is more important than direct observation of the principal by a factor of ten.

The message here, as earlier with rubrics, is not to be seduced by the components of the systems no matter how popular. Components are only important to the extent that they effectively and powerfully incorporate the “right stuff” and honor “the guiding principles.”

Misuse of Achievement Data. Our work has also allowed us to conclude that more common sense and less bombast needs to be brought to the issue of using student outcomes as a dimension of principal evaluation systems. This element is weighted between 25 and 50 percent in the five statewide systems and the Illinois Consortium, for example. Similar percentages can be seen in principal evaluation systems throughout the nation. However, student outcomes cover a good deal of ground.

All the six state systems that we developed encourage the supervisor and the principal to create academic growth goals that make sense for the school in question. They can use state tests, district norm-referenced tests, end-of-unit examinations, advanced placement scores, college placement data, and so forth. There is no magic source of achievement data.

In the area of student outcomes, we have also learned that a singular focus on growth or gain in achievement is limiting. There are three questions that require attention here: What is the level of achievement? What is the gain in achievement? And what is the equitable distribution of achievement?

There are two messages we want to leave principals. Starting with the indirect point, almost everything we have reported about principal evaluation applies to teacher evaluation, as well. Because principals are in charge, in this realm, our hope is that principals will start to view this responsibility with a more refined lens.

Second, in many cases, principals have a voice in how the evaluation systems that impact them are created and implemented. Armed with an understanding of the essential components of successful systems, principals can have a positive impact on the processes in principal evaluation systems in their districts and throughout the nation.

Guiding Principles for Principal Evaluation Systems

Foundational Principles

• Highlight learning-centered leadership.
• Be grounded on the national standard for school leaders (ISLLC ).

Process Principles

• Be evidence based.
• Have set benchmarks agreed upon in advance.
• Be transparent.
• Foster a culture of collaboration between principal and supervisor.
• B e valid and reliable.
• Be comprehensive, but not overly complex.
• Be both formative and summative.
• Include multiple measures, including student achievement.
• Tap into the views of multiple constituents.
• Have well-defined timelines.
• Provide ongoing feedback to the principal.
• Be site specific, connected to the needs of the specific school.
• Be flexible enough to allow for adjustments.

Outcome Principles

• Promote school improvement.
• Enhance academic and social learning of students.
• Motivate principals to improve.
• Promote targeted professional growth opportunities.
• Result in meaningful consequences.

Joseph Murphy is a professor of education in the Department of Leadership, Policy & Organization at Vanderbilt University.

Ellen Goldring is a professor of education in the Department of Leadership, Policy & Organization at Vanderbilt University.

Andrew Porter is the dean of the Graduate School of Education at the University of Pennsylvania.

Copyright © National Association of Elementary School Principals. No part of the articles in NAESP magazines, newsletters, or website may be reproduced in any medium without the permission of the National Association of Elementary School Principals. For more information, view NAESP's reprint policy.