Effective data system implementation is as much about leadership as it is about technology.
By Vincent Cho, Ann Allwarden, and Jeffrey C. Wayman
Principal, March/April 2016

Only a few decades ago, there were relatively few tools available to help teachers analyze and interpret student data. Today, the supposedly easy solution is to adopt computer data systems. Data warehouses, assessment systems, and other technologies promise to instantly aggregate and analyze data for teachers. In the hopes of supporting teacher decision-making, such systems offer to pinpoint students’ most pressing needs. Some even recommend lesson plans and ways to group specific students during instruction.

But buying new technology isn’t enough. Even the best systems can go underused or rejected. The cost of a failed data system is measurable not only in terms of dollars wasted, but also in lost opportunities to improve student learning. Such failures are avoidable. Our advice, informed by our experiences as school leaders and researchers, is to see beyond the usual “technology problems” such as computing power or design. Instead, leaders must also see the “people problems” that can derail data system implementation.

When Things Fall Apart
For most leaders, the technology problems of system implementation are predictable. Leaders can intuit that when selecting a new data system, they’ll want one that is easy to use, offers useful functions, delivers information in salient and timely ways, and is populated with rich, clean data. Teachers will need some basic training with the new system, such as how to log in or generate reports. In contrast, the people problems of data systems are hard to predict. They sit in the blind spot of our assumptions—that spot where we think that having the “right data” and “right technology” are enough.

Over the years, we have seen a variety of people problems derail system implementation. In one district, we saw a technology vendor simply stop returning phone calls or attending meetings with the district’s technology department. Worse yet, the technology department didn’t let district leaders know how bad things were getting until too late. On its own, the technology department was overloaded with work, lacked the expertise to solve many of the issues, and was unable to adequately train teachers. Elsewhere, we have seen principals who so distrusted their districts’ official data systems, that they funded their own, parallel systems. This was costly in terms of finances, as well as work. Staff had to enter data twice: once for the district and once for the school.

People problems can also occur at the school level. Principals who haven’t developed a strong, schoolwide vision for data use will find their teachers accessing only a sliver of the potential offered by new systems. This is because beliefs about how and why to use systems come from local vision, not from technology. Moreover, teachers will also need time and support to learn and apply new teaching practices, informed by data. Too often, principals ask teachers to look at data, but fail to help them figure out next steps in the classroom.

When systems go underused, district leaders and principals can find themselves stuck in a blame game. For district leaders, it is easy to feel frustrated that schools seem to be dragging their heels on performance goals or naive to the powers of the latest data system. For principals, it is easy to also feel frustrated, if not threatened, by district leaders’ reactions. Central office expectations might seem unreasonable or out of touch. The things holding system use back might seem a mystery.

In contrast, effective data system implementation is as much about leadership as it is about technology. It involves promoting communication, positive relationships, and common understandings.

Guiding Questions About Data Systems and Work
The following questions are designed to promote dialogue about data systems and educational work. Imagine using them when working with teachers and staff—or even at the district level.

What is our vision for teaching and learning? Addressing this question goes beyond simply reciting a mission statement or slogan. Conversations should sincerely explore beliefs about who we are, what we do, and why and how we do it. Leaders should exercise caution about thinking that these questions are easily or permanently answered.

We have witnessed schools where leaders incorrectly assumed that teachers were on board with the district vision. They were unaware that teachers rolled their eyes, felt browbeaten, or even laughed the vision off. In contrast, creating and communicating a vision gets at the heart of educators’ values, trust, and commitment to work. Vision doesn’t need to be empty. Principals can help teachers reignite their imaginations, connecting their everyday work to a bigger picture.

What is our vision for how data support teaching and learning? Do not assume that everyone knows what is meant by “data.” Pursuing this question with teachers, it quickly becomes evident that data means different things to different people.

Different groups of educators have access to different kinds of data, as well as different experiences with each data type. Thus, it is likely necessary at some point to group educators according to grade levels or subject areas. These conversations should involve evaluating the properties of particular data and their potential uses. Although much of the discussion might be about available data, leaders should also be prepared to ask whether currently available data are enough. Planning backwards from the vision, what are the data you really need?

How might we use the data system to support our vision? Worthwhile dialogue about this question requires teachers to have a strong knowledge of their data and data systems. Thus, a preparatory step might first involve exploring how to use the system in everyday work, allowing ample opportunity for reflection with colleagues about what is or isn’t effective with students. Because new reports, displays, or practices might need to be explored a little at a time, this preparatory step might take several months or even a year.

To what extent do we actually use the data system? This question encourages participants to reflect about use and barriers to use. Example follow-up questions might include: Who is (or isn’t) using the system? Why (or why not)? How frequently (or during what periods) do people use particular functions the most? Are central office departments coordinated in their support of data use?

Although responses to these questions serve as ways to evaluate the progress of implementation, they also can inform expectations about how much use is reasonable, as well as timing. For example, certain reports might be very informative at the opening of the school year, others might be used all year round, and yet others might be most valuable at punctuated periods in the year.

What knowledge and skills do we need to use the data system effectively? This question leads teachers to evaluate their progress toward being effective users of data. It’s a hard look in the mirror, but it’s one that can lead to important insights.

For example, this can be an opportunity for teachers to take charge of their professional learning, speaking up about what they really need to be better data users. Answers to this question might even change over time as educators become more experienced or as new educators join the district. This is a question that can be revisited every year.

How will we view mistakes and failure? This question is at the center of leadership and change. Mistakes will happen; some efforts will fall short. How should educators think about and react to those mishaps? In some districts, educators are told that failure is not an option. In contrast, the best organizations treat failures as opportunities for learning and improvement. Encouraging healthy responses to mistakes and failure is one way to support overall district improvement and commitment to change.

Planning the Reflection Process
These questions are geared toward leaders at the building and district levels. They address issues that principals and central office staff need to keep in mind as they support data use.

Who should talk through the guiding questions? Answers to this question should go beyond the usual suspects (i.e., just principals and teachers). For example, involve the community in discussions about data’s role in teaching and learning. This might provoke thought about academic and nonacademic data (e.g., school culture, bullying, socio-emotional development) that might be important to schools.

Further, we caution against writing off certain people as having nothing to say about data. In particular, we are often surprised when technology departments lament having been left out of important conversations about data systems. Other departments or offices might also be able to make important contributions, if only we ask.

When and where will these discussions take place? Educators think differently in different spaces and contexts. It can be energizing to get away from the usual spaces or conversation partners. For example, conversations about vision might be held on a college campus or during a reflective time (e.g., summer). Conversations about particular data might mingle together teachers from different schools. Social media might also be used to establish forums for creating momentum or exchanging ideas during this process.

What protocols will be used for eliciting dialogue and taking real action? In The Power of Protocols, Joseph McDonald and colleagues share a wealth of approaches for helping teachers share, reflect, and listen to each other. Although exceptionally useful, a stumbling point for many districts is stopping at “just talk.” In too many districts, ideas get lost, action never happens, and conversations run in circles back to old gripes. The test of leadership is to ensure that ideas turn into action.

Conclusion
Good technology is necessary for effective data use—but it is not sufficient. Although the hope is that new technologies will simply “plug in” to existing practices, the truth is that data system implementation requires addressing a variety of the people issues.

Teachers need help adjusting to ideas and routines around how the system is used, how it interacts with work, and how it supports the overall vision of teaching and learning. In this way, leaders will be on the road toward ensuring that their teachers are both better prepared to know about their students, as well as to act on what they know.

Vincent Cho is an assistant professor of educational leadership at Boston College.

Ann Allwarden is a principal in the ConVal School District in New Hampshire.

Jeffrey C. Wayman is president of Wayman Services, LLC.

Copyright © National Association of Elementary School Principals. No part of the articles in NAESP magazines, newsletters, or website may be reproduced in any medium without the permission of the National Association of Elementary School Principals. For more information, view NAESP's reprint policy.