Turn research into practice by discovering the most relevant sources to help improve your school.
By Scott Bauer and David Brazer
Principal, November/December 2012
Web Resources

These days, information overload seems to be our normal state of existence. School leaders are inundated with descriptions of best practices and programs that work. They are exhorted to use evidence in all facets of decision-making and to employ research-based strategies to improve schools. Of course, research findings are seldom definitive, and sometimes contradictory. Thus, putting research into practice in school improvement efforts isn’t easy, especially for those new to the principalship. The following ideas, designed to help principals learn how to understand and apply research to the instructional challenges schools face, are based on our book, Using Research to Lead School Improvement: Turning Evidence into Action.

Types of Sources

Research is typically written by scholars for schol­ars, which does not always yield easy reading. But knowing how different types of sources serve different purposes can help make your reading more efficient. Primary sources that report data collection and analysis are generally available through scholarly journals or via websites of research sponsors or think tanks.

Primary sources often are quite technical in nature, so your initial investigation of research may be more fruitful if you explore secondary sources. They synthesize published research to organize and simplify a field or specialty. Trade magazines and associations typically publish synthesis articles, which are especially helpful in connecting research to practice, and thus may be worthwhile places to start a collection of materials relevant to your interests. Tertiary sources, such as encyclopedias or textbooks, are even more refined and reduced for quick refer­ence. Using secondary and tertiary sources will help you to understand more clearly where you want to probe deeper.

To build a thorough understanding of a topic, it is worth your time to track down primary sourc­es for several reasons. First, no synthesis can pro­vide all the details you might need to understand a given topic. Original sources will describe the research setting, participants, and limitations of the work presented, and help you decide how relevant the findings are to your situation. Pri­mary sources also will include detailed reference lists, a source for you to identify other, potentially more relevant, work.

The Library Is on Your Desktop

Today’s information technologies place an unbelievable wealth of knowl­edge at your fingertips. This luxury comes with some peril, of course—access to Internet resources brings new meaning to the term caveat emptor. In addition to using guiding questions and keywords to find information on a particular topic, there are other guide­lines that will help you make produc­tive use of the Web.

Access electronic databases relevant to your purpose. If you are beginning to learn about a problem or topic, you likely will want to start with a free search engine such as Google Scholar (http://scholar.google.com) or the Educational Resources Information Center (ERIC, www.eric.ed.gov). If you are seeking research knowledge on solutions, you might want to look into the What Works Clearinghouse (http://ies.ed.gov/ncee/wwc), a website sponsored by the U.S. Department of Education.

Identify a small set of initial sources that appear on target for your pur­poses, and skim them. Refine your keyword list for subsequent searches. Mine the bibliography for additional sources, and to familiarize yourself with prominent scholars in the field.

Draw a diagram showing how the major ideas or themes fit together. A graphic representation will help you refine your understanding of how the research relates to your situation.

Being a Critical Consumer

Finding sources is the first part of the research challenge. Evaluating the worth of the sources you find is a criti­cal next step. Above all, you should triangulate for trustworthiness, which means seeking multiple sources to confirm an idea or assertion. You want to find a variety of quality stud­ies that come to similar conclusions to ensure trustworthiness through replication or reinforcement.

While there is no sure-fire process guaranteeing that you always know the good from the not-so-good, there are some approaches you can take that will help you. The following questions—outlined in Critically Analyzing Informa­tion Sources—will help you to assess the usefulness and quality of your sources.

What are the author’s credentials, background, and expertise? Is the author an employee of a think tank or advocacy group, on the staff of a research lab, or a faculty member at a top-flight university?

What’s the publication date? Recent­ness is important, especially if you are considering the efficacy of a program or strategy you might adopt. Even if the approach has been around, more recent work may better inform you about its worth.

Who’s the publisher? Prominent journals, universities, and indepen­dent publishers are generally trust­worthy sources. Conversely, advocacy groups and obscure journals might present biased or poorly researched information.

What’s the tone of the piece? Is it relatively objective in the presenta­tion of a problem and findings, or does it appear to be advocacy or sales oriented?

Reading Research

Now that you have collected research papers that appear to be on target, how do you make sense of them? To get started, become familiar with the typical format of empirical research.

To determine if a study is relevant to your needs, skim the introductory material: the statement of the problem, purpose, and significance. Therein, you should learn what the paper is about and why the author believes the work is important. When the study includes research questions, these will foreshad­ow the author’s findings. The conceptu­al framework or review of literature will place the study in the larger scope of related research, which also may suggest additional sources for later inquiry.

The section readers love to skip—methods—details how the study was conducted, including a description of the research design, participants, types of data collected, analytic proce­dure, and, the limitations of the study. This section contains information that is important for you to consider.

There are three broad categories of design: quantitative, qualitative, and mixed methods. In Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualita­tive Research, John W. Creswell defines quantitative research as “a type of educational research in which the researcher decides what to study; asks specific, narrow questions; collects quantifiable data from participants; analyzes these numbers using statis­tics; and conducts the inquiry in an unbiased, objective manner.” And Creswell defines qualitative research as “a type of educational research in which the researcher relies on the views of participants; asks broad, gen­eral questions; collects data consisting largely of words (or text) from partici­pants; [and] describes and analyzes these words for themes.” Mixed meth­ods combine approaches.

There are many specific research designs, each with strengths and weaknesses. To gain a deep under­standing of a topic, you may wish to read studies that employ a variety of research designs.

A second important part of meth­ods has to do with participants. Does a study involve schools like yours? Stu­dents like yours? Settings like yours? What are the differences that might affect the applicability of the findings to your school?

Quantitative studies often employ statistical tools for hypothesis testing that are referred to as inferential sta­tistics. Statistical significance describes the probability that observed differenc­es or relationships are due to chance. Researchers test to see if a statistic achieves or exceeds a critical value that corresponds to a “significance level” or probability that the observed outcome was random. Commonly used signifi­cance levels are 0.05 or 0.01, meaning that we would be either 95 percent or 99 percent confident in the assertion that the resulting statistic describes a true relationship.

An alternative metric referred to as practical significance or “effect size” also may be used to gauge results. Effect size tells you the strength of an outcome, as a small, medium, or large effect. This informa­tion is critical to your assessing the import of a finding. Results that are not statistically signifi­cant or have very small effect sizes may not be meaningful.

The evidence pre­sented in qualitative designs (such as case studies) is words rather than num­bers, generally in the form of a presentation of themes supported by verbatim quotes from research subjects. Qualitative work and mixed methods studies offer research­ers an opportunity to answer “why” and “how” questions by seeking to understand what subjects think, feel, or believe. Confidence in results is achieved through exhaustive analysis of interviews, observations, and docu­ments, and the elimination of plau­sible alternative explanations.

The final sections in published jour­nal articles present the findings and discussion. In this section, the author will present evidence and draw conclu­sions based on the research questions posed. It is also common for authors to discuss the implications of the work—for both research and practice—as well as limitations of the work.

Applying Research Knowledge

As we write in our book, “when con­fronted with a substantial, persistent student achievement problem, the search for understanding and meaning­ful responses to that problem should motivate us to want to understand the specifics of relevant research.”

Although we have offered some guidance through the jungle of research literature, and tips for under­standing published work once you identify it, we recognize that finding trustworthy research is not your goal—using the knowledge you have gained to help improve your school is what’s truly important.

It is impossible to pinpoint how best to apply research knowledge without consideration of a variety of factors such as your school’s context and culture, but we advocate thinking for­ward by anticipating what will happen as you move toward implementation of your plans. Here are a few ques­tions you might ask to help move from evidence to action:

Whose participation do you most need to make the change you’re con­templating? Ideally, you involved at least a representative sample of school staff in your planning already, but it is worthwhile to consider the ques­tion in terms of implementation. Who is most likely to be affected by the change? What are the consequences they might experience, and how can you help your staff deal with them?

What’s potentially unchangeable in your school that might serve as a barri­er to success? Think about both struc­tures and school culture. Are there rules, procedures, or beliefs that are likely to stand in the way of progress?

Who is most likely to embrace “the way we do things around here,” and thus resist change? What will their complaints be?

What skills and abilities will stake­holders need to make the change work? Anticipating staff development needs is critical to success of most changes.

Successful implemen­tation is predicated on having a clear purpose, providing resources nec­essary for the change to succeed, and anticipat­ing, as best you can, the consequences of change. From our understanding of theory and research on organizational change, what is perhaps most critical is that those who are key to implementation agree that change is necessary, and that the path you have cho­sen is sensible. Presenting what you’ve learned through examination of the research, in a brief and easily digestible fashion, can help you build your case in this regard.

We know that your time is a scarce resource, and hope that our sugges­tions will help you and your leader­ship team discover efficient and effec­tive ways to use research, contributing to significant improvements in teach­ing and learning in your school.

Scott Bauer and David Brazer are associate professors in the Education Leadership Program at George Mason University.

Your comments are always welcome, so send us an email at publications@naesp.org to let us know what you think about this issue.

Copyright © National Association of Elementary School Principals. No part of the articles in NAESP magazines, newsletters, or Web site may be reproduced in any medium without the permission of the National Association of Elementary School Principals. For more information, view NAESP's reprint policy.