Scientifically Based Research: Guidelines or Mandates for Product Purchasing?
##AUTHORSPLIT##<--->
This is the third article in a six-part series dedicated to taking a closer look at scientifically based research. In this article, T.H.E. Journal Editor-in-Chief Dr. Geoffrey H. Fletcher examines how the emerging SBR guidelines may affect school districts' choices of products. While federal officials claim the intent is "not creating a federal imprimatur," Fletcher points to New York City's experience in applying for Reading First funds as a possible future for other districts.
A key concern of educators - and especially of providers of products and programs - is whether or not a positive review by the What Works Clearinghouse (WWC) is a prerequisite for purchasing a specific product, particularly if federal funds are being used. As one technology coordinator asked: Are "ivory tower researchers and/or bureaucrats" going to make purchasing decisions by federal policy?
No, says Christine Wolfe, director of policy in the Office of the Undersecretary at the U.S. Department of Education. "The overall policy is that the department is not creating a federal imprimatur on curriculum or specific products and services," she states. "Instead, they are trying to provide a way of synthesizing what research says. They are not going to say, 'If it isn't listed in the What Works Clearinghouse you can't buy it with federal dollars.' "
Wolfe points out that some instructional areas such as reading have significantly more research in them than others. As a result, programs that oversee those areas may refer to the WWC more in their guidelines to states and districts than programs that administer other instructional areas with less research behind them. Thus, it is possible, if not probable, that different program areas will treat the WWC reports and SBR standards differently in their guidelines and requirements. In addition, the use of the WWC reports may vary, depending upon whether the monies are from a formula program or a competitive program.
WWC as a Resource
Wolfe acknowledges that policies regarding the use of WWC reports could change over time. However, Wolfe says that she "could not envision a program making legal requirements to use only products, programs, etc., which were vetted by the Clearinghouse." Furthermore, Wolfe consistently characterized the WWC as a "tool" and a "resource."
John Bailey, former director of the U.S. Education Department's Office of Educational Technology, concurred with Wolfe in an interview while he was still with the department. He noted that in Title II D, the technology section of the No Child Left Behind Act, there are no references to SBR. The education technology industry, led by the Software & Information Industry Association, convinced members of Congress that because there are such enormous changes in technology and its use in education, there has not been sufficient time to conduct SBR. Bailey said that a directive had gone around his department stating that funding for any product, program, practice or policy was not to be contingent upon vetting by the WWC.
The Case of NYC
Despite these statements from Wolfe and Bailey, the short history of SBR might suggest that the fears educators and vendors have expressed regarding a "federal purchasing policy" are not without some merit. The most visible incident where federal SBR policy and local purchasing decisions collided occurred on Jan. 21, 2003, in New York City, when Schools Chancellor J'el I. Klein announced a new reading program for all elementary students.
Klein's choice for the program, which would be at the core of his school reform effort for reading, was Month-by-Month Phonics in grades K-3, augmented by a districtwide policy to devote significant time to the practice of reading and writing. This announcement came early in the grant process for NCLB, before New York state had submitted its Reading First application.
Two days later, Dr. G. Reid Lyon, a researcher at the National Institute of Child Health and Human Development - as well as President Bush's top reading advisor and a contributing author to the Reading First component of NCLB - was quoted in The New York Times as saying: "We can find no published research indicating that this program has been tested with well-defined groups of kids and shown to be effective. And clearly one would want to know those kinds of details before incorporating any program into use."
Although Lyon, who neither works for the Education Department nor helps make decisions about grant applications, did not directly threaten the rejection of New York's Reading First application, the message was clear that the adoption of Month-by-Month Phonics could jeopardize a substantial portion of the $68 million in federal aid that New York state could get for Reading First.
On Feb. 25, Lyon was quoted in The New York Times again, this time saying that it was "unfortunate that we are making decisions about children's lives [on the basis of] untested assumptions rather than on being sure of what we know works for kids at risk of reading failure." The next day, Klein defended his choice once more, saying, "That's our program and we're prepared to protect it."
Yet, somewhere along the line, Klein backed off of his "no-prisoners" position. The New York Times reported on April 5 that Klein had added another phonics-based program to the reading curriculum - the New York City Passport program, developed by Voyager Expanded Learning of Dallas, which is a more intensive program for struggling readers.
So, while Klein kept Month-by-Month Phonics and claimed that he always intended to augment it with other materials, it was not until Month-by-Month Phonics was challenged that any word of an additional phonics-based program began to surface. Some educators theorize that he brought on a "sanctioned" Reading First program so as to not jeopardize federal funding.
Ultimately, however, Klein reversed himself completely. On Jan. 6, 2004, almost a year after the original announcement of the Month-by-Month Phonics choice, New York City schools announced that they were abandoning the program in 49 of its troubled elementary schools, presumably to ensure they received $34 million from Reading First programs. The new curriculum chosen for the 49 schools was Harcourt Trophies, a single comprehensive program, as opposed to the aggregation of multiple resources characterized by the prior approach.
Mandate by Fiat?
This episode in New York City has sent a clear message to districts that the Education Department may, if not dictate, at least strongly influence the actual materials that are purchased with Reading First funds. However, it remains to be seen the extent to which this might occur in other programs under NCLB. Even if vocal federal-level personages don't put outright pressure on localities' purchasing decisions, districts oftentimes interpret plans, visions or guidance from higher-up bureaucracies as mandates, especially if they perceive that money is tied to the proclamation.
As one former state official said, "If money is involved, states and districts do not want to jeopardize those funds, especially in this time of limited budgets." While Wolfe's belief that it would be unlikely for the Education Department to fund only those interventions vetted by the WWC may turn out to be true, the tendency of districts to overinterpret guidance into mandates may create a requirement by fiat.
Federal officials like Wolfe and Bailey caution that the What Works Clearinghouse is just starting up, and there's no evidence to draw any conclusions about the impact of its work at this time. Over the next few years, as the review process gears up, it will become clearer if federal programs, states and even local education agencies will use WWC evidence reports more as a resource or as an approved list.
Dr. Geoffrey H. Fletcher is Editor-in-Chief of T.H.E. Journal. He was a former Texas Assistant Commissioner with responsibilities for curriculum, assessment, textbooks, technology and professional development.
This is the third article in our exclusive six-part series, titled "A Closer Look at SBR," which was edited by T.H.E. Guest Editor Therese Mageau.