Policy | News

States Fail at Effective Use of Student Data, DQC Report Asserts

• By Scott Aronowitz
• 02/22/11

The Data Quality Campaign (DQC) last Wednesday released its sixth annual analysis of the data states collect on students and how the states use that data at the policymaking level to effect improvements in student achievement. The report issued by the DQC based on its state analysis, "Data for Action 2010, " concluded that, while states have made exceptional progress in collecting quality longitudinal data on individual students and their educational progress over time, many continue to lag when it comes to applying this data to making policy decisions that positively impact student achievement.

Founded in 2005 as a partnership between several administrative- and policy-focused organizations seeking to improve data use in state education policy, the DQC has identified several issue areas related to data-driven policy and has sought to develop a roadmap for state policymakers both to collect quality data and to use it effectively to increase student achievement. Today, the roadmap consists of two primary benchmarks: "The 10 State Actions to Ensure Effective Data Use" and "The 10 Essential Elements of Statewide Longitudinal Data Systems." The group's ongoing goal is to have all 50 states and Washington, DC apply all of the components of each of these lists to optimizing their systems of data collection and use.

"Data gives us the roadmap to reform," said United States Secretary of Education Arne Duncan. "It tells us where we are, where we need to go, and who is most at-risk. That cycle of continuous improvement cannot work unless states have good data and are willing to use it...."

According to "Data for Action 2010," however, the "10 Essential Elements" continue to present several states with significant difficulty. The survey concluded that 17 states are unable to link teacher and student data; 15 states do not collect course-taking information; and 11 states are unable to link K-12 and post-secondary data.

Leadership at Every Level
"The challenges are not technical in nature," maintained Elizabeth Laird, director of communications and external affairs for DQC. "They actually involve leadership and political will. Our state analysis [shows] that half of the country right now has the data they need to answer almost any policy question confronting states."

The report identified Maryland as one state in which policy leaders have successfully implemented the elements most critical to improving data collection and application. DQC recognized Gov. Martin O'Malley for instituting a program that brings stakeholders together to identify the education issues the state needs to address and the data and procedures necessary to address them.

John Ratliff, director of policy for the Office of the Governor of Maryland, oversees this program for the state's executive branch. He agreed with Laird, emphasizing that "leadership is absolutely critical. I don't think the people who control the variety of systems will necessarily want to work together, to collaborate. There are a lot of reasons for them not to in many cases. So you have to have leadership...."

The challenge, Laird continued, arises in getting state policy leaders to lead the drive in putting the data to work. "What states are not doing," she said, "is leveraging that data to make informed decisions, not just at the policy level, but in making sure that parents and teachers have the [necessary] information about their students and that students have information about themselves to make better decisions."

Ratliff also pointed to a tendency among state agency leaders away from collaboration and sharing of ideas, explaining that he had to coordinate with all the agencies involved to ensure that "this collaborating happened, that we were partnering and moving forward to integrate all of these systems. I don't think that happens naturally." He said he believes an integral aspect of Maryland's success was Gov. O'Malley's fostering of agency collaboration. "The governor proposed a structure in this state last year to govern our data system integration that included all of the key stakeholders, in our university system, in our K-12 system, in our workforce systems, [as well as] parents, principals, and university presidents. That governing board for our system is the place where the policy discussions are had, so there's kind of a permanent forum, and it's not a one-time conversation."

Ratliff also pointed to training as a critical but often overlooked factor in the effective application of data. "We don't just want to create this fancy system and not actually use it. It's not data for data's sake; we want to be able to use this at every level. For the policymakers at the state level, in both the legislative and executive branches, you want to provide training for them, [ensuring they know] what data is available, how to access it, and how to use it."

"The biggest obstacle [to progress] is the status quo," said Idaho State Superintendent of Public Instruction Tom Luna. "In education, there's a tremendous amount of resistance to doing things differently, particularly if there's not a new source of revenue to do something new."

Financial Issues
Luna was quick to emphasize the part the current nationwide economic difficulties played in erecting obstacles. "We're struggling financially like every other state. But it was incumbent upon us to not hunker down and wait until times got better to pursue meeting all of these 10 elements." However, he added, in addition to the natural resistance to change, "the arguments for slowing down magnified as the economy began to slip."

Gene Wilhoit, executive director of the Council of Chief State School Officers (CCSSO), which is a partner in the DQC, also cited financial concerns as a potential or actual roadblock for all states. "State budgets are dwindling causing a great deal of competition for scarce resources," he explained, "so unfortunately this may result in cuts to funding for necessary infrastructure, investment, and capacity needed to collect and use quality longitudinal data."

In Idaho, however, both teachers and local and state policymakers realized they had to tweak their attitudes toward financial issues in order to bring the nonetheless necessary change to fruition. "A student's need to improve academically isn't going to wait for the economy to improve," Luna said. "Kids are going to age, and we need this data if we're going to make informed decisions. We weren't going to wait for the economy to improve before we accomplished our goal."

From his vantage point as the top state education leader, he said, he knew he had to come down hard against any resistance to the project, and this meant using financial concerns as a tool rather than simply an issue. "We let districts and schools know that there was a date [by which] this information had to be accurate and verifiable in order for them to receive funding," he explained. "Anytime you tie anything to funding, you get people's attention. I don't care if it's my teenagers' allowance or funding for a government agency. When you tie accountability to receiving funding and then hold firm to that, that's step one."

External Obstacles
In addition to internal difficulties states face such as lack of leadership and collaboration and limited training, there are limitations over which the states have no control, specifically those established by the U.S. Department of Education. places a fair portion of the blame on regulations imposed by the Family Educational Rights and Privacy Act (FERPA).

"FERPA has a chilling effect on data sharing and use," said CCSSO's Wilhoit. "The law was written in the 1970s and does not adequately address the needs, abilities, technologies, and policy realities of today." He called for significant updates to FERPA to bring it in line with both the demands and the opportunities presented by 21st-century technology and policy requirements. "This includes clarifying and updating FERPA regulations so we can share data and protect student privacy at the same time. We also need to continue to invest in building systems that can make data actionable and easy to use for everyone from the policy maker and school administrator, to the classroom teacher, the parent, and the student."

Finally, said Wilhoit, "we need strong statewide policies and centralized data systems in order to provide equity to small, underserved, and rural schools that would not otherwise be able to implement systems. We also need to be able to track a student when she moves across schools [and] districts and even between states; students should not lose their educational history because of a move across town or country."

The Upside
In spite of some discouraging results, the DQC acknowledged a number of bright spots among the states, as well as their data leaders and policymakers. For instance, the group noted, in a single school year, Idaho went from adhering to three of the group's essential elements to all 10, for which Luna credited active participation on both the state and local ends of the process.

"I don't want to downplay the efforts of people at the local/district level," Luna said. "This has not been easy for them, but the vast majority have stuck with it, they've asked legitimate questions, they've been a part of finding solutions. I give a lot of credit to our local school districts, which even in tough economic times realize the importance of this project and have played a key role in getting us to this point."

And, he noted, state leaders and administrators held up their end, saying their job was "to make sure that questions are answered, that we have people on site if necessary to help districts work through their problems. We had numerous regional meetings and conference calls.... I think if we would've blinked, we wouldn't be talking about this today. We held firm and then provided assistance and resources to get us to this point."

Finally, in conjunction with the report's release, the DQC recognized three individuals with awards for their efforts to make student data collection more efficient and accurate and to use the data to greater lasting effect:

• State Policymaker: Maryland Gov. Martin O'Malley for his efforts to lead stakeholders in building robust statewide data systems that span early childhood to the workforce and protect data privacy, helping to ensure students graduate college- and career-ready.
• State Data Leader: Georgia Department of Education Chief Information Officer Bob Swiggum for developing an integrated, cost-effective statewide education data system that allows teachers and principals to easily access and use data to make informed education decisions.
• District Data Leader: Denver Public Schools Director of Assessment Technology and Accountability Jason Martinez for providing educators with access to data used for decision making.