ISKME Special Series Part 4: USING DATA TO IMPROVE INSTRUCTION

• 05/01/06

##AUTHORSPLIT##<--->

T.H.E. Journal, in partnership with the Institute for the Study of Knowledge Management in Education(ISKME), prepared this special four-part report on data-driven decision-making.

Indicators & Outcomes

Connecting the dots: how to use data to get fromdistrictwide goal-setting to school-level performance.

SPURRED BY FEDERAL and state mandates for improved performance, school districts across the country have become adept at establishing districtwide goals to raise student achievement. They have identified learning gaps among various student populations and now seek to close them. They have targeted improved performance in reading or math by grade level. Some districts have taken the additional step of aligning their human resources and finances to better achieve their student achievement goals. And most have engaged a broad community—including board members, parents, teachers, principals, business leaders, and others—insetting their strategic direction.

But while many districts have established effective processes for goal-setting, fewer have gotten traction in connecting those goals to what’s actually happening within schools. Establishing districtwide and community-driven goals for high student achievement is an important step on the way to improvement. But without some real ways to align district objectives to what the schools are doing, those lofty goals inevitablylead to action that does not produce significant results.

Drilling Down to the Schools

Albuquerque Public Schools (NM) has established several ways to link its annually updated districtwide goals for student achievement to school-level improvement. As a large urban district with more than 87,000 students (52 percent of whom are Hispanic), the district has established 12 clusters that provide a decentralized yet systematic process for feedback andsupport for schools.

Each cluster is made up of one of the district’s 11 high schools and its feeder schools. Each is lead by a principal from one of its participating schools and is supported by an instructional assistant, a program specialist, and other instructional coaching staff. A 12th cluster is made up of the alternative schools. Each cluster is responsible for overseeing the development of student achievement and other objectives at each of its school sites, and for ensuring that the objectives supportdistrict goals and priorities.

“One of the district goals is that all schools will meet or exceed Adequate Yearly Progress,” says Nelinda Venegas, associate superintendent for the Albuquerque cluster system.“How they’ll get there is individual school work.” But thecluster system, she explains, provides the schools with thesupport and feedback they need to engage in a continuousimprovement effort that is connected to overall district goals.

“I have described it like this,” Venegas says. “Here are the district goals. Here are the cluster goals. Here are the goals for each of the cluster schools. Here are the goals of the teachers in the departments (for schools that have departments). And here are the goals of each teacher. It’s an overall system of continuous improvement, and it’s all built around the standardsand toward the student.”

A similar process is at work at the Aldine Independent School District in Houston. Aldine has also established routine systems that use district goals to drive improvement efforts appropriate for each school. Aldine is a medium-sized district with 56,000 students, of whom about 61 percent areHispanic and 32 percent are African-American.

Aldine ranks among the state’s high-performing school districts, according to data from the Texas Education Agency. The district has earned seven recognized ratings since 1996, and was one of five 2004 and 2005 national finalists for the Broad Prize for Urban Education, which honors urban school districts for improving student achievement and is the largest educationaward in the country given to a single school district.

“Our first and most important objective is growth in student achievement,” says Wanda Bamberg, assistant superintendent of curriculum and instruction. “Each of the schools has to determine, in relation to that, what are the specific areas of improvement it will work on each year, and what specific objectives will it set in order to reach those goals.”

Each Aldine campus is required to use student performance data to set its priorities, based both on the district’s goals and its own patterns of success. And as in Albuquerque, the drilling down of goals extends to teachers, who are required to meet to share strategies and priorities. “Each campus looks at where they were last year and where do they want to be this year,” Bamberg says. “Then a group of teachers will look at academic growth more specifically.”

Ongoing Monitoring

While it’s important to synchronize school-level objectives with district goals, the constant monitoring of student performance is what generates the strongest link between the two. For decades, schools have developed plans that focus on improving student performance. But it’s the analysis of results that engages people in change.

“What used to be a school improvement plan was checked once at the end of the year,” says Albuquerque’s Venegas. “There was no monitoring. Now we monitor and report three times per year based on the data. We perform site visits. We have the service team, the cluster leader principal, and theinstructional assistant go out to the schools.”

Aldine’s Bamberg agrees that the ongoing monitoring of results is crucial to driving improvement. “We look at data all year long,” she says, “not just at the end of the year.” She emphasizes the importance of “making sure that the goals and action plans are truly acted upon and not just written down.” In Aldine, “the goals must have practical actions explained in an action plan,” Bamberg says, “and then the plans must be monitored through the use of scorecards. The scorecards tell us whether or not we have met our goals. And if we have not, what is the variance? How far are we from reaching the goal? Then we return to the action plan andadjust it to make sure that the goal is met.”

Aldine has invested extensively in professional development for principals and teachers in the use of these tools and processes. The training has included instruction in how to update and review campus improvement plans, how to write action plans to guide and bolster specific areas of improvement, and how to use scorecards to continue to examine performanceresults in relation to objectives.

Typically, an ongoing self-monitoring system, or continuous improvement process, needs to include the following keyelements:

• Agreement on important goals.
• Performance data that breaks down progress toward those goals.
• Diverse teams brought together to analyze the data.
• Action plans or other interventions to improve results.
• The gathering of more data to continue monitoring effectiveness, thereby completing the feedback loop and renewing the cycle of learning and improvement.

At the same time that the feedback loop—self-monitoring, analysis, and action—connects individual school performance with the district’s broader goals for student achievement, it also provides important information to teachers. According to Bamberg, “The closer down you get to students, the more frequent the monitoring is, so that it’s every three weeks for the teachers. Every third week, the students will be given a common grade-level assessment. And theteachers will train together so that their instruction is aligned.”

A Two-Way Exchange

As does the Albuquerque Public Schools, the Boston Public Schools (BPS) uses a cluster system to systematize the decentralization of districtwide goals in ways that make sense for individual schools. The district, which has about 58,000 students (44 percent African-American and 33 percent Hispanic), has seen significant increases in the percentage of 10th-gradestudents passing the state tests in math and English.

The district is organized into triads that represent three geographical regions of Boston. Each triad has three clusters of schools, which makes for a total of nine clusters across the district. Like Albuquerque, each cluster includes high schools, middle schools, and elementary schools, and each cluster is led bya principal.

The principals, according to Chris Coxon, deputy superintendent for teaching and learning, act as peer leaders for each other and as “a voicefor the cluster to the superintendent.”

The BPS superintendent, Thomas Payzant, has created a leadership team that includes not just central administrators, but also the nine sitting principals who double as cluster leaders. The cluster leaders, Coxon says, “have monthly meetings with the schools that are in their clusters. And every two weeks, they meet with the superintendent through the leadership team. So there’s agive-and-take and an information flow up and down.”

This two-way exchange of information lies at the heart of an effective self-monitoring system of improvement. For example, Boston has been focusing on boosting the achievement of children with special needs. The leadership team first analyzed data from throughout the district and identified schools that were doing well. “Then we did walk-throughs to find out what they were doing well,” Coxon says, “to see what are the good practices in BPS [concerning] educating students with special needs. From there we came back to figureout how to spread that out across the schools.”

Each triad identified 10 schools that, according to data on student outcomes, had significant populations of children with special needs who were not performing well. Site visits to those schools allowed the triads to gather qualitative data. And the district completed an asset map of teachers of specialneeds children, inquiring about the training they had, the groups of children they excelled in teaching, and the gaps in their training versus what they need to know to effectivelyteach special-needs kids.

“We provided a lot of support to teachers so they could have a significant impact on student achievement,” Coxon says. The district connected that support to the results of their datagathering efforts. “We’ve been doing this work for one year, and we’ve had early indicators of success. Next year we will add anadditional 15 schools.”

The Role of Technology

Technology can help connect districts and schools—so long as leaders leverage it as a tool for improvement rather than expecting technology to drive improvement. Boston has created a MyBPS Web portal that it has developed over time as a way to share information with teachers,parents, students, and other stakeholders.

“We [in public education] still haven’t figured out what is the right system to communicate well with everybody,” Coxon says, adding that it can be difficult for districts to determine how much data to provide to schools without overwhelmingthem.

It’s not about pointing fingers. It’s about allof us working together to improve studentachievement and make kids successful.

Nelinda Venegas, Albuquerque Public Schools

MyBPS helps address this problem by providing information to key stakeholders—when they need it. For example, the portal allows for easy access to data about student achievement. Users can now log on to the site and find out how studentsare performing in specific areas.

The Aldine school district uses a technology interface developed by Austin, TX-based Triand that allows district administrators, principals, and teachers to pull the data they need. “It allows us to disaggregate not just statedata,” Bamberg says, “but all the teachers’ tests.”

For tests developed by teachers at the local level, the assessment results have to be scanned in. But teachers can then review the results on a computer in their own classrooms, usually within 45 minutes, Bamberg says. They can then see which specific areas are presenting students with difficulties. For example, the software will perform an item analysis, which tells the teachers which test problems the students missed, so they can revise their instruction in that area andspend more time on it.

A Change in Climate

The systematic processes school districts are now using to gather feedback and conduct analysis are only half of the effort to improve student performance. The other half is environmental. New processes are great, but they can’t flourish unless changes occur within a school climate thatpromotes trust and achievement.

“One of the key levers for improvement is the quality of leadership in these buildings,” says Coxon, speaking to what is transpiring in his own district. “The cluster leaders were picked not because they said yes, but because they challenge us on things. They don’t think just about their school, but systemically about what are the supports that their cluster, their triad,and their district need.”

Similarly, the continuous improvement processes described by Venegas in Albuquerque have led to a change in perspective. “In the old system,” she says, “you got a directive from the central office, and you were supposed to do it. Now the schools participate in a process. They can see the importance of all the stakeholders: The teachers need support; the principals need to be involved; parentsneed to be included.”

“It’s great when I walk down the hall and I see a terrific teacher,” Venegas says. “But across the hall, maybe there’s not a terrific teacher there yet. In the old system, teachers didn’t have to talk to each other. Now they’re required to work together to build grade-level plans and other collaboration. Now that they’re participating in a system that provides them with support, they realize they have more hands-on time withthe kids.”

“If everybody has buy-in,” she continues, “then everybody has a lot to gain and a lot to lose. It’s not about pointing fingers. It’s about all of us together working to improve student achievement and make kids successful.”

Read each article of the four-part ISKME Special Series: Part 1, Part 2, Part 3

Thad R. Nodine and Lisa A. Petrides are vice president and president,respectively, of ISKME, a research institutein Half Moon Bay, CA.

This article originally appeared in the 05/01/2006 issue of THE Journal.