What's Causing Large Urban Districts To Struggle with Science?

• By Scott Aronowitz
• 02/28/11

Students in large urban school districts are lagging behind the nation in their grasp of natural sciences, according to a special report on science education in 17 of America's urban districts. Experts involved in the report placed part of the blame on the lack of any clear or meaningful national science strategy, though several other factors, they acknowledged, contributed to the disparity.

All of the districts involved in the report were participants in the Trial Urban District Assessment (TUDA), the results of which were released Thursday. TUDA is a supplemental program of the 2008-2009 National Assessment of Educational Progress (NAEP), overseen by the National Assessment Governing Board (NAGB). Its aim was to shed light on how urban districts compare with one another, with the student population at large, and with standards set by educators and educational experts for varying degrees of scientific knowledge and education progress at the fourth- and eighth-grade levels.

TUDA Results: Urban Districts Versus the Nation
At a forum Thursday morning at the University of Massachusetts Boston, NAGB members who work in a variety of capacities in education presented the results and offered their insights into what the data means for the cities affected. Jack Buckley of the National Center for Education Statistics delivered the results, which the panelists said they found to be underwhelming overall.

"Results for cities are very disappointing," said Alan Friedman, NAGB board member and former director of the New York Hall of Science. "There's also an interesting achievement deficit in science as compared to ... math and reading.... There's an even greater deficit [compared with the respective states and the nation] in science. These results are shouting at us, 'Whatever we're doing in the big cities, we're not doing well enough.'"

The results were gleaned from science assessments administered to between 900 and 2,200 fourth- and eighth-grade students in each of the following public school districts: Atlanta; Austin, TX; Baltimore (City), MD; Boston; Charlotte-Mecklenburg, NC; Chicago; Cleveland; Detroit; Fresno, CA; Houston; Jefferson County (Louisville), KY; Los Angeles; Miami-Dade, FL; Milwaukee; New York; Philadelphia; and San Diego. The assessments included several fundamental and a few more advanced scientific principles for each grade level, as well as questions that tested applications of many of these principles, and others that sought to determine whether students were capable of employing scientific inquiry. The tests were scored on a 300-point scale, with the following delineations:

The assessment framework, including questioning styles, topics and subtopics addressed, delineations of capability, and all included questions, was designed by a diverse panel of education professionals throughout the United States over a two-year period. Because NAEP employed a new TUDA framework for 2009, comparisons with the previous (2005) assessment are unavailable. The group, however, indicated plans to use the same framework in the assessment planned for 2013 in order to arrive at accurate comparisons.

Among the key findings were:

• The overall national average scores for both fourth and eighth grades was 149; students in TUDA districts averaged comparably lower scores, with only three districts--Austin, Charlotte, and Jefferson County--achieving average fourth-grade scores comparable to the national average, and only Austin achieving comparable eighth-grade average scores.
• The three largest minority demographics--African American, Hispanic, and Asian/Pacific Islander--had national averages of 127 (grade 4) and 125 (grade 8), 130 (grade 4) and 131 (grade 8), and 160 (grade 4) and 159 (grade 8), respectively; TUDA districts saw lower average scores, in comparison to national scores in their respective demographics, for all three groups in both grades.
• Overall scores for minority demographics notwithstanding, African-American students in seven districts achieved scores on par with the national average for the demographic, and, in two districts, Boston and Charlotte, averaged higher scores; and Hispanic students achieved scores comparable to the national average for the demographic in 10 of 15 districts, with two districts, Atlanta and Baltimore, not having sufficient sample sizes for adequate data for the demographic.
• Among students eligible for national school lunch program, which indicates lower-income families, the national average scores were 134 (grade 4) and 133 (grade 8), with such students performing worse in TUDA districts overall.

"On the average there's a very low level of achievement as compared to the states and as compared to the nation as a whole, of our big city schools," Friedman said. "Forty-four percent of our students in the fourth grade fall below even the basic level. At the eighth grade, 56 percent of students failed to reach even the basic level. And basic is basic. In this case, [it means] only partial mastery of the things kids need to know and need to be able to apply in school, in their eventual jobs, and just to live in the 21st century."

Findings and Analysis: What's To Blame?
One finding Friedman pointed out as surprising was a discrepancy in scores between students learning from traditionally trained instructors and those who took non-traditional routes, such as Teach for America and similar programs, in their training. Students of the latter, he noted, "scored more poorly, even though [the teachers] are bright, they're young, they often have very strong backgrounds in the subject matter." He acknowledged, however, that the data may not tell the entire story. "It could be that the alternate-route teachers, because they're younger, newer, don't have seniority, they're being appointed to classrooms with lower-income students. And we sure know that lower income correlates with lower scores."

Arthur Eisenkraft, director of the Center of Science and Math in Context (COSMIC) at the University of Massachusetts Boston, said the NAEP science assessments shed light on a pervasive problem in K-12 science education. Because the tests required capabilities beyond the mere memorization of principles and concepts, Eisenkraft said, students who do poorly still fare well on state and local exams.

Michael Casserly, executive director of the Council of the Great City Schools (CGCS), said he sees such disparity as both a pervasive and insidious problem. "Too many of our schools could be teaching almost anything and calling it science. If we cannot decide what is important to teach in science, then low scores should come as little surprise, not only in the cities but nationwide."

"Even students who somehow manage to find a career with no math and no science anywhere around it," Friedman noted, "still, as citizens they're going to be voting, they're going to help decide issues like climate change, energy policy, and funding for research. At the below-basic level, students are endangered in having the ability to participate wisely in vital public issues like this."

Casserly acknowledged the vast array of micro-level issues affecting scores, but he said he sees one macro-level, or all-consuming, problem as being a long-term detriment to students' capabilities in science education: "There is really no national science education strategy that has any resonance at the local level, despite our rhetorical concerns about the country's international competitiveness. I don't know of any urban superintendents, or superintendents anywhere, who would be able to tell you what our national science education strategy is."

He noted that we have some good programs and curricula, but no national strategy to which local districts can align their work. He added, without national education standards, "I fear we may never move past our incessant hand-wringing about our international science pre-eminence."

Sacramento County Office of Education Superintendent Dave Gordon said he believes the assessment results can help TUDA participant districts in a number of ways. "Very definitely the results can be used to inform the development of curricula," he suggested. "For example, the framework, or material the assessment is based around, is based on consensus documents of the major science teachers' associations. It's based on a consensus of what science and science education ought to involve. So certainly, districts can begin building curricula around that framework. They can look at building state standards."

Additionally, Gordon said, as a result of the assessment, he has seen weaker districts take the necessary steps to partner with stronger ones in other subjects in order to learn what the latter are doing successfully. Fresno, for instance, "has a very robust partnership with Long Beach, which happens to be one of the most successful of the urban districts." In math, Fresno is "literally learning the strategies that work in Long Beach, and it's really helped their program." Gordon said he hopes the new TUDA results will encourage many other districts to take the same steps in relation to science education.

Finally, U.S. Secretary of Education Arne Duncan weighed in, saying that the results "show that students in our cities are further behind in science than in reading and mathematics. In the 21st century economy, today's students will need scientific knowledge to be the world-class innovators and inventors who will sustain America's long-term economic prosperity. With 44 percent of fourth graders and 56 percent of eighth graders scoring below NAEP's basic level, these results show that large city districts aren't preparing enough students to succeed in the knowledge economy."

Complete data for the entire study, broken down by grade level and demographic groups, can be found here. In addition, the page offers links to an executive summary of the findings, a detailed explanation of the 2009 science assessment framework, some sample questions from the test given in the national study, and access to the complete report (.pdf format).