Assessment

Report: NAEP Setting Bar Too High

• By Dian Schaffhauser
• 02/01/18

Has NAEP set the bar too high for American students? That's the implication in a new report from the National Superintendents Roundtable and Horace Mann League. According to "How High the Bar?" when results from "nation's report card" proficiency assessments are compared to results from two international assessments and the Common Core, researchers found that the proficiency benchmarks of the National Assessment of Educational Progress would knock out students in almost every country.

NAEP, which issues assessments in multiple topics to students in grades 4, 8 and 12, defines "proficient" as "solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over challenging subject matter...."

Yet, according to the new report, no nation could show that even 40 percent of its students were NAEP-proficient in grade 4 reading; in Singapore, just 39 percent would pass the proficiency bar. In grade 8 math, only three countries would be able to claim proficiency of at least 50 percent — Singapore, South Korea and Japan. In grade 8 science, just Singapore would stand out.

The international surveys used for the study were the Progress on International Literacy Survey (PIRLS), which tests reading knowledge of fourth graders, and Trends in International Mathematics and Science Study (TIMSS), which is collected in fourth and eighth grades. In the most recent PIRLS results from 2016, a representative sample of students from the United States garnered an average reading score of 459, compared to the Russian Federation's score of 581, the highest among all countries tested. In 2015 results, American students came in 10th, with 79 percent of students reaching intermediate or better scores with their fourth grade math skills, compared to Singapore students, where 93 percent generated intermediate or higher scores. In grade 8, 70 percent of students reached at least an intermediate score compared to 94 percent in Singapore.

The problem is that the governing board that sets policy for NAEP misuses the term "proficient," according to the report, "The term does not mean what many assume it to mean: performing at grade level. Nor does it mean proficient as most people understand the term."

As a result, noted Jack McKay, director of the Horace Mann League, the public has become confused, and the misuse has "defeated the valuable purpose of assessment, which is to gain useful insights into school performance."

Similarly, the results of assessment of learning tied to Common Core or "career and college readiness" standards have also been damaged when they're aligned against NAEP's benchmark of proficiency, the report indicated.

"One part of the summary I thought was the most important statement is that the fault lies not with the students and it's not in the schools and it's not in the Common Core — not even in the assessments themselves. It's the flawed benchmarks that are the problem," said Carla Santorno, Superintendent for Tacoma Public Schools in Washington, in a video about the report.

"It kind of takes the air out of the bubble to consider that nobody is at that proficiency level in fourth grade, and just a few [are] for eighth grade math," added Theresa Rouse: superintendent at Joliet Public Schools District 86 in Illinois.

"This report doesn't endorse an anti-testing agenda or seek to lower standards. We believe in assessment," said James Harvey, executive director of the National Superintendents Roundtable, in a prepared statement. "But in the words of a Turkish proverb, no matter how far you have gone down the wrong road, turn back."

A digital version of the report is available for $10 on the National Superintendents Roundtable website.