Assessment

Report: PARCC and Smarter Balanced Get High Marks

• By Dian Schaffhauser
• 02/15/16

The seven states that have dropped the use of Smarter Balanced or PARCC tests may want to reconsider their decision. A two-year study by the Thomas B. Fordham Institute has found that the assessments developed under those two state consortia do the best at matching the content and depth goals laid out by the Council of Chief State School Officers (CCSSO) for English language arts/literacy and math.

Two other sets of assessments — ACT Aspire, used in three states, and MCAS, used in Massachusetts — showed sufficient quality in their test items and the depth of knowledge required for the assessments. But they both also had spotty coverage of "priority content" recommended in the Common Core state standards.

For this project, the institute brought together two "rock-star" principal investigators and almost 40 expert reviewers to study how well the four sets of assessments met the CCSSO's "Criteria for Procuring and Evaluating High-Quality Assessment" in two of its domains. Half of the reviewers were practitioners; the other half consisted of experts in content and assessment. The study examined unreleased items from the student test forms for grades 5 and 8.

This part of the work was managed by the Center for Assessment, which works with states and other educational agencies to design and implement assessment and accountability programs.

A parallel analysis by the Human Resources Research Organization looked at high-school exams along with all of the assessments for accessibility for English language learners and students with special needs.

Overall, PARCC and Smarter Balanced assessments showed the strongest matches to the CCSSO criteria. PARCC came in with "excellent" scores for ELA/literacy depth and content and "good" scores for the same in math. Smarter Balanced had an excellent score for ELA/literacy content and good scores for the remainder.

ACT Aspire and MCAS scored good or excellent for depth in both subject areas and "limited" or "uneven" for content. As the report noted, panelists found those assessments didn't "adequately" assess — or may not have assessed at all — key areas of content.

As an example, the ACT Aspire ELA/literacy assessment didn't require "writing to sources." As a result, the evaluators found, "the program insufficiently assesses the types of writing required by college and career readiness standards." In the area of math, while the MCAS grade 8 assessment sufficiently "focused on the major work of the grade," the grade 5 assessment didn't meet the 75 percent marker set for that grade's coverage of the major or most important work.

In spite of the overall positive appraisal for PARCC and Smarter Balanced, reviewers found areas of strength and weakness for all four programs. PARCC was advised, for instance, to "better meet the [math] criteria by increasing the focus on essential content at grade 5." And Smarter Balanced was counseled to put a "greater emphasis" on academic vocabulary in ELA/literacy.

The Fordham Institute report noted that all four tests evaluated "boasted items of high technical quality." At the same time, the authors emphasized that the assessments built "with the Common Core in mind have largely delivered on their promises." The tests reflect essential standards and "demand much from students cognitively." "They are, in fact," the researchers concluded, "the kind of tests that many teachers have asked state officials to build for years."

Each assessment organization volunteered to participate in the study, and was also allowed to provide responses to the study, which are included in an appendix.

The research project was funded by a number of foundations: the Louis Calder Foundation, Bill and Melinda Gates Foundation, the Leona M. and Harry B. Helmsley Charitable Trust, the William and Flora Hewlett Foundation, the Charles and Lynn Schusterman Foundation, the Lumina Foundation and the Thomas B. Fordham Foundation, the sister organization of the institute.

The 120-page study, "Evaluating the Content and Quality of Next Generation Assessments," is available for download.