Motivation a Major Factor in U.S. Student Test Performance

A new report finds that bribery substantially boosts U.S. student performance on standardized tests, leading researchers to conclude that lack of motivation, not just lack of ability, has contributed to the United States' middling standings in international rankings based on standardized test scores.

• By Dian Schaffhauser
• 11/27/17

A new study has suggested that American students do much better on tests when they've been bribed to try harder, while students in other countries may not respond in the same way. An international team of researchers shared their findings in a working paper from the National Bureau of Economic Research, which stated, "Offering incentives to U.S. students, who generally perform poorly on assessments, improved performance substantially."

The researchers noted that results from international testing, such as the Programme for International Student Assessment (PISA), tend to have major ramifications within countries. For example, when Germany underperformed in the 2000 PISA, the government "convened a conference of ministers" to propose "urgent changes"; likewise, when Finland "performed unexpectedly well" on the same assessment, it started a movement setting up its school practices as "a model for the world."

Average test scores by country and control group vs. "treatment" or incentivized group. Source: NBER's "Measuring Success in Education: The Role of Effort on the Test Itself."

In the United States, the report pointed out, relatively poor performance compared to other countries has long been blamed on socioeconomic factors, school systems and culture. The researchers proposed another possible reason: Students have no real reason to do well.

To test their theory, the team set up an experiment in the United States and China, which have traditionally shared a large gap in the results of standardized testing. The experiment took place in three high schools in Shanghai, which was ranked first in math on the 2012 PISA test, and in two U.S. high schools, where the country was ranked 36th on the same test. Students took a 25-minute, 25-question math test constructed from questions used on previous PISA exams; 13 questions were multiple choice, and 12 were fill-in-the-blank.

The researchers offered a random set of students a "surprise financial incentive" to do their best on the test, then compared those results against the performance of students not offered the incentive. To prevent extra study, the test-takers only learned about the incentive just before they took the test. The "bribe" offered to some of the U.S. students was an envelope containing $25 in one-dollar bills; they were told that the money was theirs, except that the testers would take away one dollar for each question answered incorrectly or left unanswered. In Shanghai the same offer was made, except there, students were paid in Renminbi.

In response to incentives, nothing about the performance of the Shanghai students changed; however, the scores of the U.S. students rose dramatically. With the incentives, the American students tackled more questions (especially near the end of the test) and were more likely to answer those questions correctly. The resulting effects on test scores were concentrated among students whose baseline performance was near the U.S. average; for those far below the U.S. average, the incentive had "little impact on performance," probably because they simply didn't understand the material, and incentives couldn't change that fact.

The researchers also simulated the impact on U.S. performance for the PISA assessments to understand how those scores might have been affected. The results were remarkable. "We estimate that increasing student effort on the test itself would improve U.S. mathematics performance by 22-24 points, equivalent to moving the U.S. from 36th to 19th in the 2012 international mathematics rankings," the researchers asserted.

The conclusion drawn from the experiment: Tests considered "low-stakes" (because they don't really affect the student) shouldn't be looked at in isolation. Therefore, education leaders and policy makers shouldn't over-react when the results aren't what they'd want them to be. As the researchers wrote, "Our findings suggest that in the absence of extrinsic incentives, ranking countries based on low-stakes assessments is problematic because test scores reflect differences in intrinsic motivation to perform well on the test itself, and not just differences in ability."

The working paper is available on the NBER website in digital format for $5.