Rushton Hurley on Data-driven Deception

Does educational research data tell us anything? Can it prompt us to improve what we do? The important question for me is whether research of any kind prompts me as an educator to think about ways to improve my teaching.

• By Rushton Hurley
• 05/21/12

Robert Marzano, CEO of the Marzano Research Laboratory once said, "All research is equivocal, particularly in education. And here's why: You can never account for all the factors in the classroom that impinge upon the strategy, the technology, etc. You cannot take the human being out of it. It's always going to be that way."¹

I like the word "equivocal" in this context, meaning ambiguous or questionable, and it fits my thinking perfectly about most discussions of "data" I hear as an educator.

Often data is given to us as a way to suggest that a particular product or technique is valuable for the work we do in our classrooms. But are we prepared to question what is presented to us as an opportunity to make a "data-driven" decision? Are the variables in the study genuinely related? If so, what is the basis for the contention that one variable causes the other? Can the conclusions be generalized such that the data are in any way useful for my students and me?²

Less abstractly, the data may have to do with students who have an entirely different set of challenges and opportunities than my students have. They may reflect the use of the tool or technique in a way that I could or would not replicate. Most importantly, the research may not account for important variables in such a way that there is any reason to apply it at all. Put simply, you may be looking at a set of numbers that doesn't mean squat.

Given the difficulty of isolating variables, not to mention coming up with a genuine control group in order to make any real comparisons, are we to assume that all educational research is bovine excrement? No, and certainly not if such dismissal leaves us complacent about the work we do, but we have to be able to ask good questions of data that is presented to us.

The important question for me is whether research of any kind prompts me as an educator to think about ways to improve my teaching. Knowing whatever it is the research tells me, can I present something more clearly? Can I reach more kids than I have in the past?  If I am reaching everyone, can I add more depth or breadth to what I cover?

"No sensible educator should deliberately make decisions without basing them as much as possible on the findings of educational research," said Edward Vockell, an education professor at Purdue University, "but a familiarity with the findings and methods of educational research will enable that educator to evaluate the relevance of the information [presented]."³

So how do we figure out how to use data to make a difference for our students? I believe this starts with an honest appraisal of our own effectiveness and a willingness to work with colleagues to figure out how to improve.

The best example I know of this is the work of Eric Mazur, a professor of physics at Harvard University. His work on peer instruction underlies much of what we talk about today as being effective in flipped teaching, but it is more compelling to me that he tells the story of his own experience in a way that is as personal as it is statistical.

"I thought I was a good teacher until I discovered my students were just memorizing information rather than learning to understand the material," Mazur told a group at the University of Maryland-Baltimore in 2009. "Who was to blame? The students? The material? I came to the agonizing conclusion that the culprit was neither of these. It was my teaching that caused students to fail." ⁴

Good tools, open resources, examined techniques, professional discussions about results, and meaningful sharing of ideas allow teachers to move from complacency and cynicism to professional exploration of their work and its effectiveness. This is about much more than data: Making this move improves the personal satisfaction of the teachers, and creates opportunities in their work settings.

In other words, the higher the percentage of teachers on campus who are focusing on ways to improve what they do, the better the school, and the happier its teachers will be.⁵

Footnotes:

² syllabus for Introduction to Measurement and Statistics, Linda Woolf; Webster University; no date given

³ Educational Research, Edward Vockell; Purdue University; no date given

⁴ Confessions of a Converted Lecturer, Eric Mazur speaking at University of Maryland, Baltimore County

⁵ My guess based on personal experience and no data. Always read the footnotes.