Digital Content | In Print

Maintaining Content Standards in a Digital World

A textbook follows an arduous process to get approval for use in the classroom. So who vets the curriculum when a teacher can simply pluck a learning object off a virtual shelf?

• By Dian Schaffhauser
• 04/09/12

Even though non-print-based learning objects have been in the classroom for years, the shift to digital content has gotten more attention lately. One development that's made the difference is the Apple initiative to introduce its iBooks textbooks for iPads, generating a notable amount of buzz. That and President Obama's announcement that he believes every student in every state should have an all-digital curriculum by 2017. The US Department of Education and the Federal Communications Commission have jointly issued a "playbook" to help guide district efforts to prepare for that transition. Florida already has committed to going all digital by 2015, while San Diego has distributed 78,000 digital textbooks to its students.

That's why it should surprise nobody who's looking for signs of the arrival of a digital revolution that the Vail School District (AZ)--a tiny rural district southeast of Tucson with 10,000 residents, two grocery stores, and no bookstores--is part of an advance charge to stop buying textbooks for its students. Vail's grand vision, called Beyond Textbooks (BT), began nearly five years ago with a goal of shifting to the use of open educational resources--digital content--in all of its classrooms. But it's grown into far more than that.

Today, nearly 50 other districts in the state have teamed up with BT as paid partners to gain professional development and access its wiki-based repository, which includes lesson plans, quizzes, interactive web links, ideas, presentations, and other digital matter. (In fact, the work won the Vail district the 2011 Sylvia Charp Award from the International Society for Technology in Education and THE Journal.)

Once the Common Core State Standards are in place, will Vail's work spread beyond Arizona's borders and become the rebel alliance that single-handedly overturns the rigorous textbook adoption process and takes down the textbook publishing empire composed of Pearson, Cengage Learning, McGraw-Hill, and Houghton Mifflin Harcourt?

Not likely. But Vail's efforts do raise an important question for any district that is developing its own digital curricula outside of the professional publishing world: How do you ensure quality assurance? 

Local vs. State Adoption
Arizona uses a local textbook adoption model. Each district is empowered to select its own curriculum rather than have a central textbook committee or state department of education recommend a short list. In the course of making the transition to digital content, Vail has had to sort out how to revise its legacy review steps for choosing textbooks with something that works with digital content--and not commercially published content either, but materials provided by teachers.

The question isn't simply how to reconfigure an existing practice. Textbook selection allows state and local governments and school communities to wield influence over what their students will learn. The selection process--for better or worse--brings all interested parties together to say, "We agree, this is what our children need to know."

Teachers have typically built their lessons around what's in the textbooks. But in an environment where the individual instructor can get onto a website and download a learning object to teach a concept, what happens to that vetting process? Who ticks off the checklist of standards that are met? Who defines and makes sure quality is maintained? And will teachers be teaching what our children need to know?

According to BT Director Kevin Carney, in the old days--pre-2008, when BT started--this former teacher and middle school principal remembers faculty meetings where somebody would hold up a textbook and say, "Hey, we'd like you to review these." Representatives would be chosen by grade level or subject area and told to provide feedback within a given time frame. Nonexperts would also be asked to weigh in as well. It wasn't an onerous duty. After all, he notes, textbooks would be cycled out only every five to eight years.

But that approach had its limitations. For example, while the textbook might work for a major adoption state, Arizona isn't part of the club. "So you might indeed be teaching some standards out of that textbook that are terrific things, but they don't match the Arizona state standards," says Carney. "In essence, you're putting your students and yourself at a risk for student achievement."

For states where textbook adoption is done at the state level, quality control might seem like a nonissue. After all, isn't that what the state curriculum review committees are supposed to do? In theory, yes; but not necessarily in practice. Evaluations aren't always robust.

As Nobel Prize winner Richard Feynman famously recounted in his 1985 reminiscences, Surely You're Joking, Mr. Feynman!, the physicist joined the California state curriculum commission and discovered that he was the only member of the committee evaluating math books who actually bothered to read them: "It turned out that the other members of the committee had done a lot of work in giving out the books and collecting reports, and had gone to sessions in which the book publishers would explain the books before they read them; I was the only guy on that commission who read all the books and didn't get any information from the book publishers except what was in the books themselves, the things that would ultimately go to the schools."

Don Collins, a math textbook author, professor at the University of Texas at El Paso, and a former managing editor for a major textbook publishing company who has sat in on many publisher presentations to state textbook committees, says reviews can still be that way: "Publishers give you enough books that everybody in the committee gets one," Collins says. "Even though you got a book, that's no guarantee that anybody looked at it."

Another problem is that major "adoption states" dominate the conversation for most publishers. These are the states where a central body is in charge of selecting textbooks that schools can buy--and because those states have so many K-12 students, the books are written to cater to those states' needs. Says Collins, "Texas, California, Florida, and, to some extent, Indiana dominate, because the markets are so big. They set the guidelines, and the publisher meets those guidelines."

That's not to say that that these states push for inaccurate or bad curriculum, but their influence gives other states less leverage over the texts they want to put in front of their students. In 2010, for instance, Christian evangelists and social conservatives who controlled the Texas state education board proposed changes to Texas curriculum--such as minimizing explanation of the rationale for separation of church and state in history books. In response, California passed legislation specifically to make sure Texas-specific content didn't make it into California textbooks too. (The state's governor vetoed the law, so it was never enacted.)

Also, though they're loath to admit it, textbook committee members are not adequately prepared or necessarily qualified to evaluate books. As Christopher Stream, an associate professor at the University of Nevada, Las Vegas, who has researched textbook adoption, reports, "Some committee members know the subjects; others don't. Some committee members are trained for the task; others are not. Some are given adequate time to really study the books; most are not…. Committee members, like members of Congress, are courted by publishers' sales representatives who offer free trips, special seminars, unlimited sample books, and other perks."

Seeking Successful Classroom Content
Although the Vail district's process for approving digital content isn't a committee process at all, it has its own simple methodology that's free of the broader problems that pervade some of the more formal book adoptions. First, says Carney, the teacher who's posting a resource has to try it out in the classroom. "It's not only something you had to have used, but it has to have been used with success." BT, he adds, does leave it up to the individual teacher "to define what success is."

Once somebody uploads the resource, it's live and available for other teachers to access. There's a crew of three at BT who--even as people may be downloading and trying out the digital object--examine potential copyright issues; formatting problems, such as converting a document file to a PDF; congruency to standards; and level of rigor. If there's ever a question about a given upload, Carney himself weighs in. When it's a specialty area, such as physics, BT calls on experts in the given field within its membership.

Each resource has a name, school, and school district attached to it. That allows another teacher to follow up by phone or e-mail, he points out, but it also encourages people "to be more careful about what they post because their name is behind it."

Teachers from subscribing schools also can weigh in with opinions. "That very rarely happens," Carney insists, but when it does, the team examines it. Often the issue is that the particular resource isn't at grade level. "But, as we all know, we have kids who are below grade level," he continues, "so this would be a terrific resource for [them]."

"We've found great success," Carney reports. The numbers prove it: Of 5,000 learning objects on track to be uploaded in the current academic year, "we've had less than 10 resources that have come back to me."

Crowdsourcing Content Review
Another approach for vetting digital content is crowdsourcing, a practice in use at schools in Georgia and Virginia.

The Georgia Virtual School, which has offered middle and high school-level courses to schools and parents statewide since 2005, uses an amalgam of commercially produced materials that it purchases along with open education resources. All of it adheres to the state's Georgia Performance Standards.

To simplify the process of helping educators find the resources they need for a particular lesson, the state has recently added an online service that allows teachers to rate the resources they've used, "so the cream rises," explains Program Manager Wendy Grey. "We want it to be teacher-approved." The approach is "Amazon style," allowing instructors to write reviews and give star ratings to the digital content they've tried.

Henrico County Public Schools in Virginia follows a similar strategy. The district uses a number of commercially published sources for digital content, but it also encourages development of lesson plans and digital content by its own teachers.

The district hosts an annual competition in which it solicits submissions to "Henrico 21," a public repository. The submission needs to include a lesson plan, a rubric, student handouts, links to essential resources, and a student work sample or "artifact" created through the lesson. Lessons have to be vetted first at the school level through a site-based team review. That same team also provides feedback to teachers and makes recommendations for moving lessons onto the district level.

The content added to Henrico 21 is licensed under a Creative Commons license, which allows it to be used by other teachers, schools, and districts inside and outside the state. Currently, the site hosts between 200 and 300 lessons. They can be searched by grade level, subject, tag, winner standing, application to development of a 21st century classroom, and other criteria. Individual entries can be star rated and commented on.

Vail's BT has avoided the crowdsourcing approach because, says Carney, he fears it might send the wrong message. "There are many cases where educators are told, 'You're not doing a good job; you don't know what you're doing; the schools are terrible…' When we start rating those resources, we run the risk of a fellow educator being another voice [saying] 'You're not doing a good job.'

"I want to be sensitive to the fact that these teachers are working hard, willing to share their resources. To rate them opens up a can of worms. We're telling them, 'You're less than… because someone rated your resources a one, and another was rated a four.' I just want to be careful with that."

The Pursuit of Quality
As sensitive as it is to the feelings of the teachers who contribute to BT's riches, being kind doesn't necessarily translate to being effective in the classroom. The major publishers probably would consider themselves to be in a better position to stay on top of the pursuit of quality than the typical educator.

Quality of curriculum has several aspects, and they're irrespective of format, says Pearson School CEO Peter Cohen. First, there's the content. "Is the content appropriate? Is it authentic? Does it do what the author is trying to do in terms of teaching a certain subject area?" Whether it's digital or print, he observes, schools want to make sure that the content they're delivering is accurate.

Then there's the quality control within the editing and production processes. "We have teams of people," Cohen notes, "whose sole job it is to go through and make sure there's a period at the end of a sentence, and the words are spelled properly, and the English grammar is proper. That doesn't mean it's perfect 100 percent of the time, but they're pretty good overall."

Also, there's production quality. In that arena, Cohen believes digital wins hands down. Referring to Pearson books converted for the iPad, he declares, "It's sort of infinitely superior to the quality you can put on a printed page. One of the comments that the Apple folks keep making is how simply beautiful the pages are. The graphics are just stunning. They are so realistic and lifelike. In a printed page, you're limited by the dots per inch, and there are fewer dots per inch than you can get on pixels on a screen. From that perspective, the quality is much better."

A final quality issue is ensuring that the curriculum addresses the full scope and sequence of the subject area. Could a team of enthusiastic high school biology teachers, for example, truly re-create the depth and scope of Modern Biology from Holt Rinehart Winston? Probably not, especially if you count in all of the extra materials being made available by publishers to instructors and learners as auxiliary resources.

Major open education resource advocate David Wiley recently blogged about this topic, calling for funding for a "next gen" of OER: "You have to admit that some of the things the publishers are working on are both cooler and better than almost everything that currently exists in the OER space. Can you name a single OER project that does assessment at all (and I don't mean PDFs of quizzes)? Can you name one that does diagnostic assessment or handles mastery in any meaningful way?"

Likewise, there are other areas where entrenched publishers retain a decided advantage over a crowdsourced option or a staff review of submitted open educational resources: They've been on the front line for decades addressing content bias.

"When people are looking at programs, they want to make sure that all diverse groups are appropriately represented," Cohen explains. A teacher building and sharing a lesson plan may represent only the kinds of students she has in her own classroom. Others who get that material may not find themselves reflected in the lesson.

"The person who was producing it never thought about that because it's beyond the purpose they originally created it for," he adds. "When people produce open education resources, they don't necessarily have to think about the much larger use of the material. So they don't go through that vetting process that we who are selling programs go through to assure that they are appropriate for the audiences who are buying them."

Furthermore, the big companies are able to provide functionality to their client schools that can help them more efficiently correlate the content their teachers are using in the classroom with the standards being mandated by the state. In situations where administrators keep a very close tab on what instructors are teaching, those publisher-supplied tools are useful for reporting purposes. Replicating that on the fly with a variety of digital objects would be trickier.

All of this isn't to suggest that Cohen doesn't see a place for digital content developed and sourced by teachers. Pearson, in fact, sells an online learning exchange that provides a repository and functions for doing searches and ratings. "Eventually, the best content will get well enough vetted that you can feel relatively confident that it's meeting your needs," he acknowledges.

But even then, he doesn't view the hunting and gathering efforts required in the open educational resource realm to be a good use of teacher time. "One of the questions that a school or a teacher or district needs to ask itself is, 'What is the best use of my resources in order to improve education achievement? Is it going out and curating content…or am I better off leveraging what's already been vetted, curated, tested, aligned, etc., and have my teachers spend their time engaging with students, creating activities, being involved in Socratic discussions, motivating kids by spending their time with them? Which one is going to get me a better outcome?'"

Revolutions Are Time-consuming
The Common Core State Standards could introduce a tectonic shift in how content is created and reviewed. Since most states will be adhering to the same academic standards in math and English language arts, suddenly the need for individual states to have materials created to match their unique standards "pretty well goes away," points out Geoffrey Fletcher, deputy executive director of the State Educational Technology Directors Association and former editorial director of THE Journal.

Fletcher says some states have already talked about sharing the work of creating open education resources that match with the Common Core standards, each tackling a different subject. Then each would distribute the content they'd developed under a Creative Commons "CC BY" license that would enable the others to share it.

"The important part to me isn't the cost savings--though that would be significant," Fletcher says. "More important would be that states, districts, and individual teachers would have the right and responsibility to actually modify those materials, add to them, subtract from, and change those materials to match the needs of their students." States are beginning to realize, he adds, that they need to provide school districts "with a lot more flexibility in the kinds of materials they have and use in their classrooms."

Since the standards aren't expected to be fully in place until 2014, the development of that open content is still in the future. In the meantime, energetic instructors will continue cobbling together digital content as they're able.

But it may be that during this time of upheaval, as the shift to digital content expands in the classroom, that teachers will begin to feel like Lucy did with her best friend Ethel in the famous chocolate factory episode. They can keep up with the job as long as the bonbons--those digital objects--arrive slowly and steadily. But once the conveyer belt speeds up and teachers are left to sort through content coming fast and furious from publishers, colleagues, and district and state sources, they may decide, as Lucy realizes, "I think we're fighting a losing game."

If that turns out to be the case, the easiest approach could well be to continue having big publishers go through the review gauntlet, whether with printed textbooks or digital content, and supply the bulk of the curriculum, as they've done for decades. Open educational resources and programs such as the one in Vail would continue to exist mostly on the fringe.

Collins, who instructs preservice students at U Texas in how to teach math in the classroom, states that the digital natives in his courses actually seem to prefer teaching by the textbook. "All of your stuff is there," he says. "You don't have to go someplace else."

Plus, he adds, when it comes to digital content, "Some things you can trust and some things you can't. You don't know who in the world is presenting what. You don't know what their credentials are. So you've got to be very careful with it."