Errant Texts

First of a two-part series on middle-school science curricula. Part two, “Where’s the Book?”, is available at Where’s the Book?.

Last December, the National Center for Education Statistics unveiled a series of disquieting findings from an international study of students’ math and science achievement. The data showed that among 38 surveyed nations, eighth graders in the United States ranked no better than the middle of the pack.

Thirteen-year-olds in the United States outperformed peers in Italy, Lithuania, Jordan, Turkey, Morocco, and South Africa, but they trailed their counterparts in at least 17 countries, including Singapore, Japan, Australia, and the Slovak Republic.

So, what’s new? To many U.S. educators, there was something even more troubling in this latest survey, a repeat of the Third International Mathematics and Science Study (TIMSS). The new study found that the ranking for U.S. eighth graders was lower than that of U.S. fourth graders 4 years earlier (SN: 10/19/96, p. 244).

What this means, says Gary W. Phillips, acting commissioner of the National Center for Education Statistics in Washington, D.C., is “that although U.S. students learned a lot of math and science from the fourth to eighth grade, the other nations learned more.”

The big question is, Why? Many school administrators and especially scientists are coming to the conclusion that one major problem resides in the textbooks U.S. middle schoolers use. The science texts in most of their classrooms are nothing short of an embarrassment, according to several recent studies.

The latest review of such texts, which focused on physical-science books for sixth to eighth graders, grabbed headlines in January. “We were asked to evaluate whether the information in the books was valid,” explains study leader John L. Hubisz of North Carolina State University in Raleigh. “And the answer is, No!”

Carefully acknowledging that the 12 books he and his colleagues reviewed contain many useful elements and passages, Hubisz says that each text was riddled with errors, misconceptions, and confusing presentations.

The best-trained teachers may be able to guide students around such problems, but they shouldn’t have to, argues George Nelson, director of Project 2061, a childhood-education program of the American Association for the Advancement of Science in Washington, D.C. Even among top teachers, the vast majority lack the time to develop an alternative curriculum, he says.

In fact, the latest TIMSS follow-up suggests that many U.S. math and science teachers might not be up to the job. The study found that teachers in the United States are less likely than their international counterparts to have made science or math a “main area of study” in college.

Teachers who didn’t major in science tend to “use textbooks–lean on them–more than better-qualified teachers do,” notes Arthur Eisenkraft, president of the National Science Teachers Association and the science coordinator for public schools in Bedford, N.Y.

At the moment, that kind of reliance on textbooks can pose particular risks. Nelson, whose AAAS program has also reviewed texts, observes that “we didn’t find any good middle-school science books or good high-school biology books.” Indeed, he told Science News, “we will claim that these books offer very little potential for having students learn, even if they’re used as they were intended.”

One reason for this pessimism, he says, is that the books’ deficiencies go deeper than just errors of fact. These texts probably contribute not only to the low performance by U.S. eighth graders in the TIMSS follow-up, Nelson contends, but also to the waning interest in science that afflicts a growing number of children beginning in middle-school years.

Reviewing science texts

What most disturbs Hubisz and many others contacted for this story, is that schools aren’t more irate-and publishers more embarrassed–about the quality of so many children’s textbooks.

Hubisz, a physicist, has been informally reviewing precollege science texts since the 1960s. His new study on middle-school books was sponsored by the David and Lucile Packard Foundation of Los Altos, Calif., which also funded a separate review of high-school physics texts.

After identifying some 30 books in common use in middle schools, Hubisz and a team of reviewers focused their analysis on the dozen that had dominated classrooms over the previous 5 years. They represented all major science-text publishers. Each reviewer plowed through two books, line by line, with a simple mission: Record every error, large or small. Their daunting compilation made a list 500 pages long.

Diagrams often did not display what the text or caption indicated. Sometimes, a book asked questions that were impossible to answer-either because it offered too little information (such as giving values for two dimensions when the student needed to compute a three-dimensional volume) or because explanations necessary to solve a problem wouldn’t appear for another couple pages or even chapters. The texts depicted or defined a scientific principle incorrectly with a disturbingly high frequency.

Though Hubisz pared the list of errors to 100 pages for his report, by this summer he hopes to have the entire error-catalog loaded onto a new Web site. Accompanying each error will be an explanation of how it could mislead or discourage a student. The researchers’ hope, Hubisz says, “is that this will encourage everyone to read [all classroom materials] more critically.”

Negative critiques

Hubisz’ team isn’t alone in its negative critique of science texts. Roughly 18 months ago, Nelson’s Project 2061 released its review of 10 middle-school science books. In addition to some of the most widely used texts from major publishers, it included a few new volumes from projects funded by the National Science Foundation. The latter have the potential to gain widespread use in classrooms.

Unlike the Packard study, Project 2061 “took a deep look at the philosophy inside each book, asking how does it attempt to teach ideas,” Nelson says. These analyses explored how each book approached content and instruction for six selected topics. For example, the study probed how a physical science text uses the concept of invisible atoms and molecules to explain changes of state and energy transfer.

The reviewers concluded, according to Nelson, that “even if the science had been 100 percent accurate, students still wouldn’t learn from these books, because the instruction was inadequate.”

Studies by education researchers indicate that to learn, students must confront their preconceptions of how the universe works, compare these with what they glean from books and hands-on experiments, and then discuss discrepancies among themselves and with teachers. Middle-school texts don’t foster such reflective contemplation, Project 2061 found.

Rather, they present statements, questions, and experiments with little to bind them into a whole. Indeed, many publishers crammed in legions of facts, Nelson says, not seeming to realize that “encyclopedias aren’t good textbooks.”

None of these observations surprises William J. Bennetta. As president of the Textbook League, a public-interest group in Sausalito, Calif., he’s spent the past 9 years recruiting experts to review textbooks used in his state’s middle and high schools.

Through its newsletter and Web site, http://www.textbookleague.org/, the league has issued some 200 reviews, primarily of science and math books. Detailed and amusing, most find little positive to say about middle-school books.

“Educators should avoid this book like the plague,” screamed the headline for the league’s review of the 1995 edition of one Prentice-Hall middle-school text, Exploring Physical Science. The book had recycled “dreadfully wrong material” from earlier books by the same publisher, said the review.

Hubisz’ team also reviewed the same edition of Exploring Physical Science, as well as updates of the book from 1997 and 1999. In even the latest edition, the reviewers found hundreds of errors.

Better high-school texts

Books for middle-school students seem to be particularly problematic. High-school texts are far better, says Clifford Swartz, a high-energy physicist at the State University of New York, Stony Brook, who headed the Packard Foundation-financed analysis of seven high-school physics texts. His team of 14 reviewers reported its findings in the May 1999 The Physics Teacher, a magazine that Swartz edited for 29 years.

“Compared with books at the middle school level, these got a relatively clean bill of health,” he says, “with only one having really major mistakes in physics.” He attributes the generally sound–if, at times, lackluster–science in these books to the input of physicists during the texts’ writing and production.

Hubisz says that such input from scientists appears to be absent in the middle-school texts that his group surveyed, even though the books often listed an impressive lineup of scientists as contributors. In fact, he notes, many of these apparent authors “did not even know that their names had been listed.” A few argued that their only contribution had been to review the text–which, ironically, they had “panned,” says Hubisz.

He and others focusing on middle school science books find that they’re typically authored by committees of contract writers that may wield little control over the final product.

Jennie Dusheck, a California-based biology writer, knows what it’s like to work on a middle-school book. “As the author of a college textbook, I ultimately have authority over–and responsibility for–every word. If I don’t like something [that editors have inserted], they will take it out,” she says.

Not so for middle-school books. A publisher’s relationship with these writers “is so profoundly different,” Dusheck finds, that “I would hesitate to call [the writers] authors.” In these projects, a publisher typically hires an author to write a segment of a book, then someone else edits it to conform to segments produced by other contract writers. She recalls reviewing one of her contributions after it had been through this process and asking, “Did I write this?”

Here, Dusheck argues, “the editor is really the author.” The editor sets the book’s tone and commissions elements to adorn the text: photos, sidebars, puzzles, and graphs. In Dusheck’s experience, these peripheral elements are where errors most often crop up.

Yet even when errorfree, Hubisz found, these attractive additions can diminish young students’ understanding of science. Middle schoolers aren’t yet adept at integrating information from a collection of elements, he says. He speculates that too often, these young students focus on the tangential items and miss entirely the main point of an adjacent paragraph. In one book, for example, Hubisz found a sidebar on careers in jewelry-making in a chapter on the properties of metals.

With disparate elements from a host of authors and artists being cobbled into these books, it’s not hard to imagine how mistakes occur. So, it’s especially important that scientists review the material throughout a book’s many stages of production, argues Kenneth Ford, retired director of the American Institute of Physics in College Park, Md. Ford has taught extensively, from the university level down through ninth grade.

Ironically, he observes, publishers of elementary and middle-school science books are less likely than high school and college publishers to enlist professional scientists to vet a book before its printing. Trying to fix mistakes after publication–for future editions–can prove next to impossible. He relates his own experience with an Addison-Wesley book he had criticized.

The company offered to pay him to highlight the book’s problems and possible solutions for a revised edition. However, after working on some 100 pages, Ford gave up. “The book couldn’t be fixed,” he maintains, “because it was too deeply flawed–from beginning to end.”

He notes that the book, which became a big seller, was later reviewed by Hubisz’ team. Its assessment: “No student will increase his or her understanding of science by using this text.”

Stephen Driesler, executive director of the Association of American Publishers’ school division, insists that text publishers try hard to ensure accuracy. They “go through extensive efforts of proof checking, fact verification,” says Driesler. “I have seen where literally every page of the text where there’s a fact will have to be accompanied by at least two independent sources verifying that fact. . . .Notwithstanding, any system devised by humans and implemented by humans is subject to things slipping through the cracks.”

Error-riddled books

If errors occur and reviews such as the ones by Project 2061 and Hubisz’ group’s are documenting them, why do schools continue to buy error-riddled books? The answer, says Driesler, is that these books cover more of the topics state curriculum committees have asked for than the more-accurate books do. “It is the states who are the [textbook publishers’] customers,” he observes.

Increasingly, states are assessing the performance of students and schools with standardized tests. To make sure students encounter exam topics before they actually take the tests, curriculum committees issue detailed lists of what they want in textbooks. Twenty-one states now adopt lists of approved texts. Public schools in these states can get state funding to buy those books. Schools must dig into their own pocket for books not on the list. Those states, which include the “big-volume buyers” [California, Texas, and Florida] “really drive the publishers,” Driesler says.

There is no reason why a book that includes all the topics requested by a state can’t also be accurate and simple to understand. However, Driesler argues, “I would have a hard time–and I think [Project 2061] would have a hard time–pointing you to a textbook that does both.”

In an effort to change that, Project 2061 has begun working with states and publishers on better criteria for middle- school textbooks. Last month, Nelson’s group hosted its first conference at which educators, publishers, and state-curriculum officials convened to explore ways to overcome their sometimes conflicting objectives.

As for factual errors of science, Driesler notes, “the publishers’ goal is zero.” One remedial step that publishers are beginning to take is to post textbook errors and corrections on their Web sites, Driesler says. Last fall, his organization also launched Accuracy E-line, a Web site where anyone can log reports of apparent errors in any textbook (http://www.publishers.org/home/abouta/school/eline.htm).

Better training

Although better and more accurate textbooks should be a goal, Nelson cautions that good books alone are not likely to propel science-achievement scores of U.S. students to the top of the international charts.

To make that happen, teachers will need more and better training, he argues. His own organization holds science workshops for classroom teachers. Traveling to schools around the country, science-education experts offer tips and strategies for making the best use of currently available materials. This spring, Project 2061 will also host regional, 3-day workshops in about a dozen cities.

Others, such as Paul Hickman of the Center for the Enhancement of Science and Mathematics Education at Northeastern University in Boston, are considering more-radical changes in science and math education. The ultimate answer to avoiding erroneous information in classrooms may not be to fix textbooks, says Hickman, but to do away with them entirely. Increasingly, small publishers are starting to offer curriculum packages that include experiment kits, explanatory booklets, and other nontraditional materials.

Mistakes from a science-text survey

By surveying a dozen physical science textbooks, in some cases multiple editions of the same book, John Hubisz of North Carolina State University in Raleigh and his colleagues logged 500 pages of errors. Some were silly, others substantive. All, however, run the risk of making science appear confusing or even nonsensical to students, warns Hubisz. Below are some examples of errors identified in the texts.

* One book said: “Unlike fission, fusion doesn’t happen spontaneously.” Yet fusion reactions power the sun. The sentence should have been amended with: “…at temperatures usually found on Earth.”

* This book also stated that the acceleration of gravity on the moon is one-sixth that on Earth because the moon’s mass is one-sixth of Earth’s. In fact, the moon’s mass is roughly one-eightieth of Earth’s and the initial statement ignored the fact that the acceleration due to gravity is also related to the radius of a body.

* One book defines some elephant vocal sounds at around 400 hertz–below a frequency perceptible to human ears. However, 400 Hz is about the frequency produced by keys on the middle of a piano.

* One text shows a photo of rubber–with a density of 1.19 grams per cubic centimeter–sinking to the bottom of glycerin, which has a density of 1.26 g/cm³. As the reviewer points out, “It cannot happen!”

* One passage notes that “sound travels faster through warm air than through cold air.” Twelve pages later, the same book notes “…but sound travels faster in cold air.” You can’t have it both ways.

Next week: Alternative curricula gain high marks.