A result of zero doesn’t always mean zero results

Forlorn graduate students sometimes turn to a publication called The Journal of Negative Results.* In graduate student mythology, it’s the repository for toiled-over experiments that produced nothing — no effects, no detections, no differences, nothing.

(*This actually does exist for specific disciplines. But it’s not really the salvation most grad students wish for.) 

But: a) that’s how science works, and b) negative results can still pack a punch.

Last week, two astrophysical negative results appeared in high-profile journals.In Nature, the IceCube collaboration (including then-graduate student Nathan Whitehorn) describes missing neutrinos — a paucity of particles that’s problematic for theorists suggesting that gamma-ray bursts generate ultra-high-energy cosmic rays.

Then, in the Astrophysical Journal, a team of astronomers in Chile concludes that dark matter doesn’t live within 13,000 light-years from Earth. The result is a problem for just about everything, if true, but especially for theories describing the shape of the dark matter halo surrounding the Milky Way galaxy.

“A non-detection is not as thrilling as the discovery of some new particle or phenomenon or something like that,” says astrophysicist James Buckley of Washington University in St. Louis. “But the nice thing about a non-detection is you kind of can’t argue with it.”

It’s true. Zero doesn’t get enough love in science. Some scientists even say that measuring zero is the most useful thing that’s been done so far.

But these zeros aren’t really nothing, end of story, finito.

IceCube scientists expected to see at least 8.4 neutrinos during their two-year observation period. They saw zero. Or rather, anywhere up to 2.3 neutrinos, once statistical uncertainties are incorporated. “The 90-percent confidence level is 2.3, so we’re setting an upper limit,” says coauthor Spencer Klein, at Lawrence Berkeley National Laboratory.

Zero is setting an upper limit? Yes, if you believe statistical voodoo and complicated probability theories.

The dark matter team used a new method to determine that all the mass in the solar neighborhood is accounted for by what we can see, meaning there is no invisible source of mass that implies the presence of dark matter. That’s also the equivalent of zero. But, “even though they measure technically zero dark matter, the uncertainty on that means there might well still be some fraction of dark matter present,” says astrophysicist David Law of the University of Toronto in Canada. Once that uncertainty is calculated, Law says, the value of zero — paradoxically — isn’t all that inconsistent with previous estimates.

Of course, these analyses assume the studies are well done, the interpretations accurate, and the results reproducible. This seems true for the IceCube Collaboration, which uses a mammoth subterranean neutrino detector buried beneath the Antarctic ice. “If you have a good enough instrument and you go out and look for something and it’s not there yet, that actually tells you something new,” says physicist Abigail Vieregg of the Harvard-Smithsonian Center for Astrophysics in Cambridge, Mass.

But the dark matter paper is a bit different. The authors pioneered a method for measuring nearby dark matter, and the method is raising some eyebrows. Notably, the eyebrows of Chris Flynn, an astronomer at Swinburne University of Technology in Melbourne, Australia who reviewed the paper. Flynn says he and the authors “agreed to disagree” about their results. “Overall I am really concerned that the method being applied simply doesn’t work for some reason,” Flynn notes. As a test, he tried the method on a simulated galaxy with a known amount of dark matter. The method failed.

The journal still decided to publish the study, which, although provocative, might be flawed.

In any case, both the dark matter and neutrino work will likely send theorists back to their calculations, and experimentalists scurrying to come up with bigger and better detectors.

“Facing these two mysteries in science, it’s an incremental process by which we build the largest detectors we can, try to do the finest science we can for that amount of money, and if we’re able to crack the nut, then that’s great,” says physicist Peter Gorham of the University of Hawaii in Manoa. “Sometimes the most exciting results are when you don’t find what you expect. For example, if the Higgs Boson doesn’t show up, that’s going to be very, very exciting.”