Why quantum mechanics might need an overhaul

SAN ANTONIO — Quantum mechanics is science’s equivalent of political polarization.

Voters either take sides and argue with each other endlessly, or stay home and accept politics as it is. Physicists either just accept quantum mechanics and do their calculations, or take sides in the never-ending debate over what quantum mechanics is actually saying about reality.

Steven Weinberg used to be happy with quantum mechanics as it is and didn’t worry about the debates. But as he has thought about it over the years, the 83-year-old Nobel laureate has reassessed.

“Now I’m not so sure,” he declared October 30 in San Antonio at a session for science writers organized by the Council for the Advancement of Science Writing. (Disclosure: I am a member of the CASW board.) “I’m not as happy about quantum mechanics as I used to be, and not as dismissive of its critics.”

One reason Weinberg thinks there’s a need for a new chapter in the quantum story is that those who think everything is fine with quantum mechanics take different sides in the debates about it.

“It’s a bad sign in particular that those physicists who are happy about quantum mechanics, and see nothing wrong with it, don’t agree with each other about what it means,” Weinberg says.

Quantum mechanics stirred up consternation from its beginnings. More than a century ago, physicists such as Max Planck, Albert Einstein and Niels Bohr showed that standard 19th century physics was inadequate for explaining various features of heat, light and atoms. By the 1920s, other physicists, including Werner Heisenberg, Erwin Schrödinger, Paul Dirac and Max Born, developed those early realizations into the full-fledged quantum mechanical math that today lies at the foundation of physical understanding of just about everything. Quantum mechanics, Weinberg noted, is the “basis of our understanding of not only atoms, but also atomic nuclei, electrical conduction, magnetism, electromagnetic radiation, semiconductors, superconductors, white dwarf stars, neutron stars, nuclear forces and elementary particles.”

But quantum theory’s explanatory power has come at a substantial price: the need to accept counterintuitive weirdness about reality that many physicists, including such pioneers as Einstein and Schrödinger, refused to accept.

One such objectionable aspect was the quantum rejection of Newtonian determinism, the belief that all events are fully determined by preceding circumstances. You can calculate exactly where a baseball will land, for instance, if you know its velocity and direction when it gets hit by a bat. Quantum mechanics, to the contrary, imposes a probabilistic element into the description of natural processes. When an electron bounces off an atom, no one can predict exactly which direction the electron will go; quantum mechanics just permits you to calculate the odds that it will go one direction or another. A mathematical formula called the wave function provides the instructions for calculating where an electron is likely to be — when you make a measurement of the electron, you are most likely to find it where its probability wave is most intense. Repeated measurements would find a range of results corresponding to the probabilities that the quantum math specifies.

Einstein objected, saying God does not play dice. He further objected to another weird aspect of quantum mechanics, involving its description of pairs of particles separated at birth. Two photons emerging from a single atom, for instance, could fly very far apart yet share a single quantum description; making a measurement on one can reveal something about the other, no matter how far away it is.

Attempts to explain these conundrums fall into two broad categories, Weinberg said: “instrumentalist” and “realist.” Instrumentalists contend that the wave function is merely a tool for calculating the results of experiments — there’s no way to know anything more about reality. Devotees of the realist approach contend that the wave function is a real thing out in the world, evolving over time, and at a fundamental level it is responsible for what’s really happening.

Weinberg finds the instrumentalist view unattractive. It’s “so ugly to imagine that we have no knowledge of anything out there — we can only say what happens when we make a measurement,” he says. “The instrumentalist approach takes the attitude that we just don’t know what’s going on out there.”

On the other hand, the realist view does say what’s going on “out there,” but at the cost of enormous complexity, in the form of a countless number of independent streams of reality. “What’s going on out there is a wave function that is progressing with time in a perfectly deterministic but incredibly complicated way,” Weinberg says. In this view, all possible outcomes of quantum processes (that is, everything) come to pass in one stream or another (even though nobody is aware of any of the other streams, or “histories”).

Weinberg would prefer a reality with one history. But apart from that preference, the realist approach does not explain why measurement results observe the rules of quantum probabilities. If everything actually does happen in the various histories, there seems to be no reason why the quantum rules for probability would apply inside any one stream.

So Weinberg thinks there might be something beyond quantum mechanics, a deeper theory that introduces probabilities at a fundamental level, rather than requiring a human to make measurements to get the probabilities to show up. And there is a line of research attempting to generalize quantum mechanics along those lines. But so far a compelling theory that succeeds in generalizing quantum mechanics does not exist.

Perhaps a replacement for today’s quantum theory will come together any time now. Or perhaps not. “Maybe it’s just the way we express the theory is bad,” Weinberg says, “and the theory itself is right.”

Or possibly a surprise is in store.

“There’s always a third possibility,” Weinberg said, “that’s there’s something else entirely, that we’re going to have a revolution in science which is as much of a break with the past as quantum mechanics is a break from classical physics. That’s a possibility. It may be that a paper from a graduate student tomorrow morning will lay it out. By definition I don’t know what that would be.”

In any case, Weinberg observed, there’s a danger in evaluating any theory in terms of contemporary philosophical prejudices. Newtonian gravity, Weinberg noted, was itself regarded as unacceptable by many scientists of his era.

“Newton’s theory … seemed unpalatable to his contemporaries,” Weinberg said. Newtonian gravity was action at a distance, with no tangible pushing or pulling guiding the planets in their orbits. That “seemed like the introduction of an occult element into science, and was rejected for that reason by the followers of Descartes,” Weinberg said. Furthermore, “the force of gravitation was something that couldn’t be deduced from fundamental philosophical considerations and was rejected in part for that reason by the followers of Leibniz.” And Newton also did away with the dreams of Kepler and others to deduce the size of planetary orbits from fundamental principles.

Yet over time, Newton’s theory compiled an impressive list of successes (much like quantum mechanics has).

“By the end of the 18th century, it was perfectly clear to everyone that Newton’s theory was correct, or at least a spectacularly successful approximation,” Weinberg said. “We can take the lesson that it’s not really a good idea to hold new physical theories too strictly up to some preexisting philosophical standard. We have to go with it and see where it takes us — and see whether or not perhaps we have to change our philosophical standards.”

Follow me on Twitter: @tom_siegfried