Website tests predictive powers of the hive mind

As the saying goes, “It’s difficult to make predictions, especially about the future.” The website Metaculus.com aims to make this challenging task easier by harnessing collective wisdom.

Metaculus solicits answers to questions about the future — on topics spanning science, politics and economics — and combines these predictions to infer the likely outcomes. Will 2016 be the hottest year yet recorded? Will we find evidence for aliens soon? Will we hail self-driving taxis in the next few years? The hive mind might provide answers.

The website, created by physicists Anthony Aguirre and Gregory Laughlin of the University of California, Santa Cruz, along with former postdoc Max Wainwright, is an experiment to test whether our pooled instincts can produce reliable predictions. The site may also help scientists make informed decisions about which research to prioritize. Organizations funding research on pandemics, for instance, might want to know whether people are more concerned about bioterrorism, powerful germs escaping laboratories or naturally circulating diseases like the flu.

There’s a precedent for successful crowdsourcing of predictions. A U.S. government–funded geopolitical forecasting effort, the Good Judgment Project, has found that collective predictions can be remarkably accurate, and that prediction is a skill that can be honed.

After completing a free sign-up process, Metaculus users click through yes-or-no questions and make predictions, moving a slider from zero to 100 percent to indicate their level of certainty. The site provides relevant background information on each question, and additional research is encouraged. Prognosticators can hash things out in the comments section and share resources to help others make their predictions. Users rack up points — and bragging rights — when their predictions turn out to be correct.

The hive mind isn’t perfect — Metaculus users pegged the probability that the United Kingdom would vote to leave the European Union at just 32 percent. The United Kingdom did vote to leave, but that doesn’t mean the method is flawed. “The point of this is not to get a ‘yes’ or ‘no,’ ” Aguirre says, “but to get what is the probability.” Most events aren’t predictable with complete certainty, he says, but attaching a probability to such events can be useful in planning for the future.

So far, Metaculus has about 1,300 registered participants. In a review of more than 2,000 user predictions, the results were about as expected. When users predicted an event would happen with 80 percent certainty, they were correct about eight times out of 10. When many minds join forces, even nonexperts may collectively become capable guesstimators.