Being a maverick doesn’t pay, at least not when braving a strange new world in a computer game that’s like Survivor meets Second Life. The outcome of a computerized game tournament, which finds that a copycat strategy is best, may help explain why mimicking others is so prevalent in nature.
The new study, appearing April 9 in Science, runs counter to the conventional idea that success in an unfamiliar, changing environment comes from a mixture of innovation and copying others.
“What we find is that under a very broad range of circumstances, not only does social learning beat asocial learning hands down, it annihilates it,” says study coauthor Kevin Laland of the University of St Andrews in Scotland. “There’s no balance, no mix of asocial and social learning.”
Laland and his colleagues set up a tournament in which participants wrote computer programs that controlled the behavior of avatars living in a strange and unpredictable world. The competition pitted 104 teams against each other to see who could come up with the best survival strategy and encode it in a computer program.
“In a way, it’s a bit like being dumped on a Pacific island where you don’t know anything about the environment,” says Laland. “You don’t know what’s good to eat. You could fish, you could catch crabs, you could hunt antelope, pick for tubers, you name it. Some of these will reap rich rewards and some will be fruitless.”
Going into the study, the researchers thought the optimal strategy would be some kind of mixture of copying and innovating, Laland says, both of which have drawbacks. An unknown berry might turn out to be a great food source for the person who first discovers it, or the berry might be poisonous. On the other hand, copying others might be safer, but not if the information is outdated or wrong. To the researchers’ surprise, the best method relied almost exclusively on copying.
Players were presented with a choice of 100 different moves, each with its own payoff. Each move, although described purely in mathematical terms, could be thought of as a corollary of picking berries or hunting antelope. To make the situation more like reality, the payoffs associated with each action changed with time, representing a shifting environment. At every turn, the players could copy others’ behaviors or write a program to create a new move. As the players racked up bigger payoffs, the probability that the avatar reproduced increased — a sign of success.
After many iterations of the tournament, a program called discountmachine written by a team of two graduate students from Queen’s University in Canada beat out the competition. Mathematician Daniel Cownden and neuroscientist Timothy Lillicrap won 10,000 pounds for the winning program.
In addition to discountmachine’s penchant for copying, the program relied on a feature called a neural net that tracked how fast the environment was changing. This allowed avatars to pay less attention to previous actions that were once successful, but may have become outdated.
“The results have influenced my own thinking about how people can learn from their own experience and the experience of others,” comments Robert Axelrod of the University of Michigan in Ann Arbor. In the 1980s, Axelrod conducted a similar tournament to study cooperation.
There is a caveat to the winning program’s success, though. It works only when there are other agents around to copy. “They’re effectively kind of parasitic,” Laland says. “You can think of social learners as information scroungers — they’re stealing the information produced by others.”
Evolutionary ecologist Richard McElreath of the University of California, Davis calls the new study a “valuable and unique contribution,” but points out that the current tournament defines social learning differently from previous studies and so may not be at odds with earlier models of behavior. Even so, McElreath suspects that this tournament will “become a classic.”
