Conquer Club

So Salganik's plan was to do the exact same thing, except instead of staging a competition to improve predictions of movie preference, he wanted the competition to improve predictions about the things sociologists cared about: high school GPA, which child would persevere when faced with adversity, who would become homeless. Could we, he wondered, harness the pattern-finding abilities of computers to discover new things about how individual lives turned out?

"Looking at lots of people and looking at broader patterns helps us have a fuller understanding of what's possible," Salganik says. If his competition worked well, it could make the world a better place. After all, if computers could locate the things that predicted stuff like higher grades, policy makers could design better interventions.

So Salganik set to work. He got a massive trove of data on 5,000 kids who had been followed from the day they were born, then made that information available to data geeks and researchers across the globe. Four hundred teams were given incredibly detailed information about the kids from birth until age nine, then told to predict their grades — and a handful of other outcomes — at age 15.

One day last fall, Salganik sat down to crunch the numbers, figure out which models were best able to predict where the children in the study had ended up, and what he found deeply surprised him.

What Salganik wanted to see was at least one computer model entry able to predict with reasonable accuracy the outcomes of each child in the study.

But none of the computer models did as well as Salganik expected.

If they had, the screen in front of him would have been filled with tall, colorful towers — bars stretching from the floor of the y-axis to the top, indicating that the predictions had gotten close to 100 percent accurate. Instead what he saw was a bunch of squat bars crowded around the bottom like flattened mushrooms, indicating that the predictions were a lot closer to 0 percent accurate than 100 percent.

"I would say this is not impressive," he tells me as he looks at the graph. "I think this is sad. Disappointing."

Symmetry wrote:Modelling has always seemed a poor way to predict human behaviour in general. It's a "watched too much TV" way of approaching the world.