Pundit Accuracy A Question Of Style

At first, [Stanford University research psychologist Philip] Tetlock's ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit's accuracy. . . . The media's preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.

[P]olitical philosopher Isaiah Berlin . . . in 1953 argued that hedgehogs "know one big thing." They apply that one thing (for instance, that ethnicity and language are primal; ergo, any country that contains many ethnic groups will break up) everywhere, express supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea.

Foxes, in contrast, "know many things," as Berlin put it. They consider competing views, make bottom-up inductive arguments from an array of facts and doubt the power of Big Ideas. "The hedgehog-fox dimension did what none of the other traits did," says Tetlock, who described the study in his 2005 book "Expert Political Judgment": "distinguish more accurate forecasters from less accurate ones" in both politics (will Iraq break up?) and economics (whither unemployment?).

From here, via Newsweek.

Even though "foxes" are more accurate than "hedgehogs," random chance "produces a forecast more accurate than most pundits'. Simply extrapolating from recent data on, say, economic output does even better."

The deeper question is why this should be the case. Readers surely don't try to be deceived by pundits, yet it is popularity with readers that drives a pundit's poularity of pundits with publishers, and hence a pundit's fame. Why should people consistently prefer a cognitive style in people making predictions that is consistently less accurate?

Here's one plausible guess.

Perhaps, in our evolutionary history, the predecessor to the modern pundit was the courtier, i.e. the senior advisor to a group's leader. While leaders may have learned the hard way that hedgehogs do not make better predictors of the future, in the pre-journalistic era, average people had a lot more communication with leaders themselves than people giving advice to leaders.

Democracy is a young form of government. Other than a few ancient Greek city-states, a part of Rome's history, second millenium Iceland, and the gradual democratication of the British state (not really complete until the mid-1800s), there were no democracies until the French and American Revolutions, respectively, in the late 1700s. Urban living, in contrast, is at least twenty times as old.

In a non-democratic state, citizens don't participate in daily decision making that it is the business of pundits to second guess. Citizens stay where they were born, or go into exile with a more attractive leader. They submit to a conqueror, or rebel out of loyalty to their previous leader. (This may also help explain why terrorism and civil war almost always has its roots in questions of the legitimacy of a regime, rather than its policy choices; humans may be hardwired not to violently rebel against decisions they have no opportunity to participate in making.)

In a leader, who makes his own fate to a great extent, it is a virtue to be publicly forceful, confident and decisive, rather than tentative and balanced. The goal of a leader is to motivate people, not to accurately predict the outcome of events they will not play a part in themselves. Leaders with a clear vision may have more success in changing outcomes. There is no evolutionary percentage in knowing that you are going to fail before you do if you can do nothing about it.

Wise leaders, in turn, can formulate their plans, from the tenantive and balanced counsel that they receive in private. Indeed, confidentiality is a central element of the ethics of all professions who make it their business to give advice: lawyers, doctors, psychologists, accountants, and clergy. Even more notably, in the ancient convention in parliamentary regimes, cabinet members in a government are obligated to show a united public face to the public on policy issues, regardless of the views that members have personally expressed in private cabinet deliberations.

The average newspaper reader hasn't caught up with his or her psychological legacy. Most of us hunger for decisive leadership, rather than thinking like kings ourselves.