23 February 2009

Pundit Accuracy A Question Of Style

At first, [Stanford University research psychologist Philip] Tetlock's ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit's accuracy. . . . The media's preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.

[P]olitical philosopher Isaiah Berlin . . . in 1953 argued that hedgehogs "know one big thing." They apply that one thing (for instance, that ethnicity and language are primal; ergo, any country that contains many ethnic groups will break up) everywhere, express supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea.

Foxes, in contrast, "know many things," as Berlin put it. They consider competing views, make bottom-up inductive arguments from an array of facts and doubt the power of Big Ideas. "The hedgehog-fox dimension did what none of the other traits did," says Tetlock, who described the study in his 2005 book "Expert Political Judgment": "distinguish more accurate forecasters from less accurate ones" in both politics (will Iraq break up?) and economics (whither unemployment?).


From here, via Newsweek.

Even though "foxes" are more accurate than "hedgehogs," random chance "produces a forecast more accurate than most pundits'. Simply extrapolating from recent data on, say, economic output does even better."

The deeper question is why this should be the case. Readers surely don't try to be deceived by pundits, yet it is popularity with readers that drives a pundit's poularity of pundits with publishers, and hence a pundit's fame. Why should people consistently prefer a cognitive style in people making predictions that is consistently less accurate?

Here's one plausible guess.

Perhaps, in our evolutionary history, the predecessor to the modern pundit was the courtier, i.e. the senior advisor to a group's leader. While leaders may have learned the hard way that hedgehogs make better predictors of the future, in the pre-journalistic era, average people had a lot more communication with leaders themselves than people giving advice to leaders.

Democracy is a young form of government. Other than a few ancient Greek city-states, a part of Rome's history, second millenium Iceland, and the gradual democratication of the British state (not really complete until the mid-1800s), there were no democracies until the French and American Revolutions, respectively, in the late 1700s. Urban living, in contrast, is at least twenty times as old.

In a non-democratic state, citizens don't participate in daily decision making that it is the business of pundits to second guess. Citizens stay where they were born, or go into exile with a more attractive leader. They submit to a conqueror, or rebel out of loyalty to their previous leader. (This may also help explain why terrorism and civil war almost always has its roots in questions of the legitimacy of a regime, rather than its policy choices; humans may be hardwired not to violently rebel against decisions they have no opportunity to participate in making.)

In a leader, who makes his own fate to a great extent, it is a virtue to be publicly forceful, confident and decisive, rather than tentative and balanced. The goal of a leader is to motivate people, not to accurately predict the outcome of events they will not play a part in themselves. Leaders with a clear vision may have more success in changing outcomes. There is no evolutionary percentage in knowing that you are going to fail before you do if you can do nothing about it.

Wise leaders, in turn, can formulate their plans, from the tenantive and balanced counsel that they receive in private. Indeed, confidentiality is a central element of the ethics of all professions who make it their business to give advice: lawyers, doctors, psychologists, accountants, and clergy. Even more notable in the ancient convention in parliamentary regimes of cabinet members in a government showing a united public face to the public on policy issues, regardless of the views that members personally expressed in private cabinet deliberations.

The average newspaper reader hasn't caught up with his or her psychological legacy. Most of us hunger for decisive leadership, rather than thinking like kings ourselves.

2 comments:

Michael Malak said...

Perhaps that is the reason for the apparent failure of democracy -- that people prefer "decisive leadership" over "thinking like kings", i.e. most people are followers rather than leaders.

Andrew Oh-Willeke said...

Not so sure.

It isn't at all obvious to me that democracy has failed. Since its emergence, it had become the predominant form of government, and even in the places where it is absent, single party state leaders and military junta dictators claim that despite temporary necessities that they aspire to some form of democracy.

This could, however, explain why republican government (with a little "r") works better than government via citizen initiative.

I suspect that minimally democratic democracies (i.e. those with a minimum of decisions for citizens to vote upon), like the federal government, and even systems like the U.K. where you vote only for a member of parliament and a local councilperson, are best most of time. (Switzerland being the leading counter-example in this and many other ways.)