Conspiracy Theories.

Conspiracy theories, as the term is generally used, are almost never true. I'm inspired to write about them today because of a diary I read at Daily Kos wondering if secret defense department technology didn't impact the fact that we had two major hurricanes (Katrina and Rita) in short succession in the Gulf of Mexico, in order to further nefarious administration political goals, but the issue is a general one. There are, of course, real conspiracies, but when the happen they tend to happen in particular sets of circumstances.

Why are conspiracy theories usually wrong?

1. Humans are by nature pattern recognizing animals. They see connections whether they exist or not. Most people also have a poor sense of the way probability works. Many things which seem like unlikely coincidences aren't really nearly as unlikely as they seem, largely because people do a poor job of defining the relevant event for probabilistic purposes. For example, they ask, what are the odds that I'll run into Bob Jones at the Eiffel Tower in Paris, and not, what are the odds that I'll meet one of the hundreds of people I know, sometime on my trip, given the overlap in circumstances between my life and theirs (e.g. summer vacation length, likely stops on a trip, ability to afford a trip, etc.). Coincidences are often just coincidences.

2. It is extraordinarily hard for large numbers of people to collectively keep a big secret of general interest to the public. Conspiracy theories, more or less exclusively, mean secret conspiracies. But, the vast majority of group endeavors seek publicity and recognition, rather than trying to hide from it. This is true even in the national security field. When Boeing gets a big new defense contract it usually posts it on its website and sends out a press release within minutes. When a new group wants to take political control in Washington, they normally have rallies and advertise functions. Even "secret societies" like Yale's Skull and Bones, or the Free Masons, generally want the public to know that they exist, even if their inner workings are kept secret. Leaks happen. The best way to keep secrets is to narrow the scope of what you are trying to hide and to limit dissemination of those secrets to a few people as possible. The more people know a fact, the more people there are who can leak it. No Presidential administration has ever been able to avoid embarrassing leaks. Few organizations of any kind are that disciplined.

3. Few people are competent enough to maintain a conspiracy. It is hard to run an organization in open view in a competent manner. Running a secret conspiracy is even harder work that requires competent management and competent participants. Many supposed conspirators are simply too dumb to pull it off. Likewise, incompetence, carelessness and miscommunication is a far more common cause of seemingly irrational behavior than organized malfeasance. When an employer doesn't reply to someone or a bureaucrat gives an inexplicable response, it is far more likely that someone misread or lost your paperwork, than it is that they are out to get you. The military, in particular, is far more stupid than it is given credit for being.

4. New technology is particularly hard to hide. There are some very rare examples of "black projects" which remained secret for a decade or two, most notably stealth aircraft technology. But, the vast majority of scientific and technological developments are either (1) described in public journals, (2) limited in scope by existing scientific knowledge, (3) publicly know, or (4) leaked to the public. Moreover, black projects, even when they do exist, can typically only get so far ahead of the curve, because the same scientific advances that made them possible also allow the public to come up with the same ideas. For example, several science fiction authors with no inside information made fairly accurate predictions about the atomic bomb long before any leaks to the public existed, simply by looking at the existing scientific literature and doing back of napkin calculations. Furthermore, the larger the project, the harder it is to keep secret.

5. Simple plans work best. The 9-11 attack is one of the most elaborate criminal conspiracies to ever succeed in the United States. It involves a dozen participants and a handful of people in supporting roles. The plan was pretty simple. Enter the U.S., Learn to fly 747s, board several of them in small groups, hijack the planes at the same time with box cutters, and fly the planes into a handful of high profile targets. Plans that are more elaborate or involve more people than this one are highly unlikely. Yet, most conspiracy theories are hopelessly elaborate. Every additional step and every additional participant greatly increases the margin of error. The U.S. Marine Corps trains soldiers to assume that any plan of any complexity will go wrong at some point and simply pushes them to drill sufficiently that this point comes later, rather than sooner. Very small conspiracies in very controlled conditions can be somewhat more complex. Larger conspiracies, like the flash mob craze that existed for a while (where people converged on a spot and acted weird and dispersed) or a rave, need to be even simpler and also need to be far more benign (to prevent anyone from feeling morally compelled to report them).

When conspiracies do happen, they typically don't involve public figures, don't require significant technological innovation, involve people who had links prior to entering the organization where they publicly appear, don't obviously work against the public interest, are fairly small, and are fairly straightforward. Any complexity usually arises in a highly predictable context (like a court room).