How Many People Have Been Convicted Of Felonies In The U.S.?

New research on the growth in the scope and scale of felony convictions finds that, as of 2010, 3 percent of the total US population and 15 percent of the African-American male population have served time in prison. People with felony convictions more broadly account for 8 percent of the overall population and 33 percent of the African-American male population. . . . The study estimates that as of 2010 there were 19 million people in the U.S. that have a felony record, including those who have been to prison, jail or on felony probation.

From Science Daily.

The paper is:

Sarah K. S. Shannon, Christopher Uggen, Jason Schnittker, Melissa Thompson, Sara Wakefield, Michael Massoglia. "The Growth, Scope, and Spatial Distribution of People With Felony Records in the United States, 1948–2010." Demography (2017). DOI: 10.1007/s13524-017-0611-1

Trump Is Finally Rolling Out A Tax Proposal

I have a personal interest in having some clue about what kind of tax policies may be considered or adopted by the Trump administration. I'll be part of a national panel discussing it in early December and have to get my materials in order for that talk in a month. So far, there have been some trial balloons that have mostly fizzled (deservedly), but no real comprehensive proposal, until today.

Now, the New York Times (quoted where indicated by block quotes below) has cribbed the "United Framework For Fixing Our Broken Tax Code" that his Treasury Secretary, a top economic advisor, and half a dozen influential members of Congress have been hammering out behind the scenes for weeks. This process, which kept politically savvy wonks in the room and Trump and his close cronies out of it, has produced the kind of Republican establishment tax reform proposal you'd expect from central casting.

This is a much less radical change than the repeal and replace proposal for Obamacare, so it isn't unthinkable that Republicans will be able to hold together their coalition and get some version of this legislation passed, possibly with some modest Democratic support in exchange for some modest tweaks to the proposals and reductions in the bottom line revenue reductions.

The major substantive points floated at this time are as follows:

THE PRICE TAG:

Members of the Senate Budget Committee have agreed on a budget resolution that would allow for a $1.5 trillion tax cut over 10 years. Studies of similar plans produced by Mr. Trump and House Republicans have been projected to cost $3 trillion to $7 trillion over a decade.

A preliminary estimate from the nonpartisan Committee for a Responsible Federal Budget found that the policies in the framework would cost between $2 trillion to $2.5 trillion over a decade.

The administration is trying, contrary to all econometric evidence to argue that much of the cost of the cuts will be paid for with increased revenues created by economic growth at an unrealistic sustained 4% per annum.

In fact, there is very little in the proposal for which there is any track record of stimulating economic growth when similar measures were adopted in the past. Empirically, economic growth is associated with higher top marginal tax rates, not lower ones, and the core feature of the plan is to lower top marginal tax rates.

Republicans probably don't have the votes to pass the legislation unless they can reduce the projected revenue losses to at least the Senate Budget Committee proposal levels, for example, by keeping the 39.6% tax bracket, perhaps with a higher income cutoff, and by making capital gains accrued at death immediately taxable for liquid assets.

INDIVIDUAL INCOME TAXATION:

[T]he plan would collapse the tax brackets from seven to three, with tax rates of 12 percent, 25 percent and 35 percent, administration officials said. The current top rate is 39.6 percent and the lowest rate is 10 percent. The framework also gives Congress the option of creating a higher, fourth, rate above 35 percent to ensure that the rich are paying their fair share.

The alternative minimum tax would be repealed. The standard deduction would double to $12,000 for individuals and to $24,000 for married couples.

It would also increase the child tax credit from $1,000 to an unspecified amount and create a new $500 tax credit for dependents, such as the elderly, who are not children.

It isn't clear whether the earned income tax credit (EITC) would survive. If not, a significant number of working poor taxpayers will actually experience a tax increase, because they will continue to pay FICA taxes or self-employment taxes and won't be able to offset those taxes with the EITC.

Most itemized deductions, including those widely used for state and local tax expenses, would also be eliminated. However, the plan would preserve the deductions for mortgage interest expenses and charitable giving and keep incentives for education and retirement savings plans.

ESTATE TAXES:

The estate tax and generation skipping transfer tax would be repealed. The future of the gift tax is not expressly mentioned, but presumably it would be repealed as well.

BUSINESS TAXES:

The proposal calls for reducing the corporate tax rate to 20 percent from 35 percent[.] 

A new tax rate would be created for so-called pass-through businesses. These businesses, partnerships and sole proprietorships whose profits “pass through” to their owners, would be taxed at a [maximum] rate of 25 percent, not the individual rate of their owners, like under the current law. 

It will be left to Congress to create safeguards that prevent wealthy individuals from incorporating as pass-through businesses, which would tax their income at a lower rate. Another administration official insisted that measures would be put in place so that there are not “games played” in this regard.

This includes sole proprietorship income. Interest deductions for pass-through businesses might also be limited. 

In practice, preventing "games played" is exceedingly difficult in the proposed framework and the nature of the tax code provisions put in place to prevent revenue from flowing to pass through entities to escape higher tax rates will have a huge impact on the proposal's revenue impact. Much of existing corporate tax law exists precisely to prevent this kind of tax base leakage with only modest success.

Another big change for companies would be a limitation of the deductibility for corporate interest expenses in exchange for the opportunity to immediately expense business investments. The ability to write these expenses off immediately would last only five years, and the limitations for deducting interest have yet to be determined. . . .

Section 199 treatment of "domestic manufacturing" income would end.

The plan also calls on the tax committees to eliminate most of the tax credits that businesses currently use. Among those that would remain are the prized tax credit for research and development and the low-income-housing credit[.]

INTERNATIONAL BUSINESS TAXATION:

Currently U.S. individuals and businesses are taxed on their worldwide income less a credit for taxes paid abroad on foreign income and an exclusion for certain wage and salary income earned abroad. This would be replaced by a "territorial tax system". 

In theory this means that companies would not be taxed on their overseas earnings, but to prevent erosion of the tax base, Republicans plan to impose some form of tax on foreign profits at a rate that has yet to be determined. 

The transition to the new system would also include a one-time repatriation tax to encourage companies to bring offshore profits back to the United States. There would be different repatriation rates for different types of assets, but as with many parts of the proposal, the rates would be up to Congress to decide.

It isn't clear how the IRS will continue to tax income from easily moveable intangible assets of big businesses like intellectual property rights, investment income, controlled life insurance companies, and the like.

NOT MENTIONED:

The New York Times account does not address some other critical questions that could emerge as the details are worked out.

* Will the tax treatment of capital gains change in any way, in terms of special rates, treatment at death, or other policies favorable to capital gains income?

* Will special treatment for "qualified dividends" of corporations change?

* Will the pass-through taxation rate apply to income earned by trusts and estates?

* How will health insurance benefits (especially for the self-employed) be taxed?

* Will the taxes that fund the Obamacare subsidies be repealed?

A Narrative To Understand STD Statistics

This year, just like every year in recent memory, the nation's STD statistics for 2016 tell basically the same story. This post seeks to make sense of these statistics with a somewhat oversimplified narrative, some links of which are very well documented, and others of which are merely the most plausible explanations of the data.

Some Facts

* African-Americans experience STDs at the highest rate followed by Native Americans (including Native Hawaiians), followed by Hispanics, for pretty much every STD. White and Asians have the lowest rates. Often, but not always, people with more than one race tend to track the lower risk populations.

* HPV infections, which can sometimes cause cervical cancer, but is otherwise non-symptomatic is about 4.5 times as common as the next most common form of STD, chlamydia. STD rates in black teens were almost two and a half times as great as in white and Mexican-American teens in a Denver based study.

* According to the same Denver based study, STD rates are also more than four times higher in teens who are incarcerated in juvenile detention facilities than in those who are not.

* Chlamydia, at 1.6 million case a year in the U.S. is much more common than gonorrhea at 469,000 cases as year, which in turn is much more common that syphilis at 27,814 cases per year. All three  of these STDS can be cured with antibiotics if caught soon enough and if the strains are not antibiotic resistant, but syphilis is much more serious a health risk than chlamydia or gonorrhea. Many other serious STDs like AIDs (caused by HIV infection) and hepatitis cannot be cured by antibiotics and are also more rare than gonorrhea. About two-thirds of chlamydia cases affect women, while gonorrhea affects about 40% more men than women. Antimicrobial-resistant gonorrhea is much more common in men who have sex with men than in the general population.

* About half of male chlamydia cases (about 16% of the total) involve men who have sex with men (somewhat less than half in men aged 24 and younger and more than half in men 25 and older with MSM predominant among male chlamydia sufferers in men 40 years old and older). The sample used, however, is not fully representative.

* About 45% of gonorrhea cases involve men who have sex with men, although there is considerable regional variation in this figure (for example, the percentage is, unsurprisingly, highest in San Francisco) and the sample used is not fully representative.

* Syphilis is predominantly experienced by men (about 88% of all cases). About 80% of the men with syphilis (and about 70% of all people with syphilis) are "men who have sex with men" (MSM) and are disproportionately black and Hispanic. About 47% of syphilis sufferers are also HIV positive.

MSM accounted for 80.6% of male P&S syphilis cases with information about sex of sex partners in 2016. Of MSM P&S syphilis cases, 36.8% were White, 29.1% were Black, and 24.0% were Hispanic. Relative to the percentage of the US population that is White (62.3%), Black (12.3%), and Hispanic (17.1%), this represents a significant inequality in the burden of disease for non-White MSM, which was also evident among MSW and women. In addition, among MSM P&S syphilis cases with known HIV status in 2016, 47.0% were also reported to be HIV-positive.

While the report doesn't otherwise discuss HIV or other serious STDs like hepatitis, these infections follow a similar pattern.

* There are about 37,600 new HIV infections each year in the U.S. and the vast majority of those cases involve men who have sex with men, injection drug users, and their black and Hispanic female partners and children. Of them, 27,577 (73% of new HIV infections) are men who have sex with men (of whom only about 7,570 are white MSM).

* Among women, there are about 980 new HIV cases from injection drug use and 6,391 new cases from heterosexual contact (mostly with men in high risk categories) each year. These involve 5,152 black or Hispanic women, 968 white women and about 1,251 women of other races and ethnicities.

* Among men who are not MSM, there are about 1,412 new HIV cases from injection drug use and 2,948 new cases from heterosexual contact each year. These involve about 1,926 black men and 2,334 men of other races or ethnicities (a bit less than 2,000 of whom are white).

* About 117 new HIV cases each year involve blood based transmission (including 34 children) and about 86 new cases each year are present at birth.

* About 1.1 million people in the U.S. have HIV and about 1 in 7 of them don't know that they do.

A Narrative To Explain The Facts

Non-gay, non-bisexual men who have sex with men, and many women and children with STDs

One key to understanding the statistics is that lots of "men who have sex with men", especially black, Hispanic and Native American men who have sex with men, are not gay or bisexual in the usual sense of having a natural sexual attraction to and sexual preference for other men relative to women.

Instead, a large share of MSMs in the United States are men who had sex with other men (at least initially) in jail or prison while incarcerated, despite being primarily or predominantly sexually attracted to women. Often, those sexual interactions involve prison rape, pure and simple, by leaders of the prison gang hierarchy who choose to have sex with men as a second best substitute to sex with women against victims who are at the bottom of the prison hierarchy and have no choice. Sometimes, the pressure for non-gay, non-bisexual men to serve leaders of the prison gang hierarchy is not physical force, but the threat of force or the promise of reward in a prison's crude barter economy.

Caveat: This hypothesis is not a great match to direct survey data of the general public, but given the sensitivity of a subject like same sex prison rape, it isn't unreasonable to think that the most plausible inferences from STD diagnosis based data collected by treating medical professionals is likely to be more reliable than random survey data collected by government bureaucrats or contractors hired on their behalf. Also, the percentage of men who admit to having sex with other men is significantly higher than the percentage of men who identify as gay, and the transmission rate of STDs from black men who have sex with men to black women appears to be significantly higher than for white men.

(Black and Hispanic men are also overrepresented in the military were long periods of time without access to female sexual partners is common. It isn't clear to what extent male service members have sex with men despite not having a predominantly gay or bisexual sexual orientation. No epidemiological studies of which I am aware identify military veterans as a population with above average risk, but that could be because the risks faced by veterans who are men who have have had sex with men is fully captured by the MSM risk factor, discouraging further investigation.)

A significant share of the women with syphilis and HIV are sexual partners of ex-cons who are infected when they have sex with their men after they are released.

Injection drug users who share needles spread these diseases from infected people to other men and women who are injection drug users and their sexual partners, as well providing an important source of STD infections to prison populations full of injection drug users who then end up being men who have sex with men.

Infants born with these STDs have mothers with STDS who tend to be partners of ex-cons, to be injection drug users, and/or to be partners of injection drug users.

Gay and bisexual men, especially if they are black, Hispanic or Native American

Certainly, STD rates are also much higher among genuinely gay and bisexual men who have sex with men than in other populations. There are several factors that drive this trend.

This is in part because a fairly high baseline rates of infection with STDs including AIDs were spread throughout the community of gay men before the threat was recognized (it was first recognized as a disease in 1981 but arrived around 1969) and before these conditions spread widely among straight men who weren't drug users and hadn't been in prison. But, greater awareness of the threat led to more careful efforts to avoid infection before a high baseline rate of infection was established in heterosexual community. (In contrast, in Africa, where AIDS originated, a high baseline rate of HIV infection was established in the heterosexual population before there was a widespread awareness of the risk.)

This is in part because gay and bisexual men have probably historically been less monogamous than men who have sex with women only, bisexual women and lesbians. Greater cultural acceptance of gay men and the legalization of same sex marriage may change this situation.

This is also because gay and bisexual men, not being concerned about pregnancy, use barrier contraception at lower rates than men who have sex with women (and this was true to a much higher degree before there was widespread awareness of the HIV threat), and also because anal sex is probably a more efficient transmitter of STDs than vaginal sex or lesbian sex, and because barrier contraception (i.e. condoms) are more likely to fail in anal sex involving men than in the vaginal sex involving a man and a woman for which they were primarily designed.

This is in part because men infected in prison after having had sex with other men, offer sex to other men who were not prison inmates as male prostitutes or as "survival sex" once they are out of prison because their economic prospects are bleak. This option can be more important percentage-wise for gay and bisexual men seeking sex with men because there are a lot fewer gay and bisexual men out there than there are straight and bisexual women combined, and a lot of the gay and bisexual men in lots of places are closeted and thus hard to find as fully consensual partners. This is a particular risk for black and Hispanic gay and bisexual men who are more likely to have connection to former inmates, who are particularly scarce making resort to prostitutes and transitory partners more common, and who, on average, also have less adequate access to health care that could cure these infections and could interrupt their spread.

Conclusions

The bad news is that government mismanagement of correctional institutions is responsible for a major public health threat from serious STDs that disproportionately harms black, Hispanic, Native American, gay and bisexual men.

The good news is that the government has a great capacity, should it choose to put its mind to it, to reform how it manages jails and prisons, and how it provides health care to inmates and former inmates that is pertinent to controlling the public health risk from STDs. For example, STD testing and treatment for inmates who are incarcerated and those approaching their release dates should be routine. The government also has considerable capacity to reduce incarceration rates which would greatly reduce the share of the population exposed to these extremely high STD risk experiences.

The government also has a great capacity, proven in other countries, to reduce the spread of serious STDs through the exchange of needles by injection drug users.

So, two of the principle means by which the most serious STDs are spread in the United States are very amenable to correction through public policy changes. 

Also, the fairly distinct populations at highest risk of serious STDs are quite manageable targets for public relations campaigns raising awareness of the risks and the means by which those risks can be mitigated, if there is the political will to spread those messages (something that can be done in the private sector as well as by government).

It is also the case that the risk of STDs (and especially of serious STDs like syphilis, antibiotic resistant gonorrhea and HIV) to white and Asian people who are heterosexual men and women, lesbians, and bisexual women, who don't have ex-cons or injection drug users or men who have sex with men of any type in their intimate social circles (collectively, the lion's share of the U.S. population), face risks of STDs (and especially risks of serious STDs) that is much lower than one would expect knowing only the overall averages. The current situation is much better than it was historically when STDs spread by heterosexual prostitutes particular to military service members were a much more serious problem.

The downside of the fairly focused risk group for serious STDs, however, is that it makes mobilizing the political will to change the situation very difficult, particularly because those most affected tend to have particularly little political clout.

Puerto Rico and Maria

Puerto Rico, which was already in fiscal crisis, has been devastated by Hurricane Maria.

What should be done?

Should large numbers of Puerto Ricans, at least temporarily, relocate to the mainline during the rebuilding period as many people from New Orleans and the vicinity did after Hurricane Katrina? 

A fair number of NOLA refugees were gone for good, permanently reducing the population of the city and the state of Louisiana. Does that make sense for Puerto Rico?

If not (or in any case for those who do not leave), how can the privation existing on the island be mitigated much more quickly than it is being mitigated right now? 

Should we be taking this opportunity to rebuild in a manner that is more resistant to Hurricane damages knowing that climate change makes the hurricane risk going forward higher than it has been historically?

For example, are there ways to rebuild the electrical and communications grid more robustly? How can the water supply system be made more hurricane-proof? Are there ways to make its dams less at risk of failure? Should building codes be enforced strictly now despite the shortage of inspectors, limited resources for construction, and need for rapid rebuilding work that works at odds with dealing with the current crisis? Should roads be rebuilt more sturdily, or does it make more sense to acknowledge that some infrastructure will be regularly destroyed by weather than invent only an amount that reflects its shorter than average useful life? Do there need to be better shelters in place? 

Remarkably the death toll in Puerto Rico seems to have been pretty modest for the scale of the disaster although we may not yet know everything yet since communications are still down across the island.

Are there places in Puerto Rico that are "stupid zones" that shouldn't be rebuilt (at least not with government flood insurance, etc.)?

Also:

Worse yet, the island has about $123 billion in debt and pension obligations, compared with a gross domestic product of slightly more than $100 billion, a number that is sure to fall. In the last decade, the island has lost about 9 percent of its population, including many ambitious and talented individuals. In the past 20 years, Puerto Rico’s labor force shrank by about 20 percent, with the health-care sector being especially hard hit. The population of children under 5 has fallen 37 percent since 2000, and Puerto Rico has more of its population over 60 than any U.S. state.

Via Marginal Revolution.

Maybe migration to the mainland that was already underway will simply be sped up by Maria.

Property Crime Down For 14th Straight Year In U.S.

The News

The estimated number of violent crimes in the nation increased for the second straight year, rising 4.1 percent in 2016 when compared with 2015 data, according to FBI figures released today. Property crimes dropped 1.3 percent, marking the 14th consecutive year the collective estimates for these offenses declined. 

The 2016 statistics show the estimated rate of violent crime was 386.3 offenses per 100,000 inhabitants, and the estimated rate of property crime was 2,450.7 offenses per 100,000 inhabitants. The violent crime rate rose 3.4 percent compared with the 2015 rate, and the property crime rate declined 2.0 percent.

From the FBI relying on Table 1 of the report for which this is a press release.

Property Crime

Property crimes rates per 100,000 people have decreased steadily for 15 years from 2001 to 2016, the most recent year for which data is available (one more year than the number of years the raw number of property crimes has declined since from 2001 to 2002 in increase in crime incidents was overshadowed by an increase in population).

The 2001, property crime rate per 100,000 people was 3,658.1 while in 2001 it was 2,450.7, a decline of 33% in 15 years.

Also, the 2000 to 2001 slight increase in the property crime rate was only a minor blip in a longer term downward trend. In 1997, the property crime rat per 100,000 people was 4,316.3. The 2016 property crime rate is 43% lower than it was in 1997.

Looking at sub-components of property crime:

* The burglary rate is down 49% from 1997 to 2016.

* The larceny rate is down 40% from 1997 to 2016.

* The motor vehicle theft rate is down 53% from 1997 to 2016 despite increasing from 2014 to 2016. The peak decline the motor vehicle theft rate in 2014 from 1997 was 57%. Some of this increase in excess of overall crime rate reductions may reflect technology that makes it harder to steal cars.

Burglaries and motor vehicle theft (and robberies) all net a greater average value of property stolen than simple larceny, so it is increasingly hard to earn income from property crime.

It seems plausible to me that some of the decline in the larceny rate may reflect decreased reporting as the value of tangible personal property has fallen relative to other forms of spending. Stuff that can be stolen is cheaper than it used to be and may not be worth reporting as stolen.

Violent Crime

The violent crime rate in 1997 per 100,000 people was 611.0. The 2016 violent crime rate of 386.3 per 100,000 people is 37% below the rate in 1997.

The violent crime rate hit its lowest recent point in 2014 at 361.6 per 100,000 people (41% below the peak). The violent crime rate in 2016 was about 7% above the low point.

* The robbery rate is down 45% from 1997 to 2016 and increased only 1.5% since its 2014 low point.

The 538 blog looks at the trends in murder rates, an analysis summed up in this chart:

It also notes that declines since a peak around 1991 (25 years ago) have been even greater. Robbery, burglary and auto theft (all of which are big ticket property crimes) are all down more than 62% since 1991. Murder, rape, aggravated assault and ordinary theft have fallen less strongly.

538 also offers some insights on non-index crime drug law enforcement trends:

Incarceration Rates Compared

Historically, incarceration rates soared between 1974 and 1994 (more than ten-fold over twenty years), and then continued to rise more modestly until the peaked in the period from 1994 to 2006 (about 20% over 12 years). Since 2006 incarceration rates have declined by about 11% over a decade from the peak incarceration rate.

In 2015, there were 1,476,847 people in state and federal prisons for a U.S. population of 320,896,618, a rate of 460 per 100,000 people.

In 1997, there were about 1,350,000 people in state and federal prisons for a U.S. population of 267,783,607, a rate of about 504 per 100,000 people.

At the peak raw number of people in state and federal prisons in 2006 of about 1,550,000 in state and federal prisons, when the population was 299,398,484, the rate was about 518 per 100,000 people.

Thus, during ten years during which the rate at which people were incarcerated in state and federal prisons has fallen, property crimes rates have continued to fall steadily and violent crime rates fell for eight of those ten years.

More on incarceration and crime in a Marginal Revolution post that reports a report that the marginal crime reduction from more incarceration in the U.S. has hit zero or might even be negative.

Analysis

Good News

While a couple of years of increases in violent crimes in the last two years (which other evidence suggests is mostly confined to a handful of cities) is not good news and generally speaking, we care more about violent crime than property crime, the twenty year trend in U.S. crime rates is still remarkable positive.

Mostly A Common Trend

Despite the fact that property crime rates and violent crime rates don't exactly track each other, the trends are close enough to each other (37% v. 43% over a decade) to infer that most of the decline in crime rates has a common source (also robbery, which is classified as a violent crime and makes up about 25% of the violent crime total, is also a crime that involves property).

Some of this is due to a shrinking "crime age" population as a share of the total population, but that is a pretty modest share of the total decline. Abortion rates and less lead pollution have also been credited with a targeted decline in the "at risk" crime age population, and these factors could credibly account for a larger share of the total although also certainly not all of it.

Teens Are Also Better Behaved

It also comes at a time when teen pregnancy, alcohol use and drug use are all at record lows. Fewer teens are dropping out of high school and more are going to college.

Adult Blue Collar Men Despair 

Opiate abuse (with overdose deaths), alcohol abuse, disabilities at younger ages and suicide are all up for blue collar adult men, but this apparently hasn't translated into higher crime, although an examination of the state specific data might cast more light on why other forms of despair don't seem to have produced much more crime.

Drunk driving (and traffic deaths generally), however, is down significantly.

The Economy Has Improved In The Long Run

There has been economic growth over the last 20 years, although a minor (2001) and a severe (2008) recession along the way didn't seem to increase crime rates, and most of the benefits of that economic growth has accrued to the wealthy who were already committing crimes at very low rate. Increased economic inequality has apparently not increased crime.

Incarceration Compared

It has happened despite declining rates of incarceration in state and federal prisons.

Incarceration rates have fallen much more slowly than crime rates, however. Crime rates were decreasing significantly from 1997 to 2006, while incarceration rates continued to rise. Crime rates have fallen much faster than incarceration rates since 2006.

Admittedly, incarceration is a lagging indicator, but we are now at 1970s crime rates, but we have early 1990s incarceration rates, and an average prison sentence is only several years. Sentences for crimes continued to get longer even as crime rates fell.

It is entirely possible that we incarcerated too few people in the 1970s thereby exposing the public to people who posed a high crime risk (although why were crime rates so low with so few people incarcerated in that case?). But, we are almost certainly incarcerating too many people now with the highest (or nearly the highest) incarceration rate in the world and a ratio of crime rates to incarceration rates that continues to increase.

We are getting less bang for our buck from longer sentences now because early on, longer sentences put away people who would have been released before "aging out" of the high crime age bracket, while now lots of people sentences to long sentences long ago have aged out and represent a much more modest threat to the public than they did earlier, but are still incarcerated.

Deinstitutionalization

Deinstitutionalization of the mentally ill was one major factor that coincided with a rise in incarceration. The number of people in inpatient mental health care fell by something like 900,000 while the number of people now under criminal justice system supervision is about 800,000.

Medical Care

Improved medical care could explain part of the decline in the murder rate, but not the decline in other kinds of violent crimes that has happened at the same time.

Has This Reduced Property Insurance Rates?

While it is a question for another day, it doesn't seem that property insurance rates have fallen despite the dramatic decline in property crimes. Perhaps crime is a very minor factor in property insurance claims relative to accidental damage to property (e.g. from storms and fires) and perhaps inflation overstates the relative property insurance amounts in real dollar terms. But, it is worth looking into at some point.

Vera Says Vive La Difference!

A guide to the 2016 FBI statistics from the Vera Institute emphasizes variation in crime rates from place to place with "hyper-local" problems driving the trends this year.

UPDATE October 9, 2017:

A Longer Term Perspective

These data points can be integrated with longer term crime data. This suggests that as of 2009, we were at a 100 year low in crime rates in the U.S., and probably lower than any earlier period in United States history. So, the crime rate in 2014 may be the lowest of all time in the United States.

Reinventing The Wheel

A study analyzing crime data in Philadelphia for 10 years found that rates of violent crime and disorderly conduct are higher when the weather is warmer and more pleasant, even rising sharply during warmer-than-typical winter days.

From here.

This finding would be much more impressive if it weren't for the fact that this conclusion was considered to be established enough to put in the Social Psychology textbook that one of my parents used in college, sometime at least 50 years ago. It is nice that the finding replicates, but the press release breathlessly reports it as if it is something new under the sun. The paper is:

Leah H. Schinasi, Ghassan B. Hamra. "A Time Series Analysis of Associations between Daily Temperature and Crime Events in Philadelphia, Pennsylvania." Journal of Urban Health (2017); DOI: 10.1007/s11524-017-0181-y

Gene Therapy Cures MS In Mice

An experimental gene therapy has cured MS in mice.

The paper is:

Geoffrey D. Keeler, Sandeep Kumar, Brett Palaschak, Emily L. Silverberg, David M. Markusic, Noah T. Jones, Brad E. Hoffman. "Gene Therapy-Induced Antigen-Specific Tregs Inhibit Neuro-inflammation and Reverse Disease in a Mouse Model of Multiple Sclerosis." Molecular Therapy (2017). DOI: 10.1016/j.ymthe.2017.09.001

The Little Known Tort Of Harming Others With A False Tax Return

It isn't every day that I discover a new tort after having practiced law for more than twenty years. So, it bears mentioning. I've been aware of the underlying improper tactic for a long time, but didn't know about the statutory remedy for it until now. The basic concern is that you can cause a tax mess for someone if you create a Form 1099 or K-1 which you issue to someone to them saying that they have earned income when they haven't, and there are very few limits on creating a 1099 or K-1 in the first place. 

[Internal Revenue Code] Section 7434(a) provides: “If any person willfully files a fraudulent information return with respect to payments purported to be made to any other person, such other person may bring a civil action for damages against the person so filing such a return.” . . . 

[There is a] surprisingly large amount of case law on this section to address a variety of common issues that arise in these lawsuits. For a good overview of these issues see Stephen's useful blog post here.

From TaxProf Blog.

The Marshmallow Test Results Have Improved Over Time

Children, on average, are significantly better at delaying gratification in the "marshmallow test" than they were 50 years ago. The abstract of the new paper is as follows:

Have children gotten worse at their ability to delay gratification? We analyze the past 50 years of data on the Marshmallow test of delay of gratification. Children must wait to get two preferred treats; if they cannot wait, they only get one. Duration for how long children can delay has been associated with a host of positive life outcomes. Here we provide the first evidence on whether children’s ability to delay gratification has truly been decreasing, as theories of technology or a culture of instant gratification have predicted. Before analyzing the data, we polled 260 experts in cognitive development, 84% of who believed kids these days are getting worse or are no different. Contrary to this prediction, kids these days are better able to delay gratification than they were in the past, corresponding to a fifth of a standard deviation increase in ability per decade. 

The magnitude of the change is comparable to that of the Flynn effect that was observed in the same time period, i.e. a secular increase in average IQ over time. As the introduction in the body text of the paper explains:

All cognitive abilities have undergone secular increases over the past century (Flynn, 1984). The increase generally runs between 2.3–3 points of overall intelligence per decade (around 1/5 of a standard deviation; Trahan et al., 2014).

The reason for this not known and this empirical result contradicts conventional wisdom in the field.

Some examinations of the Flynn effect have pointed to the improvements mostly coming from a reduction in the number of people scoring at the low end of IQ tests, rather than an improvement at the top, suggesting that a decline in developmental disabilities, perhaps due to better pre-natal care and reduced pollution (e.g. from lead exposure) or improved nutrition, could be a factor, which might also apply to the Marshmallow test improvements over time, even if the Marshmallow test implicates cognitive abilities orthogonal to IQ that involve different brain processes.

Notably, another mention of the related concept of "attention" has remained constant over time in a study comparing results in 1983 to those in 2012, despite increasing diagnosis of attention deficit disorder conditions.

UPDATE: The Marshmallow Test study may be seriously flawed. The truth may actually be  that  Marshmallow Test results are unchanged. Eliminating three very earlier outliers from 1968-1971 from the meta-analysis shows an unvarying trend line from 1971 to the present. The strong implication is that methodological flaws in the first few tests were not recognized and caused the time that children were willing to wait to be much shorter than in all subsequent replications of the study.

Thus, rather than supporting a Flynn effect for patience, it actually confirms the result that attention tests are unchanged in the similar time period which makes sense because the traits related to attention and the Marshmallow Test appear to be closely related (and also related to the Big Five personality trait "conscientiousness" and the trait known as "grit").

Word of The Day: Lakh

A lakh (/ˈlæk/ or /ˈlɑːk/; abbreviated L; sometimes written Lac or Lacs) is a unit in the Indian numbering system equal to one hundred thousand (100,000; scientific notation: 10^5). In the Indian convention of digit grouping, it is written as 1,00,000. For example, in India 150,000 rupees becomes 1.5 lakh rupees, written as ₹1,50,000 or INR 1,50,000. 

It is widely used both in official and other contexts in Bangladesh, Bhutan, India, Myanmar, Nepal, Pakistan and Sri Lanka. It is often used in Indian, Pakistani, and Sri Lankan English. In Pakistan, the word lakh is used mostly in local languages rather than in English media.

From Wikipedia based upon an encounter with it here.

Another notable insight, from the same source, is this: "The word lakhi is commonly used throughout Tanzania to denote 100,000 shillings and is likely to have entered the Swahili language from Indian and Pakistani immigrants."

A related term is Crore (= 100 lakh = 10 million).

U.S. Liberals Share The Same Ideas More Than Conservatives

Do liberals or conservatives have more agreement in their political attitudes? 

Recent research indicates that conservatives may have more like-minded social groups than do liberals, but whether conservatives have more consensus on a broad, national level remains an open question. 

Using two nationally representative data sets (the General Social Survey and the American National Election Studies), we examined the attitudes of over 80,000 people on more than 400 political issues (e.g., attitudes toward welfare, gun control, same-sex marriage) across approximately 40 years. 

In both data sets, we found that liberals possessed a larger degree of agreement in their political attitudes than did conservatives. Additionally, both liberals and conservatives possessed more consensus than did political moderates.

Peter Ondish, Chadly Stern, "Liberals Possess More National Consensus on Political Attitudes in the United States: An Examination Across 40 Years" Social, Psychological and Personality Science (September 14, 2017 ).

This is contrary to what has been conventional wisdom had been until quite recently (perhaps since 2015 when apparent fractures in the conservative coalition started to become more apparent in the election campaign).

Mexican Migration Surge Unlike To Recur

Trump's wall is an idea that has been obsolete for a long time.

Net migration from Mexico to the U.S. has been approximately zero for a decade.

One reason that it is not likely to recur because Mexico no longer has a rapidly growing population.

Source.

A Short 9-11 Observation

Sixteen years ago, in the wake of the 9-11 attacks, I was convinced as were most people, that the lives of my infant and two year old would be defined by a world wracked by terrorism.

This isn't what happened. In the subsequent sixteen years, there has been more domestic terrorism in the U.S. than there has been Islamic terrorism, and phenomena like gun violence, police brutality, the gay rights movement, and global warming have all had more of an impact on my children's lives than the kind of terrorism that gave rise to 9-11.

Yes, the U.S. is still fighting the war in Afghanistan that begun with 9-11 and our involvement in Syria and Iraq at this point is justified by the same legal authority that provides a basis for the ongoing war in Afghanistan. But, those conflicts have remained "small wars" that have managed to remain at the periphery of American life.

Quote Of The Day

Even if he betrays me, I'd be O.K. with it.

- Vampire Knight anime (English dubbed), Season 2, Episode 3.

It is hard to image loyalty greater than that.

Autism and mtDNA?

A new study finds a link between mtDNA clades and autism prevalence. While there is a plausible biochemical mechanism by which this link could arise, based upon other work I've seen on inheritance patterns in autism, I'm quite skeptical of the result.

The current study analyzed these mtDNA lineages among 1,624 patients with autism and 2,417 healthy parents and siblings, representing 933 families in the Autism Genetic Resource Exchange (AGRE). Using data from genome-wide association studies on this AGRE cohort previously performed by the Center for Applied Genomics at CHOP, they determined patterns of functional mtDNA variants associated with ASD risk that emerged over human history. 

The study team found that individuals with European haplogroups designated I, J, K, O-X, T and U (representing 55 percent of the total European population) had significantly higher risks of ASD compared to the most common European haplogroup, HHV. Asian and Native American haplogroups A and M also were at increased risk of ASD. 

These findings support the idea that an individual already predisposed to ASD based on their mitochondrial haplogroup may develop the disease when additional genetic variants or environmental insults occur that lower mitochondrial function, impair OXPHOS, and alter brain activity.

Among the reasons to be skeptical: the p-value is only 0.04 so it looks like p-hacking by carefully choosing results bins after the fact; the sample size is exaggerated because the samples are not independent, the characterization of the mtDNA haplogroups involved is non-standard (mtDNA haplogroup O is Australian aboriginal, so a European haplogroup O-X makes no sense; there is no mtDNA haplogroup HHV and instead there are two separate ones H and HV); the odds ratios aren't huge; mtDNA clades are strongly ancestry informative and could be tracking differences in diagnosis rates or ancestry correlated autosomal traits; these are very broad categories that aren't phylogenically coherent.

More evidence of p-hacking in this paper flows from the fact that a 2013 study with a similar design and only modestly smaller sample size found no significant association between mtDNA haplogroup and autism risk and did not show the autism-mtDNA links for  any of the specific haplogroups reported on in the 2017 paper, despite having a similar, although moderately smaller sample size in terms of independent individuals (818 cases and 1641 controls). The abstract of that paper and its citation are as follows:

Despite the increasing speculation that oxidative stress and abnormal energy metabolism may play a role in Autism Spectrum Disorders (ASD), and the observation that patients with mitochondrial defects have symptoms consistent with ASD, there are no comprehensive published studies examining the role of mitochondrial variation in autism. Therefore, we have sought to comprehensively examine the role of mitochondrial DNA (mtDNA) variation with regard to ASD risk, employing a multi-phase approach. In phase 1 of our experiment, we examined 132 mtDNA single-nucleotide polymorphisms (SNPs) genotyped as part of our genome-wide association studies of ASD. In phase 2 we genotyped the major European mitochondrial haplogroup-defining variants within an expanded set of autism probands and controls. Finally in phase 3, we resequenced the entire mtDNA in a subset of our Caucasian samples (∼400 proband-father pairs). In each phase we tested whether mitochondrial variation showed evidence of association to ASD. Despite a thorough interrogation of mtDNA variation, we found no evidence to suggest a major role for mtDNA variation in ASD susceptibility. Accordingly, while there may be attractive biological hints suggesting the role of mitochondria in ASD our data indicate that mtDNA variation is not a major contributing factor to the development of ASD.

Hadjixenofontos, A, et al., "Evaluating mitochondrial DNA variation in autism spectrum disorders." 77(1) Ann Hum Genet. 9-21 (2013 epublished November 6, 2012) doi: 10.1111/j.1469-1809.2012.00736.x. (Open access).

Another lack of association between ASD and mtDNA variation was found in a 2011 study. The 2011 and 2013 studies combined have as large a sample size, if not larger, than the 2017 paper.

So, two of three studies have found a lack of association and the one that did find the association has a p value of 0.04 which is not statistically significant after considering look elsewhere effects given that at least three such studies have been done.

Background on sibling shared autism risk can be found here. All siblings with the same mother have the same mtDNA. The odds ratio for siblings of ASD affected individuals to have an ASD is 9.4-14.7 which is much greater than the odds ratio associated with sharing an mtDNA haplogroup.

Ideally, mtDNA associations would be studied at maximal sub-haplogroup detail, in light of a phylogeny of the haplogroups implicated, and excluding cases where a de novo mutation was likely such as in cases of advanced paternal age, as well as cases where a paternal inheritance was likely due to sub-clinical or clinical ASD symptoms in the father.

The abstract and citation to the new paper are as follows:

Importance Autism spectrum disorders (ASD) are characterized by impairments in social interaction, communication, and repetitive or restrictive behavior. Although multiple physiologic and biochemical studies have reported defects in mitochondrial oxidative phosphorylation in patients with ASD, the role of mitochondrial DNA (mtDNA) variation has remained relatively unexplored. 

Objective To assess what impact mitochondrial lineages encompassing ancient mtDNA functional polymorphisms, termed haplogroups, have on ASD risk. 

Design, Setting, and Participants In this cohort study, individuals with autism and their families were studied using the Autism Genetic Resource Exchange cohort genome-wide association studies data previously generated at the Children’s Hospital of Philadelphia. From October 2010 to January 2017, we analyzed the data and used the mtDNA single-nucleotide polymorphisms interrogated by the Illumina HumanHap 550 chip to determine the mtDNA haplogroups of the individuals. Taking into account the familial structure of the Autism Genetic Resource Exchange data, we then determined whether the mtDNA haplogroups correlate with ASD risk. 

Main Outcomes and Measures Odds ratios of mitochondrial haplogroup as predictors of ASD risk. 

Results Of 1624 patients with autism included in this study, 1299 were boys (80%) and 325 were girls (20%). Families in the Autism Genetic Resource Exchange collection (933 families, encompassing 4041 individuals: 1624 patients with ASD and 2417 healthy parents and siblings) had been previously recruited in the United States with no restrictions on age, sex, race/ethnicity, or socioeconomic status. Relative to the most common European haplogroup HHV, European haplogroups I, J, K, O-X, T, and U were associated with increased risk of ASD, as were Asian and Native American haplogroups A and M, with odds ratios ranging from 1.55 (95% CI, 1.16-2.06) to 2.18 (95% CI, 1.59-3) (adjusted P < .04). Hence, mtDNA haplogroup variation is an important risk factor for ASD. 

Conclusions and Relevance Because haplogroups I, J, K, O-X, T, and U encompass 55% of the European population, mtDNA lineages must make a significant contribution to overall ASD risk.

Dimitra Chalkia, et al., "Association Between Mitochondrial DNA Haplogroup Variation and Autism Spectrum Disorders" JAMA Psychiatry. (Published online August 23, 2017). doi:10.1001/jamapsychiatry.2017.2604

Triage

One of the behind the scenes things that lawyers spend a fair amount of time doing is triage evaluation of potential clients.

Few people other than lawyers in private practice have such a keen appreciation for the myriad situations that may involve legal issues to some extent, but for which they are not cost effective solutions, or even for which they are not effective solutions at all.

Private attorneys are excellent solutions to some kinds of problems (e.g. unpaid debts owed by solvent debtors), and are horrible solutions to many other kinds of problems.

Heritability Estimate For Schizophrenia

About 79% of schizophrenia variance is explained by genetics.

A Short History of Modern Terrorism

A prepared a fairly extensive answer to a question at Politics StackExchange about the history of modern terrorism. I'm reprinting my answer in this post.

Short Answer

Modern terrorism as we understand it today originated in the late 1960s and has gradually spread across different movements as a tactic, including Islamic terrorism movements, since then.

The rise of this tactic in the Middle East more or less coincided with its appearance in places like Ireland, Continental Europe and Sri Lanka by non-Islamic forces and with the rise of asymmetric warfare tactics in the Vietnam War.

This tactic arose for military reasons, because alternative means of conventional warfare became less viable.

These tactics were used at all, in the Middle East, initially, as emerging fundamentalist Islamic movements fueled by rising literacy the removed the intermediation of formally trained religious leaders interpreting religious texts. This grass roots fundamentalist Islamic movement (in the sense of getting back to fundamentals of religious texts with less interpretation) arose as what was seen as a widely unifying less corrupt means to counterbalance the twin opponents of Israel on one hand, and secular leaning totalitarian communist and absolute monarchs that were perceived as oppressive, on the other. The absolutist regimes grew indifferent to the masses whose fundamental economic productivity lagged behind rising wealth from oil facilitated by OPEC at first, while those with access to oil wealth prospered.

The regimes against which Islamic terrorism was directed came to power in a manner that arose from missteps in the process of breaking up the Ottoman Empire and flawed process of facilitating Zionism without unduly disrupting pre-existing residents in a massively disruptive way that led to multiple rounds of war. 

Long Answer

There is more than one meaningful answer to your question considered at different levels of remoteness and with different kinds of causation. Obviously, this answer could stretch to many books in length. This answer attempts to hit the most pivotal high points and to demonstrate what was happening by example where possible.

The Long War Of Ottoman Succession

To some extent, almost all of the conflicts in the Middle East are partially traceable to the poor job that was done partitioning the Ottoman Empire, which was dismantled by the winning "Great Powers" following World War I. As Wikipedia summarizes at the link in this paragraph (references omitted):

The partitioning of the Ottoman Empire led to the rise in the Middle East of Western powers such as Britain and France and brought the creation of the modern Arab world and the Republic of Turkey. Resistance to the influence of these powers came from the Turkish national movement but did not become widespread in the post-Ottoman states until after World War II.  

The League of Nations mandate granted French Mandate for Syria and the Lebanon and British Mandate for Mesopotamia (later Iraq) and British Mandate for Palestine, later divided into Mandatory Palestine and Emirate of Transjordan (1921-1946). The Ottoman Empire's possessions in the Arabian Peninsula became the Kingdom of Hejaz, which was annexed by the Sultanate of Nejd (today Saudi Arabia), and the Mutawakkilite Kingdom of Yemen. The Empire's possessions on the western shores of the Persian Gulf were variously annexed by Saudi Arabia (Alahsa and Qatif), or remained British protectorates (Kuwait, Bahrain, and Qatar) and became the Arab States of the Persian Gulf.   

After the Ottoman government collapsed completely it signed the Treaty of Sèvres in 1920. However, the Turkish War of Independence forced the European powers to return to the negotiating table before the treaty could be ratified. The Europeans and the Grand National Assembly of Turkey signed and ratified the new Treaty of Lausanne in 1923, superseding the Treaty of Sèvres and solidifying most of the territorial issues. One unresolved issue, the dispute between the Kingdom of Iraq and the Republic of Turkey over the former province of Mosul was later negotiated under the League of Nations in 1926. The British and French partitioned the eastern part of the Middle East, also called Greater Syria, between them in the Sykes–Picot Agreement. Other secret agreements were concluded with Italy and Russia. The Balfour Declaration encouraged the international Zionist movement to push for a Jewish homeland in the Palestine region. While a part of the Triple Entente, Russia also had wartime agreements preventing it from participating in the partitioning of the Ottoman Empire after the Russian Revolution. The Treaty of Sèvres formally acknowledged the new League of Nations mandates in the region, the independence of Yemen, and British sovereignty over Cyprus.

This doesn't tell the full story, however.

The trouble is that the new regimes were despotic. The process frequently put minority factions in positions of power over majorities. The process allowed those in power to be corrupt and ignore the popular will. The process did not accurately draw lines reflecting ethnic human geography in the region splitting ethnic groups between states and creating ethnic minorities that were oppressed.

But, for half a century, this fragile arrangement nonetheless held together more or less, until it was really stressed as colonial powers withdrew their control voluntarily in favor of local despots who had previously been restrained by colonial oversight, starting mostly around 1960. It took only a few years for the legacy regimes to prove unstable and be replaced by more stable, totalitarian arrangements.

The Anti-Israel Movement

Shifting multi-national coalitions of Middle Eastern countries attempted to wipe out Israel, after it gained formal international recognition in the wake of the Holocaust of World War II, in a succession of short wars including an Arab invasion of Israel in 1948, the June 1967 Arab-Israeli War, a failed attempt to recapture the Golan Heights in 1973, and other shots fired in anger which history does not dignify with formal names. Needless to say, they all failed.

Many Middle Eastern countries adopted a formal policy favoring the elimination of the state of Israel, and the Arab League, founded in 1945, had long organized opposition to Israeli treatment of Palestinians on a regional international level.

President Carter the brokered the Camp David Accords in 1978 which secured buy in from Israel's neighbor, Egypt, to not invade or try to wipe out Israel, in exchange for military and civilian economic aid and a desire for a more lasting peace. This resulted in a 1979 peace treaty between Israel and Egypt. The United States used its influence in a similar, but less formal way to disabuse Saudi Arabia (which does not have formal diplomatic relations with Israel) of its desire to eliminate the State of Israel around the same time. More or less, that peace has held since then.

But, the legacy of having almost all of the sovereign states of the region formally declaring Israel to be their enemy deserving destruction or at least woefully opposed due to its treatment of the Palestinians, the cause of the ill treated Palestinians displaced by Zionist migration of Jews to Israel, and a shift towards religious fundamentalism discussed below, all legitimatized those who turned to terrorism as a tactic to use against Israel in this ongoing territorial and religious clash of international proportions.

Terrorist attacks in Israel start to surge in the late 1960s:

The Fall of Western Style Societies In The Middle East

While the partition of the Ottoman Empire set the stage for later terrorism in the modern sense, though in the 1960s and 1970s countries like Afghanistan, Lebanon, Egypt and Iran had moved very far in a Western direction under monarchies or socialist regimes, were relatively free of organized internal violence, and were experiencing expanding economic and expression rights for their citizens, although most regimes were still fundamentally either one party regimes or monarchies.

For example, in 1964, the monarch of Afghanistan instituted a new liberal constitution that was supposed to transition the country from an absolute monarchy to a constitutional monarchy with an elected parliament, a strict regulation of daily life according to fundamentalist readings of Islamic law was relaxed.

A park in Afghanistan in the 1970s

These economies also tended to be somewhat hollow with weak fundamentals propped up with revenues from oil resources for which high prices were maintained by the OPEC oil cartel, formed in 1960, which provide prosperity that could support a liberal lifestyle.

A Conservative Religious Revival

One of the important consequences of this process, however, as described by my Islamic studies professors in college from first hand knowledge and intensive scholarship, was that the rate of literacy among "the masses" dramatically increased. This allowed large swaths of the population that previously had access to Islamic religious writings mediated through religious scholars who embedded those writings in interpretive doctrines that mitigated how they were applied in practiceto read them directly, to interpret them without the long religious tradition associated with them in a very literal way, and to become self-appointed religious leaders. This is explained here:

What would become Protestantism was inextricably linked to the advent of mass literacy, as a growing number of believers were no longer dependent on the intercession of clerics. With the New Testament translated for the first time into German and other European languages, the faithful could directly access the text on their own.  

The Muslim world, by comparison, has already experienced a weakening of the clerics, who, in being co-opted by newly independent states, fell into disrepute. In Europe, the decline of the clerical class and mass literacy laid the groundwork for secularization. In the modern Middle East, these same forces coincided with political Islam’s ascendancy. The Egyptian Muslim Brotherhood disproportionately drew its leadership from the professional sectors of medicine, engineering, and law. The movement, founded in 1928, was decidedly non-clerical and, in some ways, anti-clerical. In the 1950s, Cairo’s al-Azhar, the Arab world’s preeminent center of Islamic thought, was co-opted and politicized by Gamal Abdel Nasser’s regime, the Brotherhood’s chief antagonist.  

The much more literalist Salafis also had little time for the religious establishment. The premise of Salafism was that centuries of intricate and technical Islamic scholarship had obscured the power and purity of Islam, as embodied by the Prophet Muhammad and his companions. Salafi leaders told their followers that the Quran’s meaning could be accessed by simply reading it and following the example of the Prophet. Salafism—and for that matter groups like al-Qaeda and ISIS—would be inconceivable without the weakening of the clerics and the democratization of religion interpretation.

Then, existing conservative theological movements such as Salafi Islam (often with support from wealthy elites from Saudi Arabia and from elites in other Arab Muslim monarchies), who had resented and opposed the Westernization of these societies for their corruption and immoralitystepped in to lead the emerging grass roots movement.

Of course, the process is far more complex than this thumbnail sketch, and is influenced by a lot of complex internal structure of the relationship between different factions within Islam.

The transition bore strong similarities to the rise of Evangelical Christianity in the United States where independent, not very well educated grass roots revivalist preachers sought to return Christianity to its fundamentals after they were freed of the constraints and clergy dominated filter of religious knowledge to the masses and interpreted Christian religious texts literally based on their own readings of them without regard to later religious traditions in the Second Great Awakening.

The U.S. and the U.K. supported these movements at first, as Wikipedia explains (references omitted from quotation):

During the Cold War following World War II, some NATO governments, particularly those of the United States and the United Kingdom, launched covert and overt campaigns to encourage and strengthen fundamentalist groups in the Middle East and southern Asia. These groups were seen as a hedge against potential expansion by the Soviet Union, and as a means to prevent the growth of nationalistic movements that were not necessarily favorable toward the interests of the Western nations. By the 1970s, the Islamists had become important allies in supporting governments, such as Egypt, which were friendly to U.S. interests. By the late 1970s, however, some fundamentalist groups had become militaristic leading to threats and changes to existing regimes. The overthrow of the Shah in Iran and rise of the Ayatollah Khomeini was one of the most significant signs of this shift. Subsequently fundamentalist forces in Algeria caused a civil war, caused a near-civil war in Egypt, and caused the downfall of the Soviet occupation in Afghanistan. In many cases the military wings of these groups were supplied with money and arms by the U.S. and U.K.

A more detailed timeline of this involvement can be found in this 2006 article.

Needless to say, the U.S. and U.K. came to regret their support of these movements later on.

The current generation of people carrying out Islamic terrorist attacks in Western countries also carry some common characteristics; they are:

second generation; fairly well integrated at first; period of petty crime; radicalisation in prison; attack and death – weapons in hand – in a standoff with the police. 

Another characteristic that all western countries have in common is that radicals are almost all “born-again” Muslims who, after living a highly secular life – frequenting clubs, drinking alcohol, involvement in petty crime – suddenly renew their religious observance, either individually or in the context of a small group. . . . 

In almost every case, the processes by which a radical group is formed are nearly identical. The group’s membership is always the same: brothers, childhood friends, acquaintances from prison, sometimes from a training camp. The number of sets of siblings found is also remarkable. 

the radical preachers’ rhetoric could basically be summarised as: “Your father’s Islam is what the colonisers left behind, the Islam of those who bow down and obey. Our Islam is the Islam of combatants, of blood, of resistance.” 

Radicals are in fact often orphans . . . or come from dysfunctional families. They are not necessarily rebelling against their parents personally, but against what they represent: humiliation, concessions made to society, and what they view as their religious ignorance.

These men aren't crazy. These men see themselves a bit like Batman, wealthy men using their power and education to secure what they see as justice outside the law, or if less affluent, like the demon hunting brothers of Supernatural, or as medal of honor candidates who are heroes willing to put their lives on the line for the cause.

The cause, in turn, is born of religious revival in response to bad, foreign influenced local regimes, and the methods flow from the futility of trying to achieve their ends with either politics or a conventional military force.

Regime's Response To The Rise Of Islamic Fundamentalism

Different regimes responded to rising fundamentalist Islamic religious sentiment in different ways.

In Afghanistan, a 1973 coup to remove the monarchy was followed by a communist coup in 1978, followed by "imprisoned, tortured or murdered thousands of members of the traditional elite, the religious establishment, and the intelligentsia. The government launched a campaign of violent repression, killing some 10,000 to 27,000 people and imprisoning 14,000 to 20,000 more, mostly at Pul-e-Charkhi prison." The Communist regime led to a Soviet invasion in 1979, and Afghanistan anti-communist forces, some of the more effective of which were Islamic fundamentalist with Saudi Arabian support, were important players in a civil war that continued with only brief interruption until the 2001 arrival of U.S. forces in the wake of the 9-11 attacks. Terrorism was a tactic there against various other factions including the U.S. and its allies into the present.

In Lebanon, a civil war broke out on religious lines in 1975 and continued until 1990.

In Iran, protest movements in 1977 and 1978 gave rise to Iran's Islamic Revolution of 1979. This was followed in short order by the Iran-Iraq War from 1980 to 1988 in which new tactics including suicidal "human wave attacks" by people with explosives indoctrinated to be martyrs, widespread attacks on civilian targets, and the use of chemical weapons, set precedents about acceptable means that were then widely adopted elsewhere in smaller scale terrorist attacks as part of asymmetric warfare.

Iraq tried to prevent terrorism from breaking out not by "joining them" as the Iranian Revolution's regime did, but by imposing a totalitarian regime that muted ethnic and religious tensions somewhat at the cost of forcing people to chafe under its own regime's unpopular restrictions.

In Egypt, Syria, Iraq and Turkey, secularly inclined military forces and one party regimes clamped down on dissent to prevent political movements that they didn't control from emerging and suppressed religious political movements.

Syria became a one party regime in 1963 that was controlled by a comparatively liberal and pro-Western Alawite minority that had been favored during the French League of Nations mandate of Syria. It intervened in a Jordian-Palestinian Civil War in 1970 that gave rise to official Syrian support of the PLO which engaged in anti-Israeli terrorism when it withdrew in 1970.

Tactics used in one conflict spread to others, for example in the bombing of U.S. forces in Beirut in 1983.

Wealthy, often unemployed but religiously educated lesser members of the royal family in Saudi Arabia held spread ultra-conservative Salafi Islamic approaches to Sunni Islam throughout the Middle East as a way to counter corrupt secular communist regimes in the region thus expanding their own worldview as well: "Salafi groups ... began developing an interest in (armed) jihad during the mid-1990s."

Bottom Line

So, while the source of modern Islamic terrorism in the Middle East has somewhat deep political roots, it can really be traced to a religious reaction of the rise of Israel on the one hand, and secular communist Western leaning regimes on the other, starting around the late 1960s and early 1970s and escalating over time since then.

In Europe and the United States, a primarily Middle Eastern Islamic terrorism phenomena spread via immigration to countries that were destinations of immigrants by minorities who favored radical action in a larger movement largely directed a local, Western supported corrupt and oppressive tyrants in their homelands. This started to become more common in the early 2000s, following a brief low point in European based terrorist activity (see the chart below).

The rise in Islamic terrorism in the West (Europe and the U.S.) in the 2000s pretty much boils down to deliberate policy choices made by a couple of international jihadist organizations: the al-Qaedaterrorist organization and the thinly linked subsequent Islamic State organization.

Islamic terrorist tactics spread beyond the Middle East to places like Libya, East Africa, the Sahel zone of Africa, Pakistan, Thailand, Indonesia, and the Philippines, largely following the model of Middle Eastern jihadists.

Caveats And A Larger Global And Historical Context

Terrorism is not and never has been exclusively or primarily Islamic. It is really just one type of asymmetric warfare which has been more common historically than the wars between equally matched forces in international wars that dominate the history books.

But, starting around the late 1960s, asymmetric warfare became the truly dominant means of conflict internationally because technologies like machines guns, tanks and warplanes made conventional warfare between well equipped government forces and ill equipped insurgent forces futile. The total and rapid victory of the U.S. and its allies against Iraq in the Gulf War in Iraq from 1990 to 1991, cemented this widespread belief.

Also, the threat of mutual assured destruction in a nuclear war during the Cold War discouraged international conflicts in which a nuclear power such as Russia, the U.S. or China sided with a party to the conflict.

The antiquity of terrorism is illustrated, for example, by the term assassin, which derives from derives from what we would call today a Shia Islamic terrorist group ca. 1000 CE, with roots in Northern Persia fighting back against both Crusaders and Sunni Islamic oppressor regimes.

Terrorism was a central tool in the conflict between the English and the Irish in Ireland and later once Ireland gained independence, in Northern Ireland. "The Troubles" in Northern Ireland which were heavily marked by terrorist activity started around 1969.

Basque and Northern Italian separatists in Europe turned to terrorism. The first terrorist attack attributed to the Basque ETA was in 1968. The Red Brigades championing Northern Italian autonomy engaged in terrorist attacks starting in the 1970s.

Modern terrorist attacks have continued at a slow and steady rate by various factions of the day in Europe (mostly domestic autonomy movements) since the 1970s and fell off towards the end of the 1990s.

Image via Datagraver.

Terrorist tactics were widespread in Sri Lanka in connection with a 26 year civil war between Tamil forces and the government that continued from the early 1980s through 2009.