There are exceptions. Death penalty convictions are overturned much more frequently, for example.
One important reason that federal habeas corpus petitions are of so little use to state prisoners, is that relatively few state prisoners can meet the federal law requirement that they exhaust remedies under state law before being entitled to have their claims considered by federal courts. The vast majority of federal habeas corpus petitions are dismissed for procedural flaws or for failing to state a claim for relief, even if the facts alleged were true. It takes about five years, on average, to exhaust all state court remedies, according to data in the Essay cited below (which is familiar and consistent with other studies). Yet (citations omitted and some notes brought into the body text from footnotes):
For the latest year with available data (2004), 60% of all defendants who were convicted of felony crimes in state court did not receive any prison sentence at all, and the average prison sentence for the remaining 40% was less than five years. According to 2002 data, a prisoner serving an average sentence is released in less than three years. . . almost 30% of all noncapital habeas petitions were filed by inmates serving life sentences, even though only 1% of all prison sentences are for life (life sentences make up 1.1% of prison sentences but 24.1% of sentences for murder or nonnegligent manslaughter). On the other hand, only 12% of all noncapital habeas petitions were filed by those serving sentences of five years or less, even though that group represents the majority of all those who are sent to prison. Because most federal habeas cases will not be resolved until years after the original conviction and sentencing, only inmates who receive life or other very long prison sentences will be in custody long enough even to file.
Also, more than 90% of convictions are the result of plea bargins, something that greatly narrows the grounds for appeal.
This isn't neessarily a bad thing, however. Indeed, there are two positive aspects of the exhaustion requirement and limited remedies available in plea bargined cases.
First, it limits federal intervention in state criminal justice largely to cases where the stakes for the incarcerated individual are high. This subcategory of cases is important, about 20% of people incarcerated in California, for example, are serving life sentences.
Second, we care more about state criminal justice decisions that were not consented to by the defendant in open court.
Third, in a more subtle point, a broader proportion of cases in states that skimp on post-conviction review processes are subject to review by federal courts. In a state with only direct appeals, federal habeas corpus review would be available after just a couple of years. Habeas corpus was strengthened, historically, to discourage defiance by state courts of federal rulings and it has had that effect to a great extent, by causing legislatures to implement extensive post-conviction review procedures. While the Essay cited below argues strenuously that habeas corpus review does not offer a meaningful incentive for state courts to behave, I don't fully agree. On a case by case basis this might be true, but some rights are reviewable only on collateral attack of a conviction (e.g. ineffective assistance of counsel) and that review was previously unavailable for all practical purposes in state court, and state appellate courts are informed by federal decisions in direct review of convictions and sentences.
Another important reason that federal habeas corpus petitions are of so little use to state prisoners is that there is no right to counsel for a federal habeas petition. Complying with the highly technical law of federal habeas corpus without a lawyer is very difficult, and few state prisoners not on death row can secure the representation that they need to make out their claims properly. The is not a trivial issue. Indigent defendants are bureaucratically inept. Seven out of ten “fall in the lowest two out of five levels of literacy—marked by an inability to do such basic tasks as write a brief letter to explain an error on a credit card bill, use a bus schedule, or state in writing an argument made in a lengthy newspaper article." It is little wonder that they can't do the work of lawyers well.
The authors argue that the cost of providing lawyers in 18,000 federal habeas cases a year, many meritless by any measure, would be too great to be politically acceptable. This to is an issue I doubt. We provide public defenders of some sort in millions of cases per year. In cases involving plea bargains, the issues to analyze are narrow. It costs money, but probably not as much as taking the case to trial in the first place, in many cases, and legal representation might also dramatically reduce the cost of processing the claims in the courts. Handling cases with pro se parties is often much more difficult for the system than handling cases with lawyers involved. According to the Essay, these cases are mostly decided based upon paperwork:
[T]hese cases may appear to be less complex or demanding than other civil cases that states may litigate in federal court. Discovery and evidentiary hearings, for example, are rarely granted. . . . (0.3% of noncapital cases included deposition or examination; 0.4% included evidentiary hearing).
The cost of providing counsel could be lowered even more, if professionals with competence limited to criminal law or even merely to habeas corpus law (similar to tax preparers), were recognized, on the theory that they could jusifiably be paid less per hour than other legal professions.
Indeed, one solution may be to make the right, like the filing of all other federal lawsuits, contingent upon paying a filing fee, which in practice, would require some sort of third party (like an ACLU or a relative) to put some faith in the case, making a filing less than cost free. This very simple adjustment in the process would prevent habeas corpus cases from being a financial drain on the judiciary (indeed, it might become a cash cow, because habeas cases are handled very quickly by the magistrates and law clerks for the most part), while at the same time requiring somebody on the outside to vouch for the merit of a claim, providing a non-govermental screening process.
Two academics have written an essay which proposes a dramatic narrowing of the habeas corpus right on the largely practical grounds that allowing relief in classes of cases where the probability of success is very low is a waste of money better spent elsewhere (paragraph breaks added for ease of reading in blog form):
This Essay argues that federal habeas review of state criminal cases squanders resources that the federal government should be using to help states reform their systems of defense representation.
A 2007 empirical study reveals that federal habeas review is inaccessible to most state prisoners who have been convicted of noncapital crimes and offers no realistic hope of relief for those who do reach federal court. As a means of correcting or deterring constitutional error in noncapital cases, habeas is failing and cannot be fixed.
Drawing upon these findings as well as the Supreme Court’s most recent decision applying the Suspension Clause, the authors propose that Congress eliminate federal habeas review of state criminal judgments except for certain claims of actual innocence, claims based on retroactively applicable new rules, or death sentences.
The federal government should leave the review of all other state criminal judgments to the state courts and invest, instead, in a new federal initiative to encourage improved state defense services. This approach can deter and correct constitutional error more effectively than any amount of habeas litigation ever could.
I'm don't entirely agree, although they do have a point. Certainly, the federal government has the power to increase funding of state defense services, and probably to impose minimum standards on the provision of these services under the enforcement clauses of the 14th Amendment and other constitutional amendments, and it might do some good if they did. It isn't obvious that this proposal is any stronger if it is linked to federal habeas corpus review -- it isn't even clear that the politics of the proposal change much. And, strategically, a temporary budget change is a poor reason to change a permanent law governing judicial review of state court judgments.
The authors note that historically, there was little role for federal review of state convictions. Federal statutes rarely directly regulate the state court criminal justice system, and there are serious questions, even in modern federalism which favors federal power generally, over whether such statutes, if enacted, would be constitutional in many cases. So, the grounds for federal court jurisdiction in most state criminal cases is the United States Constitution. But, prior to the 1960s, few rights guaranteed under the federal constitution were held to be applicable to state courts. The "incorporation doctrine" which holds that the 14th Amendment protection for due process necessarily protects the rights identified in much of the Bill of Rights in criminal cases, constitutionalized criminal procedure, creating many more issues that could be raised in federal court.
In the 1950s, as a result, few federal habeas corpus petitions were filed and few prevailed. Some of the key facts are laid out in footnote 19 of the Essay (citations omitted, paragraph breaks added):
In 1958, only 755 applications for habeas relief were filed in federal district courts; in 1962, only 1232. In 1960, the state prison population stood at 189,924. Assuming that about twelve hundred habeas petitions were filed in 1960 (as there were in 1962), that is a rate of roughly one petition for every 166 prisoners, even though federal habeas was the only meaningful postconviction remedy for most prisoners because most states had not yet adopted their own effective postconviction procedures.
By 2004, this ratio had more than doubled, to about one federal petition for every sixty-seven state prisoners . . . ( . . . 18,552 petitions were filed in federal district court in 2004[; there was a] state prison population of 1,244,311 in 2004).
Rates of habeas relief were also low during the late 1950s. [In 1956] on average federal courts order release of only one habeas petitioner per year [and] between 1949 and 1958 there were 6239 applications for habeas, but only 98 petitioners were successful in district courts from 1946 to 1957); [O]nly five of 668 habeas applications—less than 1%—were granted in 1955.
The grant rate for habeas petitions remains very low:
The grant rate for noncapital cases has dropped from 1% in the early 1990s to only 0.34% today [likely due to changes in federal law]. Only eight of the 2384 noncapital habeas filings the study examined resulted in a grant of habeas relief, and one of those eight grants was later reversed on appeal. At this rate, we estimate that fewer than sixty-five of the more than 18,000 petitions filed each year by noncapital petitioners will eventually be granted by district courts.
Other studies agree finding (citations omitted) that:
(grant rate of 1% for more than 2000 habeas cases terminated in 1992); (grant rate of 3.2% for 1899 cases from six districts filed from 1975 to 1977); (grant rate of 3–4% for 585 cases filed from 1973–1975 and 1979–1981 in Southern District of New York); (analyzing more than 1600 cases terminated in 1990 and 1992 from eight districts in four states and finding grant rate of less than 1% for most claims); (grant rate of less than 4% in District of Massachusetts between 1970 and 1972); (average petition grant rate of 3.5% in fiscal years 1963 (forty-two of 1662 granted), 1964 (125 of 3220) and 1965 (154 of 4186)).
One important reason that grant rates are so low is that it is hard to prevail on an ineffective assistance of counsel claim, yet 90% of cases plea bargain, often making that the only possible argument in those cases. In 2004, "95% of state felony convictions followed guilty plea." Numerically, this is an important issue:
A claim of ineffective assistance of counsel in trial or appellate proceedings was raised in about half of the 2384 noncapital cases the Vanderbilt-NCSC study assessed. Only one of those claims was granted; that grant was later reversed.
While defense counsel are good, the notion that counsel was not constitutionally ineffective in any of the roughly 50% of 18,000 non-capital federal habeas corpus petitions where it was raised each year (an aggrieved subset of all persons sentenced to long felony terms each year), seems implausible.
Notably, in "2007, there were 232 capital petitions and 18,907 noncapital; in 2006, 236 and 18,959; in 2005, 229 and 18,961; in 2004, 214 and 18,431." The U.S. Supreme Court averages six capital cases per term. Capital cases are much more likely to prevail in habeas corpus review: "During the years prior to AEDPA, federal courts invalidated capital judgments in two of every five cases. After AEDPA, that rate appears to have declined, but remains much higher than in noncapital cases." The most recent study showed a grant rate of 12–13% among 267 terminated capital cases, with 95 cases still pending.