Earlier this week I posted on shoddy inference based on the Mitchell Report. Today's NYT has a column from Jonathan Cole and Stephen Stigler:

An examination of the data on the players featured in the Mitchell report suggests that in most cases the drugs had either little or a negative effect.

For pitchers identified by the report, we looked at the annual earned run average for their major league careers. For hitters we examined batting averages, home runs and slugging percentages. We then compared each player’s yearly performance before and after he is accused of having started using performance-enhancing drugs. After excluding those with insufficient information for a comparison, we were left with 48 batters and 23 pitchers.

For pitchers there was no net gain in performance and, indeed, some loss. Of the 23, seven showed improvement after they supposedly began taking drugs (lower E.R.A.’s), but 16 showed deterioration (higher E.R.A.’s). Over all, the E.R.A.’s rose by 0.5 earned runs per game. Roger Clemens is a case in point: a great pitcher before 1998, a great (if increasingly fragile) pitcher after he is supposed to have received treatment. But when we compared Clemens’s E.R.A. through 1997 with his E.R.A. from 1998 on, it was worse by 0.32 in the later period.

Hitters didn’t fare much better. For the 48 batters we studied, the average change in home runs per year “before” and “after” was a decrease of 0.246. The average batting average decreased by 0.004. The average slugging percentage increased by 0.019 — only a marginal difference. So while some batters increased their totals, an equal number had falloffs. Most showed no consistent improvement, several showed variable performance and some may have extended the years they played at a high level, although that is a difficult question to answer.

Some players improved and some declined. But the pattern for the individuals’ averages was consistent, and the variability of players (with the exception of home run counts) was low. There is no example of a mediocre player breaking away from the middle of the pack and achieving stardom with the aid of drugs.

They do offer a caveat:

It is possible (but not addressable by these data) that one effect of drugs is to help players compensate for decline as they age, and thus to extend their careers. But there is no evidence in these data for performance enhancement above previous levels.

HT: JC Bradbury