The 2012 Brewers bullpen had the worst earned-run average (ERA) in baseball: 4.66.
The 2013 Brewers bullpen had the fifth-best ERA in baseball: 3.19.
Because this dramatic improvement in ERA coincided with an overhaul of the bullpen, it’s tempting to chalk it up to great personnel moves by Doug Melvin, the Brewers General Manager. More than a few writers already have.
Unfortunately, when you look closely, it becomes clear that this improvement was driven primarily by luck, not better pitching.
The first clue that something is amiss comes when you compare the amount of Wins Above Replacement (WAR) generated by relievers from the 2012 to the 2013 club. As measured by Fangraphs, they compare as follows:
|Season||Innings Pitched||Wins above Replacement (WAR)|
That’s right: despite pitching 11 more innings, and doing so with an ERA over 1.5 runs lower, Fangraphs’s WAR formula says that the 2013 bullpen pitched only half as well as the 2012 bullpen. How could that be?
The answer comes when you look at some underlying statistics for those two bullpens:
The 2012 and 2013 bullpens differed significantly on their Batting Average on Balls in Play (BABIP) and their Left on Base Percentage (LOB%). These two statistics are extremely important, and they explain the differences in ERA we already highlighted. We talked about BABIP last week. BABIP is the extent to which balls put in play end up in outs, and a defense wants that number to be as low as possible. Great defenders can help bridge the gap, but BABIP is one of the most inconsistent pitcher characteristics across seasons; sometimes the ball heads into the defender’s glove, and sometimes it does not. In a previous article, I explained why I suspect that Kyle Lohse does have the ability to limit his BABIP. But unfortunately, the Brewers bullpen does not consist of Kyle Lohse clones. For them, variations in BABIP across seasons are going to be largely driven by luck.
LOB%, also known as a pitcher’s “strand rate,” is similarly volatile and difficult to repeat across seasons. LOB% is essentially the flip side of BABIP; while BABIP measures the number of balls in play that happen to result in hits, LOB% measures the percentage of runners who were never able to come around and score. Most hits in baseball are not home runs, and scoring on those remaining hits requires stringing them together when runners happen to be on base. While it would be nice to believe that some pitchers have the ability to suddenly “bear down” with runners on base, seasonal statistics do not bear this out. Rather, hits are hits, walks are walks, and sooner or later they will happen sufficiently close together to cause runs to score.
All baseball statistics have a tendency to regress to the mean — that is, to move back to the league average — over the course of any player’s career, and that is especially true of luck-driven statistics like BABIP and LOB%. The Brewers, however, have taken this to the opposite extreme. In 2012, they had the worst BABIP and LOB% luck in baseball; in 2013, the Brewers had some of the best:
|Season||BABIP Rank||LOB% Rank||Overall Luck (FDP)|
|2012||Tied for Last||Second to Last||Last|
As you can see, the 2012 Brewers bullpen had the worst defensive luck in the league, as measured by the Fielding-Dependent Pitching (FDP) statistic, which combines the effect of BABIP and LOB%. For the statistically inclined, the 2012 bullpen’s BABIP was more than 2 standard deviations over the league average, and their LOB was not far behind. Certainly, this should jive with your memories of that season, particularly that mid-summer stretch in which the bullpen melted down late, game after game. Walks were happening consecutively, hits came in bunches, and runners kept scoring. When those combinations keep happening in the clutch, it is easy to forget the seemingly meaningless innings in which those same pitchers allowed nobody to score.
The 2013 Brewers bullpen, on the other hand, was continually blessed by good fortune. The 2013 relievers were the seventh-most-fortunate in avoiding hits on balls that landed in play, and eighth-best at spacing out hits and walks to strand the runners they did allow on base. Overall, the Brewers relievers were the fifth-luckiest team in the league, and it showed in their results.
Now comes the part when we show you the effect of good luck:
Again, we’ve provided the ERA for each bullpen year. But now we’ve also provided the FIP (Fielding Independent Pitching) statistic for each bullpen. FIP strips out the effects of BABIP and strand rate, giving you a more reliable indication of each bullpen’s actual skill. The “E-F” column shows you the difference between each bullpen’s ERA and FIP figures, thereby demonstrating the effect that luck had each year. What you see is that the 2012 Brewers bullpen was really a 3.93 runs-allowed team; but with .72 extra runs of bad luck, they end up with a terrible-looking 4.66. The 2013 bullpen was a more of a 3.80 runs-allowed team; but with .61 runs shaved off by good luck, they end up with a sparking 3.19 ERA. It’s good to be lucky.
So, how did the 2012 and 2013 teams actually compare in terms of skill? You could rely on the FIP column above to say they were basically the same. But, baseball statisticians have come up with two measurements that do an even better job of measuring pitcher performance: xFIP and SIERA. Let’s see how the two bullpens measure up with those metrics:
xFIP goes even further than FIP, stripping out the luck factor not only from balls that land in play, but from home runs as well. Under xFIP, the 2012 bullpen is only .05 wins worse than its rebuilt successor. SIERA, on the other hand, focuses on rewarding pitchers for striking out more batters, among other things. According to SIERA, the 2012 team pitched slightly better than the 2013 team. How about that? In other words, the two bullpens reached wildly different results while pitching with about the same level of skill.
The bottom line is that while the 2013 Brewers bullpen may have been “retooled,” these new tools so far have not proven any sharper than those they replaced. I don’t doubt that Doug Melvin and his staff believe the 2013 relievers were more skilled than their predecessors, but these successors need to prove it in a season where the breaks are not falling their way. In the meantime, if someone tries to tell you about that terrible 2012 bullpen, you can tell them about how disastrous a little bad luck can be.
Follow Jonathan on Twitter @bachlaw.