POPing the Productive Out Myth | Disciples of Uecker

Disciples of Uecker

We'd like to go to the Playoffs, that would be cool.

Most baseball fans would say that Kyle Lohse made three mistakes in his game against the Atlanta Braves last Tuesday. The first mistake was the 2-0 sinker he threw Jason Heyward in the top of the fifth. Heyward hammered it for a two run home run and the lead. Lohse’s second mistake was the 3-2 slider that Freddie Freeman roped over the right field fence in the top of the sixth. Freeman’s laser beam extended the Braves lead to 3-1. But Lohse’s third mistake was different than the other two. It occurred earlier in the game and wasn’t a pitch. The mistake was his inability to get down a bunt with runners on first and second with no outs in the bottom of the second.

In place of a “productive out”, Lohse struck out after bunting a two-strike pitch foul. Carlos Gomez followed by grounding into a double play to end the inning – a situation that would almost never have happened if Lohse had bunted successfully. Instead, the inning was over. The rally killed. All because Lohse failed to make his out “productive”.

Ten years ago, Buster Olney championed a new stat deemed “POP” or “productive out percentage.” Developed by ESPN and Elias Sports Bureau, and introduced by Olney. Outs were deemed “productive” if:

  1. A baserunner advances with the first out of an inning.
  2. A pitcher sacrifices with one out.
  3. A baserunner is driven home with the second out of an inning.

Olney’s piece landed a month after Moneyball: The Art of Winning an Unfair Game hit the shelves. The abstract to Olney’s article, Smallball vs. Moneyball, perfectly summarized the two schools of thought on outs:

Many say it’s a baseball sin to waste your allotted 27, but teams like the Tigers say they’re the key to success.

Olney aimed to push back on the idea that giving away outs was bad. “Productive outs” and POP were intended to be the measuring stick that proved otherwise. So how does that debate look ten years later? Let me answer that question by asking another: how often do you hear POP mentioned or referred to today?

The sabermetric response to POP was skeptical and swift. Olney’s article didn’t present the hard data, only the results. So statheads, like Larry Mahnken, ran the numbers. Mahnken disliked POP as a stat. For him, POP’s parameters didn’t make much sense and, more importantly, a high POP didn’t appear to correlate with winning. Instead of creating a stat around an idea, sabermetricians suggested looking at the raw data to see if the premise behind POP was even statistically possible.  Could an out ever be statistically beneficial to the offense?

Sabermetricians staked out the ground that giving away an out was never a good idea, while Olney, and gritty hitting coaches, claimed otherwise. Luckily, Tom Tango had already mined the hard data. Tango’s “Run Frequency Matrix”, compiled from figures between 1999-2002, calculated how often runs scored following each base/out situation until the end of the inning. For example, here are a team’s odds to score a run at the start of each inning, with no outs, and no one on base:

Base Outs 0R 1R 2R 3R 4R 5+R
Empty O 0.707 0.154 0.074 0.035 0.016 0.013

So 70.7% of the time, no runs are scored during an inning. One run is scored 15.4% of the time and so forth. Now, if that first batter draws a walk, considered a cardinal sin by pitching coaches, here’s how the odds change:

Base Outs 0R 1R 2R 3R 4R 5+R
1st O 0.563 0.176 0.132 0.067 0.034 0.028

A lead-off walk suddenly gives the offense a 14.4% better chance of scoring that inning. Not only does the chance of scoring one run increase by 2.2% but the odds of having a big inning also increase dramatically. For example, the offense’s odds of scoring two runs almost doubles – from 7.4% to 13.2%. Meaning that the old mentality of lead-off walks leading to big innings is statistically justified.

So what does Tango’s Run Frequency Matrix say about Lohse’s bunt situation? Would a successful bunt have been statistically beneficial for the Brewers? Would the out have been “productive”? Or, like the sabermatricians argue, would giving away the out statistically lower the Brewers’ odds of scoring? Let’s see what the Run Frequency Matrix has to say:

Base Outs 0R 1R 2R 3R 4R 5+R
1st_2nd O 0.359 0.219 0.165 0.127 0.07 0.059
2nd_3rd 1 0.305 0.285 0.218 0.101 0.053 0.038

Turns out both sides have a point. First, this proves that an out can be “productive”, or, in other words, statistically beneficial. If Lohse had successfully dropped down that bunt, the Brewers’ odds of scoring in the second inning of Tuesday’s game would have increased by 5.4%. The odds of scoring one run would have gone up 6.6% and the odds of scoring two runs 5.3%. But here’s where the sabermetrician’s side of the argument takes over. Though the bunt would have increased the overall odds of scoring, it also would have depressed the odds of the Brewers having a big inning. Giving away the out would have lowered the odds of scoring three runs 2.6%, four runs 1.7%, and five or more runs 2.1%.

So, Lohse failure to bunt definitely hurt the club, but how much? Here’s what the odds say:

Base Outs 0R 1R 2R 3R 4R 5+R
1st_2nd O 0.359 0.219 0.165 0.127 0.07 0.059
1st_2nd 1 0.574 0.161 0.11 0.088 0.038 0.028

Overall, the Brewers’ chances of scoring fell 21.5% because of the failed bunt. This precipitous drop plays into the sabermatrician’s hand. In their eyes, bunting cuts both ways. A good bunt can statistically increase a team’s chance of scoring a run or two while simultaneously decreasing the chance of a big inning. In addition, a failed bunt attempt can cripple on offense’s chance of scoring at all.

So the truth behind “productive outs” is more nuanced than old clubhouse coaches, POP, and sabermatricians would suggest. Case in point: bunting a runner from first to second with the first out of the inning is statistically not a “productive out.” It lowers a team’s overall odds of scoring by 3.1%. The chance to score multiple runs in the inning sinks across the board. Yet, moving that runner to second with one out does increase a team’s chance of scoring one run by 5.4% – the very definition of playing for one run.

In Olney’s original article on POP, the Angels were held in high regard as a team that successfully used “productive outs” to their advantage. Watching the Brewers’ style of play to start the season, it’s easy to see that Ron Roenicke came from the Angels’ smallball school of thought. Honestly, that makes me sweat a little bit, though not because I believe there’s no room for bunting, stealing, or “productive outs” in the Brewers’ game. There is a place and statistical justification for it, especially in the NL with the pitcher batting. Yet the Run Frequency Matrix shows that the odds behind smallball don’t always reinforce the clubhouse myths, and each “productive out” situation comes with a myriad of caveats and risks.

Roenicke’s offensive philosophy is exciting, but a bit too aggressive for my tastes. It is proven that “productive outs” have a place in the game but the odds aren’t as beneficial as most think. While Lohse could probably care less about his POP and Tango’s Run Frequency Matrix, as a veteran, I’m sure he would be the first to tell you that he hurt the club by not getting down the bunt. Sometimes the age-old perceptions and the statistical data match-up. And that one mistake lowered the Brewers’ chance of scoring by 26.9%. Not a statistical deathblow, but a small cut that, compounded with other minor mistakes, can drain the chances of winning right out of a game.

Share Our Posts

Share this post through social bookmarks.

  • Delicious
  • Digg
  • Newsvine
  • RSS
  • StumbleUpon
  • Technorati


Tell us what do you think.

  1. dbug says: April 7, 2014

    Trying to convince some people that sac bunts are usually a bad strategy is like trying to debate the existence of God.

    • Vin B says: April 7, 2014

      The problem also is that we don’t have enough sample size for bunts to know how many times it’d be successful. You can expect that starting pitcher probably get on base < 20% of the time when not bunting which makes it easier to pull the trigger on a bunt when there's a runner on base. But what about for position players?

      Using Tom Tango's win expectancy matrix, In the bottom of the 9th with runner on 3rd and 1 out has a higher probability of winning than runner on 2nd with 0 outs. This would be the only ideal case for a sac bunt* but that doesn't take into account the probability of the bunt to begin with.

      Squeeze plays at home are even harder to analyze; Tie game, bottom of the 8th runner on 3rd, 1 out. Do you squeeze? Your win expectancy rises from 74.3% to 84.8% which means the squeeze is worth if it works 87.6% of the time. Does the squeeze play work 87.6% of the time? Who the heck knows?? We don't have enough sample sizes to say for sure.

  2. Jason says: April 7, 2014

    I don’t disagree with the idea that people can become over-infatuated with productive outs, but productive outs are an important tool in winning baseball. You don’t get extra win points for blowing a teams out, so not all runs are created equal. As Ken Macha was fond of saying, “Usually when we win, we scored more in one inning than the other team did all game.” Station-to-station baseball is a viable strategy, but it doesn’t help you maximize your winning chances in a single game.

    Psychology and momentum play a big part in baseball. Manufacturing a run to break a tie game is more important to winning than an improved, but still statistically small, chance of having a big inning. . The flip side would be if you are down 3 runs or more. You’d rarely ever give up an out to merely advance a runner (early in the game with the pitcher up is the only time I can think of where the move makes sense).

    TLDR: Productive outs are important to the game and should be used more often, but sometimes are not the right choice in certain game situations.

    • dbug says: April 7, 2014

      I think you are partly right. Productive outs are in important part of of one strategy for some very specific situations, but it’s a strategy that is vastly overused. The only time you should be playing for one run is late in games. Mathematically speaking, it just doesn’t make much sense to sacrifice but unless you only need one run or the batter is very, very bad. I think I read some analysis somewhere that concluded that a batter is better off swinging away unless their odds of getting a hit are less than 1 in 10. Essentially, only a poor hitting pitcher should be asked to sac bunt.

      • Jason says: April 8, 2014

        Even good hitters make outs most of the time. An out is not a sin, but squandering opportunities-especially with the pitching staff they have this year-probably cost them at least one win against Atlanta, and will probably cost them more wins as the year goes on.

        Unless you can’t afford to give up two outs for a run, there is no reason a man on 2nd with no outs shouldn’t score. It is simply a matter of execution. That is something the Brewers need to get better at.

        • dbug says: April 8, 2014

          Yes, in that situation using a sac bunt to move the runner over is a wasted opportunity unless you are in the situation that calls for you to play for a single run. The odds of that runner on 2nd scoring is actually higher with no outs than it would be if he was on 3rd with one out. Plus, you are also reducing your odds of scoring multiple runs. There is a time and a place for that strategy, but it is not generally applicable.

        • Nicholas Zettel says: April 8, 2014

          I disagree that the Brewers need to get better at productive outs; if you rummage through Roenicke’s Brewers squads, they are typically average or better at most situational aspects of the game. Specifically, he has put together average or better baserunning approaches in 2011 and 2012, and his clubs have also been pretty good at outscoring opponents in manufactured runs.

          I don’t know why Brewers fans typically think their club is bad at productive outs. if anything, the club needs to work on pitch selection, patience, and driving the ball to score more runs, rather than playing productive situational ball.

          • dbug says: April 8, 2014

            Absolutely right. There is an infatuation with small ball and bunting that just doesn’t make sense. It may have made sense 100 years ago, but the game has changed enough that it is no longer the “right way” to play the game. It’s over-managing and it ends up costing teams runs.

          • Jason says: April 8, 2014

            OK. I’m willing to be convinced. I say the Brewers are not very good at advancing the runner to 3rd in the above situation. Two areas where I think they are particularly bad are striking out and hitting grounders to 3rd & SS. What is the league average in those two situations and how do the Brewers do?

  3. just fyi says: April 7, 2014

    what is often forgotten in the statistics is to account for who in at bunt when the play is called, who is the follow up bat, how capable is the batter at bunting and what is the game situation. any of those situations can change the call. when I used to coach third base I would often hear why did you not bunt the runner over after the fact. If my batter was a poor bunter but good stick, if it was early in the game, the follow up batter after the bunt was not productive or I had my 3-4 batters coming up I would more than likely ignore the bunt and hope for the big inning. A pitcher not hitting his weight makes a bunt much more attractive than a position player who is hitting well. those differences are not a part of the stats presented as they are a composite of every situation.

    • Okinawan Gorman Hai says: April 8, 2014

      So let’s see what the numbers say. Why not run the data for a sample consisting solely of instances when pitchers are at bat with runners on and compare scoring rates for ABs where the pitcher attempted to bunt versus scoring rates where the pitcher did not attempt to bunt?

  4. What he said says: April 7, 2014

    Run Frequency Matrix aggregates data over all comers. So, it doesn’t matter whether it’s Mike Trout or Kyle Lohse in the box with no outs and runners on 1st and 2nd – the matrix is the same. We know that’s not true.

    A more precise analysis would really need to take into account the chances of the batter successfully dropping a sac bunt versus the chances of the batter hacking away and not making an out.

  5. Jerry says: April 13, 2014

    The problem with this is you’re using a run expectancy table (which tells you what happened in every situation regardless of quality of hitter and pitcher) and applying it to a pitcher at the plate. This analysis would work fine if you were analyzing this situation with an average player, but most players who bunt are below average to terrible at the plate. While bunting is often overused in baseball, citing run expectancy tables as proof is lazy and often incorrect. If you want to figure out in it was the right situation for a bunt, you need to adjust for quality of hitter, quality of pitcher, game situation, run environment, and whether the hitter was actually trying to bunt for a hit. Bunting and stealing bases didn’t make sense in the crazy late-90′s run environment, but today’s RE is severely depressed and aggressiveness on the bases increases in value as a result. Bunts can be annoying to watch and often feel like giving up, but do your research before you call it bad decision.

    It’s also bizarre you would chose Lohse and his career -15 wRC+ as evidence that you shouldn’t bunt. I don’t know how you can think a pitcher swinging away with less than two outs is a good plan.


Websites mentioned my entry.

  1. Daybreak Doppler: New Bucks Owners On The Way? | PocketDoppler.com

Add a Comment

Fill in the form and submit.