OneStat, Take Two
Through some spirited discussions with John from Washington Baseball Blog, I focused my attention away from the K/9*K/BB/(1+HR/9) formulation and towards a statistic based on expected runs saved vs. expected runs allowed based on a pitcher's pitcher-controlled stats (K/BB/HR).
In short, the formula is expressed:
Expected Runs Saved per K * K - Expected Runs Allowed per BB * BB - ExRA per HR*HR
That gets you the amount of saved runs to a team from a pitcher's defense-independent performance.
The only problem was to derive the expected run values of strikeouts, walks, and home runs. I started with the Baseball Prospectus 2004 expected run values by situation (I'd link it, but it is subscriber-only). Using that matrix, I created similar matrices for the expected runs added (or, in the case of strikeouts, subtracted) by the contribution of a marginal strikeout, walk, or homerun.
A strikeout situation is easy; you just take the current value of the situation and subtract out the value of the situation one out later. For example, if a team expected to score 0.8 runs with runner on first and none out, but 0.4 runs with runner on first and one out, I calculated the value of a K in that situation as 0.4 runs saved.
Walks are easy as well; you just take the difference between the current situation after a walk and the current situation without a walk. Thus, if a team expects to score .4 runs with a man on 1st and 1 out, but expects to score .8 runs with men on first and second and 1 out, the value of the walk in that situation is 0.4 expected runs. With the bases loaded, the value of a walk is 1 run.
Homers are a little counter-intuitive. With bases empty, the value of a home run is 1 run (obviously). With runners on, it's a little difference. For example, if there is a runner on 3rd and none out, the expected run value is 1.45 runs. If a batter homers, then the team gets 2 runs, but is left with a situation in which there are none on and none out - a situation with an expected run value of 0.54 runs. So the difference (1.45 - 0.54) must be subtracted out, leaving the value of that home run of 1.08 runs. The calculation is a little jarring at first, until you realize that your team is pretty much going to get that guy in anyway, so the real value you provide by hitting the homer is getting yourself around the bases.
Then I weighted the K, BB and HR matrices for the relative occurence of each cell in the real world. There were 188,539 plate appearances in MLB this year, and 103,387 (54.84%) came with no one on base. I got the plate appearances on a runner-situation basis from MLB.com, although it was not cross-referenced with out-situation. So I had to weight runner-situations by the relative occurence of out situations. Out situations are extremely evenly weighted - 34.5% came with none out, 33.2% with one out, and 32.32% came with two outs. If someone has the weightings of each cell in the 24-situation matrix, I could refine the data further.
The expected run values across MLB 2004 of a K, BB and HR are -.294422, +.327641, and +1.39299, respectively. I plugged these values into the formula, and created a runs saved per 9/IP figure by dividing by innings pitched and multiplying by nine. I calculated the values for all MLB pitchers in 2004 and plotted the runs saved per 9/IP against ERA. Here's the XY scatterplot I got for all pitchers with 20+ innings pitched in 2004:
It's an interesting calculation, and I'll have to noodle it further. For the time being, here are the top 10 pitchers (with 100 or more IP) in terms of runs saved per 9/IP through defense-independent pitching efforts:
| Pitcher | ERA | ERV/9 |
|---|---|---|
| Randy Johnson | 2.60 | 1.68 |
| Ben Sheets | 2.70 | 1.23 |
| Johan Santana | 2.61 | 1.06 |
| Jason Schmidt | 3.20 | 0.94 |
| Scott Shields | 3.33 | 0.91 |
| Jake Peavy | 2.27 | 0.84 |
| Roger Clemens | 2.98 | 0.73 |
| Curt Schilling | 3.26 | 0.65 |
| Roy Oswalt | 3.49 | 0.63 |
| A.J. Burnett | 3.68 | 0.62 |
posted by SuperNoVa @ 10:24 PM
8 comments
8 Comments:
TMK- if you send me an e-mail at natsblog@gmail.com, I'll send you every shred of data I've got.
And, you are right, it does break down for pitchers with few innings to some extent (just like ERA, K/9, etc all do). That's why my cut off was 20+ IP. You could probably group the fewer-than 20 IP pitchers by strata or something to include them in the data, but they do act weird.
I still can't accept that dERA is right if Odalis Perez has a .70 lower dERA than Kris Benson. It simply doesn't match reality.
And none of the values I used are park adjusted. But park adjustment is a really nasty thing, though, for several reasons (1) pitcher dependent stats don't always match park-factors- home run rates; (2) I have a hard time using year-by-year park factors - it's hard for me to believe that, for example, Anaheim Stadium shot up to a 107 batter park factor in 2001, despite the fact that it didn't change its dimensions at all and was back under 98 the very next year.
(1) I'm not hot on Benson. I'm hot on his wife, though.
(2) "All the stats that I've looked at" - which ones are those? Are they defense-dependent?
Because the stats I've looked at, HR, BB, K (and HBP), the stats that the pitcher has direct control over, say that Benson is better because he gives up fewer walks. I don't care what the defense does behind Perez.
I just want to hear one argument based on the defense-independent performances of Perez and Benson that Perez was beter. Citation to a statistic (dERA), for which you have no formula is not proof and cannot explain how it weights home runs versus walks, strikeouts and a team's defensive performance, is conclusory and circular. Perez isn't better because his dERA is better. Unless dERA is explained and it appears to take into account the relevant defense-independent performances better, I cannot accept dERA alone as an argument.
(3) Four-year park factors make more sense. I'd still want to weight the years, but that's OK.
John, you're right, I meant home run rate when I said walk rate. Benson has a huge advantage over Perez when it comes to home run rate - the 11 fewer home runs are well worth the 17 fewer walks(which I've said time and time again).
Percentage of line drives is an interesting stat (I don't have access to that data), but it strikes me that the % of line drives on BIP for the two men accounts for 10 more line drives off Benson. I'd trade 10 more line drives for 11 fewer homers.
I don't see how you calculate a 2 HR difference.
As for being weighted towards starters, the post clearly discusses that the total is a per-9 total. In fact, the best pitchers with 20+ innings are Brad Lidge, Gagne, etc., who have very low home run rates and very high K rates.
John, I think you think ERA+, dERA which are useful stats, proves much more than they do. I tell you what, I will STOP comparing Odalis Perez and Kris Benson for the purposes of this post.
Compare these two lines:
Odalis Perez:
196 1/3 IP 26 HR, 44 BB, 128 K, 3.25 ERA, 127 ERA+
Pitcher B:
185 1/3 IP 28 HR, 46 BB, 141 K, _____ ERA, ____ ERA +
Who would you think is a better pitcher? Looks like they are pretty much the same, don't they? Pitcher B gave up 2 more walks and 2 more homers in 9 fewer innings, but also struck out 15 more guys in those 9 fewer innings (a wash from an expected pitcher run saved standpoint).
Let me fill in the blanks for Pitcher B...4.52 ERA and 89 ERA+. Both of those number worse than Benson.
Pitcher B is Odalis Perez in 2003. Perez had a dERA of 4.81 in 2003 and 3.74 in 2004. Same team, same defense behind him, roughly same walk rate, homerun rate and strikeout rate.
Please explain why dERA and ERA+ explain Odalis Perez's defense-independent performances between the two years. They can't - he was the same pitcher. And if he gave up more line drives in 2003, wouldn't that be a start down the path that the amount of line drives given up by a pitcher is not a repeatable performance from year to year?
John, first you say that dERA is a much better way than any other statistic to take into account defense-independent pitching performance and now you are saying "you can't look at any one stat"? You've contradicted yourself so many times in this discussion I don't know where you come out any more. I try to limit the issue to one thing - defense-independent pitching performance (the Three True Outcomes) and you keep trying to raise extraneous things and backpedaled and jumped around. And at no time have I ever suggested that anything I have proposed is the sole and exclusive basis to judge a pitcher at all - one of my first posts on the subject used ERA+ to measure pitching performance.
Let me get this straight - I'm the one that doesn't get this sabermetric thing and you start talking about non-quantifiable things like "Perez started out brilliant and then fell apart" ... when the truth is that he went to Colorado with a 3.23 ERA in late May 2003 and gave up 9 runs in 3 innings - enough to jack up his ERA another run. He had another bad game against the White Sox in June, one against the Cardinals in July and got bombed for 10 runs in August vs. Houston. Four bad games.
In 2004 he had two blow up games - one in Colorado (again) and at our Expos in August. The difference between the two seasons lies almost exclusively in the two bad outings versus the four bad outings.
"He let more people on base" - well, it wasn't through the walk - it was through the base hits allowed. Again, not necessarily a defense-independent proposition.
For the benefit of our probably one or two remaining readers, I want to say that I think John's thoughts on this issue have been helpful, and were extremely valuable in deflating OneStat as a highly valuable measure (I still think it's a pretty OK one, though, taken with a grain of salt).
John, you are always welcome here. I do get frustrated when you imply that I'm stupid or sabermetrically challenged, because it's not helpful. One, if I am sabermetrically challenged, it's better to explain it to me than to call me dumb. Why not help everyone be sabermetrically learned? Two, if I am dumb, everyone else would see that and you don't need to say it.
I have tried to maintain a consistent position throughout this - trying to come up with a defense-independent measure for pitchers based on the outcomes that pitchers SOLELY control - HR, BB, K - not anything else. And I think ERV9 (what I am now calling OneStat, Take Two), is an excellent way of taking that into account. In fact, without naming names, one BP author e-mailed me indicating that he liked ERV9.
My argument as to dERA is this - the dERA formula is not published, as far as I know. However, to the extent that dERA is used as a stat to measure defense-independent pitching performance, it fails to do so, based on the Benson vs. Perez 2004, and Perez 2003 vs. Perez 2004 comparisons. I cannot and will not impugn the value of dERA (which, again, I am unaware of the exact formula for calculating), to the extent that it measures other things to calculate a pitcher's value.
I think we may have agreement here - there is a difference between purely defense-independent and defense-adjusted. I agree that dERA looks like it is defense-adjusted. But that doesn't mean it isn't luck adjusted. To the extent that dERA takes into account park effects and the team defensive performance (e.g., Dodgers are 2% better at turning batted balls into outs), yes it is a good measure. But it is not defense-independent, because the remaining stats incorporate defense-involved activities such as hits, etc.
So I think that dERA is valid as a defense-adjusted measure but not a defense-independent measure, which I don't even believe it purports to be. Make sense?
Post a Comment
<< Home