Tuesday, January 04, 2005

More on ERV Scoring

I've scored a few Expos games from last year with the ERV scoring method I talked about in this post, and it's been a lot of fun. To make it more clear how it works, here's an sample inning from this game between the Expos and Reds, the top of the fourth, Expos batting. Here's the ERV Chart I'm using:

Outs ---1-- -2-12---3 1-3 -23123
059 12 1515 19 21 23
1 3 5710 101215 16
2 1235 4 5 68

Note that I have multiplied the Estimated Run Values by 10 and rounded to the whole number, which makes it easier to score this on paper (for me, anyway). So I record "tenths"of a run -- 1 run equals 10 RVs, a half a run equals 5 RVs. With bases loaded and no outs, the average team will score 2.3 runs, or 23 RVs.

So here's what happened in the Expos 4th inning:

Vidro leads off with a double. He gets 7 RVs. Before, 5 RV (O on, O out); After, 12 RV (2nd, 0 out) an increase of 7.

Batista flies out to right, pretty deep, but Willie Moe Pena makes a strong throw to third to hold Vidro at second. This is a good example of how to score fielding with this system. In my judgment, the ball was deep enough to get the runner to third, but for Pena's throw. So I give Batista the RVs he would have gotten if that happened, -2. (Before: 12 ERV, After (1 out, Runner on 3d): 10 ERV, difference of -2). But I give Pena the difference between what should have happened and what did happen, or -3 RV (Runner on 2d, 1 out: 7 RV, instead of runner on 3d, 1 out: 10 RV). So I record the -2 in Batista's box and a -3 in the margin for Pena.

Johnson hits a grounder to second that Jimenez muffs, Vidro to Third. The official scorer ruled this a hit, but I thought the average fielder would have made the play, so I give Jimenez an error. Another good example here. I give Johnson the RVs as if the play had been made, or -3 (Before, 2nd and 1 out: 7 ERV minus After: 3d and 2 out: 4 ERV). Then I give Jimenez the RV that represent the error, the difference between what happened (1st and 3d, 1 out: 12 ERV) and what should have happened (3d, 2 out: 4 ERV) or a +8 RV for the error. (When I compute Jimenez's total RV for the game, this turns into a -8 when combined with his offensive RV -- he had a bad game, since his Batting RV was -11 and his Fielding RV was -8, for a total of -19. He cost the Reds almost 2 runs, and they lost 4-2).

Cabrera hits a sac fly that scores Vidro, Johnson stays at first. Before: 12 ERV (1&3, 1 out); After: 2 ERV (1st, 2 out), plus 10 for the run scored equals a RV of 0 for Cabrera.

Sledge Walks. Before: 2 ERV (1st, 2 out); After: 5 ERV (1st & 2d, 2 outs), so Sledge gets 3 RV.

Schneider flies out to end the inning. Before: 5 ERV; After: 0 ERV (Expos can't score anymore), so Schneider gets a -5 RV.

So here's how my scoresheet looks:

Vidro: 7 RV, 2B, run scored
Batista: -2 RV, F8, Footnote A: Pena gets -3 RV for good throw.
Johnson: -2 RV, E4, Footnote B: Jimenez gets a +8 RV for Error.
Cabrera: 0 RV, Sac F8
Sledge: 2 RV, BB
Schneider: -5 RV, F7

Total Offensive RV: 0
Total Defensive RV: +5
Total RV: +5

Why is the total RV 5 (1/2 a run) if they actually scored 1 run? Because we are recording marginal runs above the average. Recall that the ERV for 0 on, 0 out -- the start of the inning -- is 5 or 0.5 runs. That means the average team scores approximately 0.5 runs each inning, or 4.5 per game. So when they actually score 1 run, they are scoring 0.5 above the average, or 5 RVs. Note that the ERV scoring shows that the run scored here was half due to the Expos batters and half due to the Reds fielders. Also note that Jimenez's error is the largest RV -- scoring this way shows how errors really kill, because the turn outs into bases and/or runs.

It seems that each inning should result in a Total RV that is a multiple of 5. A 1-2-3 inning is a -5 (-2, -2, -1). One run scored should be +5, two runs +15, three runs +25, etc. I didn't expect things to work out like that, i.e. it be a zero-sum thing, but I guess it makes sense, but I need to think more about that.

As for the pitcher, I usually assign them the negative of the Offensive RV number, here it was 0, so they don't suffer the errors of their fielders. This is usually a multiple of 5, except where there are errors. In this game, in the Ninth, Rocky Biddle got an 11 RV (runs saved in his case), because there was one error in the inning, yet the Reds did not score (the error gave the Reds 6 RVs). He essentially gets credit for getting four outs in one inning.

I pulled a Word document from this great site and modified it for ERV scoring, but I'm still working on it. If anyone is interested, use the e-mail link to the right and I can send it to you, especially if you are good at form design and can improve it.

At 10:03 AM,  dexys_midnight said...

This is really fun stuff DM!

The only worry I have is how subjective you were with just that one half an inning. Worries me that the system could devolve into something so subjective that it doesn't help us. Let's just say that the guy who was the official scorer was at the game with an ERV scoresheet like you. He would have given Johnson the hit and his total for Johnson for the game would have been drastically different than yours. Can you imagine if this became a widely used barometer in the public and someone lost the "ERV Title" because the ERV scorer scored the play differently than the official scorer?? Talk about your press frenzies! I can see wanting to account for the good throw, but countering the official scorer could be dangerous--also, when someone makes a really good fielding play in general, I imagine you have been giving them fielding ERV, right? Also subjective but in line with what you have been doing.

Anyway, it is still cool, and funny how different people think but come to the same conclusions...my first instinct before even reading your post was to check and MAKE SURE that your total was a zero-sum game, which of course it was as you pointed out for the exactly correct reasons.

At 10:37 AM,  DM said...

Well, I don't think you can judge fielding without being a bit subjective, just as we judge errors now. My standard is whether the play would have been made by the average fielder, and for good plays I have a pretty high threshold. The Pena throw in this inning drew oohs and ahhs from the crowd, so it met the standard. In the three games I've scored, it was the only good play I recognized.

You could score this way only on the official scorer for errors and not do good fielding plays, to be more consistent. But that doesn't strike me as much fun.

At 11:20 AM,  SuperNoVa said...

I'm having a hard time with the fact that, offensively, the ERV scoring mechanism only credited the 'pos with about .5 runs. While I realize that the ERV matrix is based on average run scoring, I'm still a little uncomfortable with not giving the batters credit for a whole run (if they deserved it).

I'd be more inclined to say that of the .5 increase in expected runs, Vidro accounted for 70% etc., and then applied that to the actual runs scored. So Vidro would get .7 runs credited, rather than .35... But there's no question you are doing it accurately the way you're doing it.

I think that this DEFINITELY is the way that statistics could take into account defensive performance. I'm pretty sure that zR takes into account the various "zones" in the field to which balls are hit. For example, if you could say that, of 120 balls hit to zone 95 with a runner on second, 80% of the runners on second went to third, you might say that the fielder saved (ERV runner on 3rd, 1 out) - (ERV runner on 2nd, 1 out) runs * 80%.

It would take a MONUMENTAL amount of numbers crunching for someone to figure that one out, though. You'd literally have to figure out, for each of the 24 offensive situations, what happened when a ball was hit to each of the zones.

At 1:47 PM,  dexys_midnight said...

Since the point is really just to measure one player vs. another SNV, I don't have any problem with it being .5 rather than 1. I think any tinkering with it to make it percentages would just make it harder in an unecessary way. More importantly perhaps, it would make it harder to deal with negatives. A zero run inning should result in a -.5 ERV total. Starting from zero would make that hard.

I really like what you have suggested on zone ratings, although again, it might make scoring during a game seem like far too much work as opposed to simple ERV, which wouldn't be any harder than just regularly scoring a game.

As I am uncomfortable with the subjectiveness of fielding calls in the ERV analysis, I suggested to DM (which I understand why he would reject it as less fun (and you could even make an argument, less accurate)), that if you want to do fielding, you could have two tables. One that was just pure offensive ERV and used the official scorer's ruling, and a separate one for defensive judgments. I think the defensive calls over all would even out for the offensive scores over a season anyway. In fact, I would just have a column for defensive ERV in each player's boxscore, so that you could add up his defensive contribution/loss for the game and at the end of the season have each player's total offensive ERV and defensive ERV.

And DM, I am totally up for scoring the season this way with you. As I mentioned to you, I look forward to the day when people around the stadium are talking to each other and saying the beautiful words: "buck says this guy adds 5 ERV or more this at bat."

At 11:33 AM,  John said...

This is certainly the easiest thing to do manually, but this bugs me:

"Cabrera hits a sac fly that scores Vidro, Johnson stays at first. Before: 12 ERV (1&3, 1 out); After: 2 ERV (1st, 2 out), plus 10 for the run scored equals a RV of 0 for Cabrera."

Cabrera deserves some credit. That's why I'm going to be happier getting PGP data emailed to my phone (if I get around to finishing), so that I can see how Carbrera has impacted the team's probability of winning! If I do get around to it, I'll probably automatically post team hero and goat on a daily basis after the online scoresheet becomes official.

At 12:51 PM,  SuperNoVa said...

If DM carried the decimal out another point...you'd see that the expected run value of the sac fly was actually a positive .02 runs (.0224 if you want go out a little further).

I do think that John has a point inasmuch as there may be a more significant difference in the probability of winning the game based on Cabrera's sac fly...according to BP's expected win matrix, the sac fly took the Expos from a .556 win probability (down two runs, 1st and 3rd with 1 out in top of the 4th) to a .367 win probability (down one run, man on 1st with two outs in the top of the 4th). So, according to BP, Cabrera's sac fly actually made the Expos expected win value decline from .556 to .367. Ouch.

As you can tell, John, there are problems with the expected win values of a particular act in a game...namely that the sample sizes are getting smaller and can be affected by a few games where someone hit a 3 run homer in that situation, etc. Using BP's matrix, a team would be better off two down in the top of the 4th with men on first and third (.556 expected winning %) than being TIED in the top of the 4th with men on first and third and NOBODY out (.538 expected winning %). And we know that just ain't true...it's a problem of small sample size for the year 2004. Give me an expected win matrix for the last 40 years, and then it may be more useful.

At 1:47 PM,  dexys_midnight said...

Again, I have no problem with DM's ERV scoring, and really think it is better for a fan at the game than the projected winning % idea, and perhaps even in general.

SNV, I think that Cabrera's net effect based on ERV sounds EXACTLY right to me. We already accept as gospel that bunting a runner over is almost always a loss. Well, you are doing the same thing here, but of course getting a run. So I think it evens out to a zero. Remembering that if Cabrera never existed and you replaced him with an average guy, the runner on 3rd with one out would have likely scored anyway and Cabrera is lowering the chance that the runner on 1st would score.

The reason I don't like using the winning % matrix is twofold: 1) you can memorize ERV's much easier to use them at a game and make quick calculations; 2) if you believe that as a general rule, clutch hitting does not exist, it should give you no better a sense of a player's abilities than ERV because those solo homers down by 8 in the 9th and the grand slams when down by 3 should even out over the season. I guess ERV just takes enough account of situational effects for my tastes.

At 1:57 PM,  John said...

Hmm, thinking about it more, I can buy Cabrera being credited with a 0, plus or minus something insignificant.

I plan on using at least a couple of years worth of data to determine values. It's not perfect, but sample size isn't the biggest problem (though I haven't looked at the confidence interval yet).

ERV is definitely a lot easier if you're going to score it yourself... but why do math when you can have stuff mailed to your phone, is my thought :-)

At 2:01 PM,  SuperNoVa said...

You know what bugs me a bit about ERV? That a sac fly with men on 1st and 3rd and zero outs is actually worth -0.4 runs. That's a pretty harsh penalty for bringing a guy home with a sac fly. Given the possibility of a double play in that situation, I have no problem with the guy putting some lift under the ball and getting the bird in hand. At the end of the day, you do have to get that guy in from third. As Mr. Miagi might say, you can't expect success, you must do success.

At 2:12 PM,  dexys_midnight said...

See, that sounds about right to me too (well, maybe -.2 or .3). The absolute standard here is that ZERO is average (possibly the biggest thing to get used to). So, if I had a guy up with 1st and 3rd, no outs, I would want him to forward our inning as opposed to just get the run in. My instinct might be -.2 for no outs and +.1 or +.2 with one out instead of -.4 and zero, but I'm not shocked at the numbers.

At 2:59 PM,  John said...

I think the sac fly is clearly more or less valuable depending on circumstances. If it's a game-winning action, it makes your probability of winning 1, and if you're down by 4 with one inning left, you don't want to waste the out.

At 4:17 PM,  DM said...

SNV, it depends on what your definition of "success" is. Your post reminds me of different views on investing. You seem to take the conservative approach, and would consider one run for one out a "success" or fair price. The ERV data indicates that you may be too conservative, in that your paying too high a price for that certainty. I don't know which is true, but I like scoring this way because it forces you to think about whether your judgment is right in that circumstance. As Dexys points out, many managers still believe it's a good deal to bunt a guy to second. In investing terms, they are stuffing cash under their mattress and ignoring inflation.

At 5:39 PM,  SuperNoVa said...

Oddly enough, you've described my investing strategy pretty well....Dexy's knows that I like cash in hand (dividends) more than anything. Of course, had I held onto FRO like Dexy's did, I would have had a much better year in 2004 than I did (but it was pretty good in its own right).

There's another point here that can't be lost. Not all runs are of equal value. If the average runs you allow in a game is 4.5, the first 5 runs you score are more valuable than runs 6-8, which are more valuable than runs 9-11 in their own right.

One run down in the 8th inning, I'm going to take the sure run every time over a 55% the potential for 2 runs, even though I'm losing 1/10th of a run in the exercise.

At 9:39 PM,  DM said...

SNV, you're definitely right that runs have different values, which is why (I think) John is exploring PGP and a measure of that value, based on how the run relates to the ultimate outcome of the game.

It seems to me, though, that the situations where the runs have different values may be limited to late inning situations, where outs become more valuable. It seems you could ERV score a game, then review it for those events that would fit into this "more valuable" category (or less valuable, for the blowout games), and perhaps make adjustments if need be. So that the sac fly in the top of the Ninth to put a team ahead gets some added benefit. I bet the Expected Win Percentage data could help us derive these values. Maybe instead of adding in 10 RV for a run, you add more when the inning is late and the game is close. Or have an separate ERV chart for the late innings of a close game, reflecting the added value of the runs at that point.

At 1:56 PM,  John said...

DM,

I agree with what you say.

The expected win percentage table (which Baseball Prospectus also publishes) is essentially what you would use for PGP scoring. It's just not going to fit on a scorecard :-)