Wednesday, April 13, 2005

Win Probability Scoring

As an alternative to ERV scoring, the Cheat's blog just did a White Sox game based on Win Probability Scoring - how much a particular player adds to a team's win probability by his performance. Here's the link. Although it's intriguing as a concept - and has been suggested by others - there are some data problems with WPA at this point. First, the data set for win probabilities may be a little off at this point. The data, as I understand it, is from 1979-1990. And, as it turns out, baseball has changed a little bit since 1990.

In addition, the WPA statistic is not terribly predictive in nature - a batter gets more WPA points if he hits a home run in the 9th inning of a 1-0 game than he does if he hits it in the 5th inning. In addition, I wonder how well it tracks the performance of a pitcher who pitches well but loses. It's interesting, and I think you'd be interested in reading the Cheat's scoring as well as the Hardball Times article linked.


At 10:37 AM, Blogger DM said...

Excellent post. I've been tinkering with a way to adjust the RV values we come up with for ERV scoring to reflect the impact on winning the game. I may post those figure up with the Friday Morning Figures someday soon.

At 10:49 AM, Blogger DM said...

Turns out under Win Prob Guzman still hurt us. We went from a 26 % chance of winning to a 14 % chance of winning with his sac fly.

Calculated using the
Win Expectancy Finder, which is certain to decrease my productivity today.

At 11:21 AM, Blogger dexys_midnight said...

I am not a fan of Winning Percentage for a number of reasons. First, and biggest, as Chris said, the data is from 1979-90, which isn't as bad as taking 64-72, but is pretty darn bad when trying to compare it to 2005. I can't imagine it even being close.

Second, the sample size is so small for a bunch of the scenarios that there are a large number of situations where you can just LOOK at the numbers and know they are wrong. For example...I asked it for a Win % on if I were the home team, down a run, with men on 1st and second, 1 out--the percentage was 55%. But what if I subtract and out and load the none out, sacks juiced...percentage must jump a nice amount, right? No, it goes down to 53%. You can find bunches of examples like that with just a few minutes on there.
Third, there have been a bunch of studies on how clutch hitting doesn't exist, and this has an inherent clutch quality to it. Combining that with the low sample size and I'd say straight ERV is a much better evaluator of performance.

At 11:46 AM, Anonymous Dan said...

dexys_midnight: What inning is it in your example?

At 11:50 AM, Anonymous dan said...

I think WPA is clearly a lousy measure for individual performance, but does it accurately predict a 2005 team's probability of winning a game? I don't see why not. Baseball isn't all that different now than it was in the '80s, is it?

At 11:53 AM, Blogger DM said...

This comment has been removed by a blog administrator.

At 12:06 PM, Blogger DM said...

This comment has been removed by a blog administrator.

At 12:09 PM, Blogger DM said...

Here are some good links:

This one does a good job of explaining the strengths and weaknesses of Win Probability

And the guy who did this one appears to have data from 1972-2002, but the page only shows one table for the start of each inning

At 1:25 PM, Blogger dexys_midnight said...

Sorry, I can't believe I didn't put the inning down. It was the 4th.


Post a Comment

<< Home