Tom Tango highlighted my previous post on comparing computed RotoValue prices from projection systems, and he and others in the comments had some good suggestions for improving the player pool. So I’ve run some more data with slightly different sets of players.
First, for each league configuration, I’m simply using the top 230/240 players in the league as each system would project, and computing averages from that. The point is to compare only players that are projected be on somebody’s roster. So here if a system projects a player to be a star, but he gets hurt or is a bust, that system gets a big error on that player.
Second, I’m taking the actual top 230/240 players by RotoValue for the league, and comparing those to the projection. So in these averages, a bust who winds up not worth owning doesn’t count against a system, but failing to project an actual star at all does hurt that system. I do two variations in this table, first just averaging the subset of the top players that a system has projections for, and then assigning a price of zero (and thus the entire price becomes the error) for any player the system does not project. This comparison looks at the players that, in retrospect, deserved to be on a fantasy roster.
The short summary is that Steamer was still the best performing of these systems against 2012 data. Some other observations:
- Including a zero price for unprojected players actually lowers the average errors sometimes, but raises it others. The effect is rather small overall.
- The average price of the top players projected by a system should indeed be a constant, just the total league salary cap ($2600) divided by the number of players (230/240).
- When looking at the top players in 2012 actual RotoValue, the projection system’s average prices are themselves an interesting metric, with higher of course being better.
- Marcel beat Steamer in the shallow 5×5 MLB format when averaging the top actual players, and when looking at zero adjusted missing projections, RotoValue edged it out in the 4×4 NL. Steamer had the lowest errors in all other formats.
- Unadjusted 2011 data does surprisingly well, particularly when looking at the top players in 2011 prices.
- In the shallowest league, the 5×5 mixed, the average prices of the actual top players were less than half of what those players ultimately earned.
- Not surprisingly, the errors are much higher in that format also.
- These results are consistent with my previous post, although I do think this method is better targeted to fantasy players.
I’d like to get data for other systems for 2013, and then rerun this analysis next year.
When showing the top 240 actual players, the MAE0 and RMSE0 columns are computed assuming any non-projected player has $0 price, while the earlier columns are only averaging those players projected by the system.
First the 4×4 AL League – $260 Cap:
Top 240 players projected by each system
| Source | Num | Avg Price | MAE | RMSE |
|---|---|---|---|---|
| 2012 | 240 | 10.832 | 0.000 | 0.000 |
| Steamer | 240 | 10.833 | 7.848 | 10.127 |
| Marcel | 240 | 10.833 | 8.608 | 10.876 |
| 2011 | 240 | 10.833 | 8.747 | 10.896 |
| RotoValue | 240 | 10.832 | 8.880 | 11.223 |
| CAIRO | 240 | 10.833 | 9.119 | 11.451 |
Top 240 actual players players in 2012
| Source | Num | Avg Price | MAE | RMSE | MAE0 | RMSE0 |
|---|---|---|---|---|---|---|
| 2012 | 240 | 10.832 | 0.000 | 0.000 | 0.000 | 0.000 |
| Steamer | 231 | 8.286 | 7.421 | 9.661 | 7.294 | 9.543 |
| RotoValue | 211 | 8.911 | 7.765 | 9.851 | 7.699 | 10.092 |
| Marcel | 222 | 8.405 | 8.056 | 10.283 | 7.968 | 10.218 |
| 2011 | 218 | 8.751 | 8.564 | 11.000 | 8.280 | 10.742 |
| CAIRO | 235 | 6.722 | 9.179 | 11.731 | 9.042 | 11.617 |
Next 4×4 NL League – $260 Cap:
Top 230 players projected by each system
| Source | Num | Avg Price | MAE | RMSE |
|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 |
| Steamer | 230 | 11.304 | 8.927 | 11.042 |
| 2011 | 230 | 11.304 | 9.646 | 12.145 |
| RotoValue | 230 | 11.304 | 9.907 | 12.370 |
| Marcel | 230 | 11.304 | 9.741 | 12.443 |
| CAIRO | 230 | 11.304 | 10.966 | 13.533 |
Top 230 actual players players in 2012
| Source | Num | Avg Price | MAE | RMSE | MAE0 | RMSE0 |
|---|---|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 | 0.000 | 0.000 |
| Steamer | 229 | 7.820 | 8.809 | 10.967 | 8.775 | 10.943 |
| RotoValue | 209 | 8.413 | 8.910 | 11.177 | 8.559 | 10.869 |
| Marcel | 222 | 7.934 | 8.952 | 11.336 | 8.862 | 11.277 |
| 2011 | 215 | 8.232 | 9.507 | 11.990 | 9.260 | 11.769 |
| CAIRO | 229 | 6.812 | 10.657 | 13.109 | 10.615 | 13.080 |
Now 5×5 AL League $260 Cap:
Top 240 players projected by each system
| Source | Num | Avg Price | MAE | RMSE |
|---|---|---|---|---|
| 2012 | 240 | 10.833 | 0.000 | 0.000 |
| Steamer | 240 | 10.833 | 7.252 | 9.327 |
| Marcel | 240 | 10.831 | 8.042 | 10.053 |
| RotoValue | 240 | 10.832 | 8.140 | 10.143 |
| 2011 | 240 | 10.833 | 8.273 | 10.254 |
| CAIRO | 240 | 10.833 | 9.124 | 11.095 |
Top 240 actual players players in 2012
| Source | Num | Avg Price | MAE | RMSE | MAE0 | RMSE0 |
|---|---|---|---|---|---|---|
| 2012 | 240 | 10.833 | 0.000 | 0.000 | 0.000 | 0.000 |
| Steamer | 230 | 8.372 | 7.106 | 9.269 | 6.941 | 9.118 |
| RotoValue | 213 | 8.990 | 7.441 | 9.337 | 7.408 | 9.643 |
| Marcel | 222 | 8.511 | 7.728 | 9.881 | 7.661 | 9.871 |
| 2011 | 219 | 8.771 | 8.238 | 10.565 | 7.994 | 10.361 |
| CAIRO | 234 | 6.722 | 8.961 | 11.337 | 8.780 | 11.199 |
Next 5×5 NL League – $260 Cap:
Top 230 players projected by each system
| Source | Num | Avg Price | MAE | RMSE |
|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 |
| Steamer | 230 | 11.304 | 8.341 | 10.400 |
| 2011 | 230 | 11.304 | 9.124 | 11.430 |
| Marcel | 230 | 11.304 | 9.125 | 11.572 |
| RotoValue | 230 | 11.303 | 9.246 | 11.620 |
| CAIRO | 230 | 11.304 | 11.528 | 13.899 |
Top 230 actual players players in 2012
| Source | Num | Avg Price | MAE | RMSE | MAE0 | RMSE0 |
|---|---|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 | 0.000 | 0.000 |
| Steamer | 229 | 8.158 | 7.976 | 10.032 | 7.945 | 10.011 |
| RotoValue | 213 | 8.540 | 8.314 | 10.517 | 8.103 | 10.350 |
| Marcel | 224 | 8.193 | 8.469 | 10.563 | 8.456 | 10.591 |
| 2011 | 218 | 8.397 | 8.862 | 11.215 | 8.754 | 11.134 |
| CAIRO | 229 | 6.269 | 11.006 | 13.314 | 10.961 | 13.285 |
And finally a shallow league 5×5 Mixed League – $260 cap:
Top 230 players projected by each system
| Source | Num | Avg Price | MAE | RMSE |
|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 |
| Steamer | 230 | 11.304 | 11.361 | 14.281 |
| 2011 | 230 | 11.304 | 12.435 | 15.448 |
| Marcel | 230 | 11.304 | 12.747 | 15.938 |
| RotoValue | 230 | 11.304 | 12.731 | 15.957 |
| CAIRO | 230 | 11.304 | 14.519 | 17.929 |
Top 230 actual players players in 2012
| Source | Num | Avg Price | MAE | RMSE | MAE0 | RMSE0 |
|---|---|---|---|---|---|---|
| 2012 | 230 | 11.304 | 0.000 | 0.000 | 0.000 | 0.000 |
| Marcel | 223 | 5.148 | 11.454 | 14.260 | 11.394 | 14.172 |
| Steamer | 229 | 5.102 | 11.635 | 14.617 | 11.607 | 14.589 |
| RotoValue | 221 | 4.464 | 11.881 | 14.856 | 11.837 | 14.913 |
| 2011 | 223 | 5.070 | 11.820 | 14.886 | 11.713 | 14.748 |
| CAIRO | 230 | 3.683 | 13.002 | 16.041 | 13.002 | 16.041 |
1 comment
Comments are closed.