# Revised Projected RotoValue Comparison

Tom Tango highlighted my previous post on comparing computed RotoValue prices from projection systems, and he and others in the comments had some good suggestions for improving the player pool. So I’ve run some more data with slightly different sets of players.

First, for each league configuration, I’m simply using the top 230/240 players in the league as each system would project, and computing averages from that. The point is to compare only players that are projected be on somebody’s roster. So here if a system projects a player to be a star, but he gets hurt or is a bust, that system gets a big error on that player.

Second, I’m taking the actual top 230/240 players by RotoValue for the league, and comparing those to the projection. So in these averages, a bust who winds up not worth owning doesn’t count against a system, but failing to project an actual star at all does hurt that system. I do two variations in this table, first just averaging the subset of the top players that a system has projections for, and then assigning a price of zero (and thus the entire price becomes the error) for any player the system does not project. This comparison looks at the players that, in retrospect, deserved to be on a fantasy roster.

The short summary is that Steamer was still the best performing of these systems against 2012 data. Some other observations:

• Including a zero price for unprojected players actually lowers the average errors sometimes, but raises it others. The effect is rather small overall.
• The average price of the top players projected by a system should indeed be a constant, just the total league salary cap (\$2600) divided by the number of players (230/240).
• When looking at the top players in 2012 actual RotoValue, the projection system’s average prices are themselves an interesting metric, with higher of course being better.
• Marcel beat Steamer in the shallow 5×5 MLB format when averaging the top actual players, and when looking at zero adjusted missing projections, RotoValue edged it out in the 4×4 NL. Steamer had the lowest errors in all other formats.
• Unadjusted 2011 data does surprisingly well, particularly when looking at the top players in 2011 prices.
• In the shallowest league, the 5×5 mixed, the average prices of the actual top players were less than half of what those players ultimately earned.
• Not surprisingly, the errors are much higher in that format also.
• These results are consistent with my previous post, although I do think this method is better targeted to fantasy players.

I’d like to get data for other systems for 2013, and then rerun this analysis next year.

When showing the top 240 actual players, the MAE0 and RMSE0 columns are computed assuming any non-projected player has \$0 price, while the earlier columns are only averaging those players projected by the system.

First the 4×4 AL League – \$260 Cap:

Top 240 players projected by each system

Source Num Avg Price MAE RMSE
2012 240 10.832 0.000 0.000
Steamer 240 10.833 7.848 10.127
Marcel 240 10.833 8.608 10.876
2011 240 10.833 8.747 10.896
RotoValue 240 10.832 8.880 11.223
CAIRO 240 10.833 9.119 11.451

Top 240 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 240 10.832 0.000 0.000 0.000 0.000
Steamer 231 8.286 7.421 9.661 7.294 9.543
RotoValue 211 8.911 7.765 9.851 7.699 10.092
Marcel 222 8.405 8.056 10.283 7.968 10.218
2011 218 8.751 8.564 11.000 8.280 10.742
CAIRO 235 6.722 9.179 11.731 9.042 11.617

Next 4×4 NL League – \$260 Cap:
Top 230 players projected by each system

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
Steamer 230 11.304 8.927 11.042
2011 230 11.304 9.646 12.145
RotoValue 230 11.304 9.907 12.370
Marcel 230 11.304 9.741 12.443
CAIRO 230 11.304 10.966 13.533

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
Steamer 229 7.820 8.809 10.967 8.775 10.943
RotoValue 209 8.413 8.910 11.177 8.559 10.869
Marcel 222 7.934 8.952 11.336 8.862 11.277
2011 215 8.232 9.507 11.990 9.260 11.769
CAIRO 229 6.812 10.657 13.109 10.615 13.080

Top 240 players projected by each system

Source Num Avg Price MAE RMSE
2012 240 10.833 0.000 0.000
Steamer 240 10.833 7.252 9.327
Marcel 240 10.831 8.042 10.053
RotoValue 240 10.832 8.140 10.143
2011 240 10.833 8.273 10.254
CAIRO 240 10.833 9.124 11.095

Top 240 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 240 10.833 0.000 0.000 0.000 0.000
Steamer 230 8.372 7.106 9.269 6.941 9.118
RotoValue 213 8.990 7.441 9.337 7.408 9.643
Marcel 222 8.511 7.728 9.881 7.661 9.871
2011 219 8.771 8.238 10.565 7.994 10.361
CAIRO 234 6.722 8.961 11.337 8.780 11.199

Top 230 players projected by each system

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
Steamer 230 11.304 8.341 10.400
2011 230 11.304 9.124 11.430
Marcel 230 11.304 9.125 11.572
RotoValue 230 11.303 9.246 11.620
CAIRO 230 11.304 11.528 13.899

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
Steamer 229 8.158 7.976 10.032 7.945 10.011
RotoValue 213 8.540 8.314 10.517 8.103 10.350
Marcel 224 8.193 8.469 10.563 8.456 10.591
2011 218 8.397 8.862 11.215 8.754 11.134
CAIRO 229 6.269 11.006 13.314 10.961 13.285

And finally a shallow league 5×5 Mixed League – \$260 cap:

Top 230 players projected by each system

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
Steamer 230 11.304 11.361 14.281
2011 230 11.304 12.435 15.448
Marcel 230 11.304 12.747 15.938
RotoValue 230 11.304 12.731 15.957
CAIRO 230 11.304 14.519 17.929

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
Marcel 223 5.148 11.454 14.260 11.394 14.172
Steamer 229 5.102 11.635 14.617 11.607 14.589
RotoValue 221 4.464 11.881 14.856 11.837 14.913
2011 223 5.070 11.820 14.886 11.713 14.748
CAIRO 230 3.683 13.002 16.041 13.002 16.041