Projected Playing Time Comparison

My last post effectively compared projection systems by how well they predicted player skill level: I scaled each projection to match 2012 actual ab/ip, but used the rates each system projected to generate stats.
Tango asked if I could run the reverse, a comparison where I used prorated actual 2012 stats but kept only the playing time projections of various systems. So I’ve run those numbers now also for the same 5 league formats I’ve been doing. Steamer usually has the lowest errors in this format, with my old RotoValue model usually second lowest. This seems reasonable, as the other models don’t make any specific attempt to project playing time.

As in my last post, RMSE0 and MAE0 are the root mean square error and mean absolute error when I assume any player not projected by a system has a price of $0.00.
First up, the 4×4 AL League – $260 Cap:

Source Num Avg Price MAE RMSE
2012 240 10.832 0.000 0.000
PTSteamer 240 10.833 4.419 6.251
PTRotoValue 240 10.833 5.084 6.954
PTMarcel 240 10.833 6.100 9.042
PT2011 240 10.833 8.627 10.724
PTZiPS 240 10.833 7.730 10.795
PTCAIRO 240 10.833 7.901 10.851

Top 240 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 240 10.832 0.000 0.000 0.000 0.000
PTSteamer 231 9.829 4.308 6.153 4.298 6.138
PTRotoValue 207 11.250 4.834 6.734 5.110 7.483
PTMarcel 222 10.061 5.385 7.561 5.498 7.712
PTZiPS 228 8.721 6.241 7.900 6.395 8.272
PTCAIRO 235 8.393 7.007 9.289 6.914 9.203
PT2011 218 8.822 8.534 10.964 8.252 10.709

Steamer winds up on top here, followed by RotoValue, and then the other systems further back. This makes intuitive sense, because none of Marcel, ZiPS, or CAIRO make any effort to projecting playing time. My old RotoValue model does account for known preseason injuries at the time I last run the projections, and it also tries to adjust time via an algorithm when team totals, or even position totals within a team, get too high or too low.
Now the 4×4 NL League – $260 Cap:

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
PTSteamer 230 11.304 5.605 7.534
PTRotoValue 230 11.304 5.947 8.086
PTMarcel 230 11.304 6.438 8.477
PTZiPS 230 11.304 7.683 10.475
PT2011 230 11.304 9.630 12.141
PTCAIRO 230 11.304 8.983 12.515

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
PTMarcel 222 9.913 5.851 7.480 5.869 7.559
PTSteamer 229 9.277 5.822 7.726 5.800 7.710
PTRotoValue 206 10.969 5.951 7.903 5.921 7.937
PTZiPS 221 8.971 6.604 8.484 6.596 8.459
PTCAIRO 229 7.957 7.522 9.498 7.493 9.478
PT2011 215 8.275 9.517 12.003 9.270 11.782

Here Marcel wound up with the lowest errors among the actual top 230, even after accounting for  players it did not project (I realize Tango has defined a Marcel projection for anybody not in the file, but that basically gives a fantasy player no predictive information on which non-projected players might do better; also I’d need to hack something to make that adjustment).
Among the system’s projections, Steamer was lowest, followed by RotoValue and then Marcel.
Next the 5×5 AL League $260 Cap:

Source Num Avg Price MAE RMSE
2012 240 10.833 0.000 0.000
PTSteamer 240 10.833 4.707 6.485
PTRotoValue 240 10.832 5.439 7.310
PTMarcel 240 10.832 6.275 9.047
PT2011 240 10.833 8.173 10.068
PTZiPS 240 10.833 8.178 11.031
PTCAIRO 240 10.833 8.560 11.277

Top 240 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 240 10.833 0.000 0.000 0.000 0.000
PTSteamer 230 9.609 4.483 6.175 4.428 6.111
PTRotoValue 209 10.751 4.997 6.912 5.237 7.601
PTMarcel 222 9.587 5.333 7.503 5.446 7.694
PTZiPS 228 7.968 6.705 8.172 6.804 8.468
PTCAIRO 234 7.793 7.465 9.604 7.321 9.489
PT2011 219 8.860 8.200 10.520 7.959 10.319

Steamer again had the lowest errors, followed by RotoValue. I do notice that RotoValue had the fewest projected players among actual in many formats, and its error averages rise when counting such players at $0.00, which is appropriate (a system should be penalized for failing to project valuable players).
The 5×5 NL League – $260 Cap:

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
PTSteamer 230 11.304 5.500 7.251
PTRotoValue 230 11.304 6.219 8.240
PTMarcel 230 11.304 6.552 8.490
PTZiPS 230 11.304 8.225 10.994
PT2011 230 11.304 9.071 11.392
PTCAIRO 230 11.304 9.791 13.392

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
PTSteamer 229 9.208 5.616 7.420 5.595 7.404
PTMarcel 224 9.595 6.044 7.594 6.095 7.724
PTRotoValue 210 10.354 6.098 8.059 6.106 8.121
PTZiPS 223 8.351 7.373 9.027 7.387 9.026
PTCAIRO 229 7.220 8.077 9.811 8.045 9.789
PT2011 218 8.428 8.863 11.215 8.755 11.134

Here Marcel beats RotoValue as runner up when looking at actual best players, while Steamer still has the lowest errors overall.
Finally the shallow 5×5 Mixed League – $260 cap:

Source Num Avg Price MAE RMSE
2012 230 11.304 0.000 0.000
PTSteamer 230 11.304 7.585 10.562
PTRotoValue 230 11.304 8.221 11.345
PTMarcel 230 11.304 9.029 13.301
PT2011 230 11.304 12.369 15.394
PTZiPS 230 11.304 12.033 17.864
PTCAIRO 230 11.304 13.038 18.748

Top 230 actual players players in 2012

Source Num Avg Price MAE RMSE MAE0 RMSE0
2012 230 11.304 0.000 0.000 0.000 0.000
PTSteamer 229 7.891 7.084 9.897 7.076 9.881
PTZiPS 225 5.999 8.497 10.448 8.521 10.477
PTMarcel 223 7.145 8.016 10.791 8.061 10.799
PTRotoValue 218 7.675 7.942 10.879 8.045 11.130
PTCAIRO 230 5.046 8.960 11.188 8.960 11.188
PT2011 223 5.096 11.786 14.839 11.680 14.703

It’s interesting to see ZiPS performing rather well in this model among 2012 top players, although in a shallower league, the player pool is mostly good players all expected to get a lot of time. So it’s not so surprising that simple methods like some weighted average of recent performance could hold up rather well. Also I think ZiPS projects players based on previous playing time, be it in MLB or the minors, and so it projects a lot of time for several players who will simply not be playing much at the MLB level. Some of those players may make the top 230 prices based on ZiPS projections, and thus its errors in that format would be much larger than simply looking at the actual top players.

Leave a comment

Your email address will not be published. Required fields are marked *