{"id":1100,"date":"2014-02-05T13:00:32","date_gmt":"2014-02-05T18:00:32","guid":{"rendered":"http:\/\/blog.rotovalue.com\/?p=1100"},"modified":"2014-02-05T13:00:32","modified_gmt":"2014-02-05T18:00:32","slug":"expanded-2013-woba-projections-comparison","status":"publish","type":"post","link":"https:\/\/blog.rotovalue.com\/index.php\/2014\/02\/05\/expanded-2013-woba-projections-comparison\/","title":{"rendered":"Expanded 2013 wOBA Projections Comparison"},"content":{"rendered":"<p><em><strong>7 February 2014:<\/strong>\u00a0I found a bug in the program that generated the second table, the one using wOBA &#8211; 0.020 for any players not forecast, so I&#8217;ve replaced the older table with a corrected version, and adjusted some of the other text to reflect that.<\/em><br \/>\nI&#8217;ve just posted a <a title=\"Monster 2013 Projection Review\" href=\"http:\/\/blog.rotovalue.com\/monster-2013-projection-review\/\">comparison<\/a> of nearly 20 different projection sources, using batting average for offense. I prefer a more comprehensive metric, and used Tom Tango&#8217;s <a href=\"http:\/\/www.insidethebook.com\/woba.shtml\">wOBA<\/a>\u00a0in my <a title=\"2013 Projection Systems Review: wOBA\" href=\"http:\/\/blog.rotovalue.com\/2013-projection-systems-review-woba\/\">earlier<\/a> analysis.<br \/>\nUnfortunately the data Dr. Larson&#8217;s <a href=\"http:\/\/www.bbprojectionproject.com\/\">site<\/a>\u00a0had was not sufficient to compute wOBA for many systems, so to include more systems, I went with the lower common denominator. Many fantasy leagues care about HR and average, but not (directly) 2B, 3B, or (often at all) BB. Certainly a real baseball team cares about more detailed offense.<br \/>\nMuch of the extra data from Dr. Larson does let me compute wOBA, as does the Oliver data Brian Cartwright shared, so this post runs the same analysis for wOBA, using only those sources where I have the data to compute it. If you&#8217;re reading this and your source was in my last post but not this one, send me more detailed data (to geoff at rotovalue dot com) and I&#8217;ll update this post to include your system.<br \/>\nThe sources I&#8217;m using here are:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.rlyw.net\/\">CAIRO<\/a>\u00a0&#8211; from S B of the\u00a0<a href=\"http:\/\/www.rlyw.net\/\">Replacement Level Yankees Weblog<\/a>.<\/li>\n<li><a href=\"http:\/\/www.tangotiger.net\/marcel\/\">Marcel<\/a>\u00a0&#8211; the basic projection model from Tom Tango, coauthor of\u00a0<a href=\"http:\/\/www.insidethebook.com\/\">The Book<\/a>. This year I\u2019m using Marcel numbers generated by\u00a0<a href=\"http:\/\/www.baseballheatmaps.com\/marcel-database-download\/\">Jeff Zimmerman<\/a>, using\u00a0<a href=\"http:\/\/summerofjeff.wordpress.com\/2011\/01\/14\/python-code-for-marcel-projections\/\">Jeff Sackmann\u2019s<\/a>\u00a0Python code.<\/li>\n<li><a href=\"http:\/\/morps.mlblogs.com\/\">MORPS<\/a>\u00a0&#8211; A projection model by Tim Oberschlake.<\/li>\n<li><a href=\"http:\/\/steamerprojections.com\/\">Steamer\/Razzball<\/a>\u00a0&#8211; Rate projections by Jared Cross, Dash Davidson, and Peter Rosenbloom, and playing time projections from Rudy Gamble of\u00a0<a href=\"http:\/\/razzball.com\/\">Razzball.com<\/a>.<\/li>\n<li><a href=\"http:\/\/blog.rotovalue.com\/?p=181\">RotoValue<\/a>\u00a0&#8211; my current model, based largely on Marcel, but with adjustments for\u00a0<a href=\"http:\/\/blog.rotovalue.com\/projecting-wins-saves-and-holds\/\">pitching decision stats<\/a>\u00a0and assuming no pitcher skill in BABIP.<\/li>\n<li><a href=\"http:\/\/www.baseballthinkfactory.org\/\" target=\"_blank\" rel=\"noopener\">Zips<\/a>\u00a0&#8211; A projection model from Dan Szymborski.<\/li>\n<li><a href=\"http:\/\/fantasynews.cbssports.com\/fantasybaseball\/stats\/sortable\/points\/SP\/standard\/projections\" target=\"_blank\" rel=\"noopener\">CBS Sportsline<\/a><\/li>\n<li><a href=\"http:\/\/games.espn.go.com\/frontpage\/baseball\" target=\"_blank\" rel=\"noopener\">ESPN<\/a><\/li>\n<li><a href=\"http:\/\/rotochamp.com\/\" target=\"_blank\" rel=\"noopener\">RotoChamps<\/a><\/li>\n<li><a href=\"http:\/\/www.fangraphs.com\/projections.aspx?pos=all&amp;stats=bat&amp;type=fan\" target=\"_blank\" rel=\"noopener\">Fangraphs Fans<\/a><\/li>\n<li><a href=\"http:\/\/www.hardballtimes.com\/introducing-oliver\/\">Oliver<\/a>\u00a0&#8211;\u00a0Brian Cartwright&#8217;s system (2013 data available at\u00a0\u00a0<a href=\"http:\/\/www.fangraphs.com\/projections.aspx?pos=all&amp;stats=bat&amp;type=oliver\">Fangraphs.com<\/a>).<\/li>\n<\/ul>\n<p><!--more--><\/p>\n<table>\n<tbody>\n<tr>\n<th>Source<\/th>\n<th>Num<\/th>\n<th>Avg wOBA<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>310<\/td>\n<td>0.3312<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>310<\/td>\n<td>0.3383<\/td>\n<td>0.0232<\/td>\n<td>0.0300<\/td>\n<\/tr>\n<tr>\n<td>AllConsensus<\/td>\n<td>310<\/td>\n<td>0.3421<\/td>\n<td>0.0235<\/td>\n<td>0.0305<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>310<\/td>\n<td>0.3375<\/td>\n<td>0.0238<\/td>\n<td>0.0306<\/td>\n<\/tr>\n<tr>\n<td>Consensus<\/td>\n<td>310<\/td>\n<td>0.3397<\/td>\n<td>0.0237<\/td>\n<td>0.0310<\/td>\n<\/tr>\n<tr>\n<td>ZiPS<\/td>\n<td>310<\/td>\n<td>0.3394<\/td>\n<td>0.0234<\/td>\n<td>0.0311<\/td>\n<\/tr>\n<tr>\n<td>FangraphsFans<\/td>\n<td>310<\/td>\n<td>0.3467<\/td>\n<td>0.0244<\/td>\n<td>0.0313<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>310<\/td>\n<td>0.3396<\/td>\n<td>0.0243<\/td>\n<td>0.0314<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>310<\/td>\n<td>0.3436<\/td>\n<td>0.0245<\/td>\n<td>0.0321<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>310<\/td>\n<td>0.3367<\/td>\n<td>0.0244<\/td>\n<td>0.0321<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>310<\/td>\n<td>0.3445<\/td>\n<td>0.0249<\/td>\n<td>0.0326<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>310<\/td>\n<td>0.3475<\/td>\n<td>0.0258<\/td>\n<td>0.0328<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>310<\/td>\n<td>0.3540<\/td>\n<td>0.0247<\/td>\n<td>0.0328<\/td>\n<\/tr>\n<tr>\n<td>y2012<\/td>\n<td>310<\/td>\n<td>0.3420<\/td>\n<td>0.0295<\/td>\n<td>0.0387<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Removing systems with less data also gives a larger set of commonly projected players, which is good. So not only is this a better overall statistic for comparison, the subset of players projected by all systems gives better insight into how the systems performed.<br \/>\nThe range of errors in this group is actually tighter than what I saw with Avg, with just 0.0028 separating the lowest RMSE (Steamer\/Razzball) from the highest, even though wOBA numbers are higher than batting averages. So I&#8217;d say all these systems do a pretty good job here on these players.<br \/>\nWhen I add in average wOBA minus 0.020 for players not forecast by a system, I get this:<\/p>\n<table>\n<tbody>\n<tr>\n<th>Source<\/th>\n<th>Num<\/th>\n<th>wOBA<\/th>\n<th>MLB<\/th>\n<th>wOBA<\/th>\n<th>StdDev<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<th>Missing<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>634<\/td>\n<td>0.3236<\/td>\n<td>634<\/td>\n<td>0.3236<\/td>\n<td>0.0439<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>504<\/td>\n<td>0.3357<\/td>\n<td>634<\/td>\n<td>0.3319<\/td>\n<td>0.0254<\/td>\n<td>0.0261<\/td>\n<td>0.0357<\/td>\n<td>163<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>1445<\/td>\n<td>0.3014<\/td>\n<td>634<\/td>\n<td>0.3295<\/td>\n<td>0.0276<\/td>\n<td>0.0263<\/td>\n<td>0.0358<\/td>\n<td>23<\/td>\n<\/tr>\n<tr>\n<td>AllConsensus<\/td>\n<td>1514<\/td>\n<td>0.3031<\/td>\n<td>634<\/td>\n<td>0.3351<\/td>\n<td>0.0255<\/td>\n<td>0.0261<\/td>\n<td>0.0358<\/td>\n<td>15<\/td>\n<\/tr>\n<tr>\n<td>ZiPS<\/td>\n<td>1000<\/td>\n<td>0.3120<\/td>\n<td>634<\/td>\n<td>0.3309<\/td>\n<td>0.0285<\/td>\n<td>0.0262<\/td>\n<td>0.0362<\/td>\n<td>29<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>507<\/td>\n<td>0.3363<\/td>\n<td>634<\/td>\n<td>0.3337<\/td>\n<td>0.0257<\/td>\n<td>0.0269<\/td>\n<td>0.0364<\/td>\n<td>178<\/td>\n<\/tr>\n<tr>\n<td>Consensus<\/td>\n<td>786<\/td>\n<td>0.3322<\/td>\n<td>634<\/td>\n<td>0.3333<\/td>\n<td>0.0243<\/td>\n<td>0.0267<\/td>\n<td>0.0364<\/td>\n<td>80<\/td>\n<\/tr>\n<tr>\n<td>FangraphsFans<\/td>\n<td>331<\/td>\n<td>0.3449<\/td>\n<td>634<\/td>\n<td>0.3416<\/td>\n<td>0.0264<\/td>\n<td>0.0269<\/td>\n<td>0.0365<\/td>\n<td>311<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>750<\/td>\n<td>0.3306<\/td>\n<td>634<\/td>\n<td>0.3367<\/td>\n<td>0.0243<\/td>\n<td>0.0273<\/td>\n<td>0.0373<\/td>\n<td>105<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>539<\/td>\n<td>0.3367<\/td>\n<td>634<\/td>\n<td>0.3384<\/td>\n<td>0.0259<\/td>\n<td>0.0277<\/td>\n<td>0.0378<\/td>\n<td>164<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>434<\/td>\n<td>0.3490<\/td>\n<td>634<\/td>\n<td>0.3468<\/td>\n<td>0.0270<\/td>\n<td>0.0282<\/td>\n<td>0.0384<\/td>\n<td>231<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>751<\/td>\n<td>0.3293<\/td>\n<td>634<\/td>\n<td>0.3295<\/td>\n<td>0.0282<\/td>\n<td>0.0279<\/td>\n<td>0.0396<\/td>\n<td>105<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>751<\/td>\n<td>0.3359<\/td>\n<td>634<\/td>\n<td>0.3346<\/td>\n<td>0.0435<\/td>\n<td>0.0319<\/td>\n<td>0.0461<\/td>\n<td>88<\/td>\n<\/tr>\n<tr>\n<td>y2012<\/td>\n<td>611<\/td>\n<td>0.3298<\/td>\n<td>634<\/td>\n<td>0.3289<\/td>\n<td>0.0489<\/td>\n<td>0.0361<\/td>\n<td>0.0522<\/td>\n<td>129<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Now the spread is wider. The filter of being able to compute wOBA removed most of the systems with very few projected players, <del>but it is interesting to see that the lowest errors come from Fangraphs Fans projections, which also happens to project the fewest players by far. Shifting to this test sees the Steamer\/Razzball RMSE rise by 0.0020, while the Fangraphs RMSE drops by 0.0036, more than 10%.<\/del>\u00a0and there&#8217;s no obvious relationship between number of players forecast and the RMSE errors of a system.<br \/>\nSteamer\/Razzball still tops this chart, with Oliver and AllConsensus in a virtual tie now.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>7 February 2014:\u00a0I found a bug in the program that generated the second table, the one using wOBA &#8211; 0.020 for any players not forecast, so I&#8217;ve replaced the older table with a corrected version, and adjusted some of the other text to reflect that. I&#8217;ve just posted a comparison of nearly 20 different projection&hellip; <a class=\"more-link\" href=\"https:\/\/blog.rotovalue.com\/index.php\/2014\/02\/05\/expanded-2013-woba-projections-comparison\/\">Continue reading <span class=\"screen-reader-text\">Expanded 2013 wOBA Projections Comparison<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,11,14],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts\/1100"}],"collection":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/comments?post=1100"}],"version-history":[{"count":0,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts\/1100\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/media?parent=1100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/categories?post=1100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/tags?post=1100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}