{"id":1342,"date":"2015-02-14T19:21:56","date_gmt":"2015-02-15T00:21:56","guid":{"rendered":"http:\/\/blog.rotovalue.com\/?p=1342"},"modified":"2015-02-14T19:21:56","modified_gmt":"2015-02-15T00:21:56","slug":"comparing-2014-projections-era-and-whip","status":"publish","type":"post","link":"https:\/\/blog.rotovalue.com\/index.php\/2015\/02\/14\/comparing-2014-projections-era-and-whip\/","title":{"rendered":"Comparing 2014 Projections &#8211; ERA and WHIP"},"content":{"rendered":"<p><a title=\"Comparing 2014 Projections \u2013 wOBA\" href=\"http:\/\/blog.rotovalue.com\/comparing-2014-projections-woba\/\">Yesterday<\/a> I ran comparisons of several projections systems for an all-inclusive batting statistic, wOBA. Today I&#8217;m running the same tests, computing\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Root-mean-square_deviation\">root mean square error<\/a>\u00a0(RMSE) and\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Mean_absolute_error\">mean absolute error<\/a>\u00a0(MAE), for two commonly used fantasy statistics, ERA and WHIP. These tests are bias-adjusted, so what matters is a player&#8217;s\u00a0ERA or WHIP relative to the overall average of that system, compared with the player&#8217;s actual statistic relative to the actual overall average. The lower the RMSE or MAE, the better a projection system predicted the actual data.<br \/>\nI have data for these projection models:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.cs.virginia.edu\/~rjg7v\/AggPro.pdf\">AggPro<\/a> \u2013 A projection aggregation method from Ross J. Gore, Cameron T. Snapp, and Timothy Highley.<\/li>\n<li>Bayesball \u2013 Projections from Jonathan Adams.<\/li>\n<li><a href=\"http:\/\/www.rlyw.net\/\">CAIRO<\/a>\u00a0\u2013 from S B of the\u00a0<a href=\"http:\/\/www.rlyw.net\/\">Replacement Level Yankees Weblog<\/a>.<\/li>\n<li><a href=\"http:\/\/fantasynews.cbssports.com\/fantasybaseball\/stats\/sortable\/points\/SP\/standard\/projections\" target=\"_blank\" rel=\"noopener\">CBS<\/a>\u00a0Projections from CBS Sportsline.<\/li>\n<li><a href=\"http:\/\/claydavenport.com\/projections\/PROJHOME.shtml\">Davenport<\/a>\u00a0Clay Davenport\u2019s projections.<\/li>\n<li><a href=\"http:\/\/games.espn.go.com\/frontpage\/baseball\" target=\"_blank\" rel=\"noopener\">ESPN<\/a>\u00a0Projections from ESPN.<\/li>\n<li><a href=\"http:\/\/www.fangraphs.com\/projections.aspx?pos=all&amp;stats=bat&amp;type=fan\" target=\"_blank\" rel=\"noopener\">Fans<\/a>\u00a0Fans\u2019 projections from <a href=\"http:\/\/www.fangraphs.com\/\">Fangraphs.com<\/a>.<\/li>\n<li><a href=\"http:\/\/www.bbprojectionproject.com\/\" target=\"_blank\" rel=\"noopener\">Larson<\/a>\u00a0Will Larson\u2019s projections.<\/li>\n<li><a href=\"http:\/\/www.tangotiger.net\/marcel\/\">Marcel<\/a>\u00a0\u2013 the basic projection model from Tom Tango, coauthor of\u00a0<a href=\"http:\/\/www.insidethebook.com\/\">The Book<\/a>. This year I\u2019m using Marcel numbers generated by <a href=\"http:\/\/www.baseballheatmaps.com\/marcel-database-download\/\">Jeff Zimmerman<\/a>, using <a href=\"http:\/\/summerofjeff.wordpress.com\/2011\/01\/14\/python-code-for-marcel-projections\/\">Jeff Sackmann\u2019s<\/a>\u00a0Python code.<\/li>\n<li><a href=\"http:\/\/morps.mlblogs.com\/\">MORPS<\/a> \u2013 A projection model by Tim Oberschlake.<\/li>\n<li><a href=\"http:\/\/bats.blogs.nytimes.com\/author\/dan-rosenheck\/\" target=\"_blank\" rel=\"noopener\">Rosenheck<\/a>\u00a0Projections by Dan Rosenheck.<\/li>\n<li><a href=\"http:\/\/www.hardballtimes.com\/introducing-oliver\/\">Oliver<\/a>\u00a0\u2013 Brian Cartwright\u2019s projection model.<\/li>\n<li><a href=\"http:\/\/steamerprojections.com\/\">Steamer<\/a>\u00a0\u2013 Projections by Jared Cross, Dash Davidson, and Peter Rosenbloom.<\/li>\n<li><a href=\"http:\/\/steamerprojections.com\/\">Steamer\/Razzball<\/a> \u2013 Steamer rate projections, but\u00a0playing time projections from Rudy Gamble of <a href=\"http:\/\/razzball.com\/\">Razzball.com<\/a>.<\/li>\n<li><a href=\"http:\/\/blog.rotovalue.com\/?p=181\">RotoValue<\/a>\u00a0\u2013 my current model, based largely on Marcel, but with adjustments for<a href=\"http:\/\/blog.rotovalue.com\/projecting-wins-saves-and-holds\/\">pitching decision stats<\/a> and assuming no pitcher skill in BABIP.<\/li>\n<li><a href=\"http:\/\/www.rotovalue.com\/cgi-bin\/Search?year=2014&amp;league=51&amp;source=RV%20Pre-Australia\">RV Pre-Australia<\/a>\u00a0\u2013 The RotoValue projections taken just before the first Australia games last year. Before the rest of the regular season I continued to tweak projections slightly.<\/li>\n<li><a href=\"http:\/\/www.baseballthinkfactory.org\/oracle\/discussion\/2012_zips_projections_spreadsheets_v._1\">ZiPS<\/a>\u00a0\u2013 projections from Dan Szymborski of\u00a0<a href=\"http:\/\/www.baseballthinkfactory.org\/\">Baseball Think Factory<\/a>\u00a0and\u00a0<a href=\"http:\/\/search.espn.go.com\/dan-szymborski\/\">ESPN<\/a>.<\/li>\n<\/ul>\n<p>First up is ERA, comparing the 75 pitchers projected by all systems:<!--more--><\/p>\n<table>\n<tr>\n<th>Source<\/th>\n<th>Num<\/th>\n<th>Avg  ERA<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>75<\/td>\n<td>3.4261<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<\/tr>\n<tr>\n<td>Larson<\/td>\n<td>75<\/td>\n<td>3.7608<\/td>\n<td>0.5834<\/td>\n<td>0.7644<\/td>\n<\/tr>\n<tr>\n<td>All Consensus<\/td>\n<td>75<\/td>\n<td>3.6435<\/td>\n<td>0.5780<\/td>\n<td>0.7693<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>75<\/td>\n<td>3.5056<\/td>\n<td>0.5723<\/td>\n<td>0.7700<\/td>\n<\/tr>\n<tr>\n<td>Steamer<\/td>\n<td>75<\/td>\n<td>3.8116<\/td>\n<td>0.6015<\/td>\n<td>0.7781<\/td>\n<\/tr>\n<tr>\n<td>Zips<\/td>\n<td>75<\/td>\n<td>3.5525<\/td>\n<td>0.5873<\/td>\n<td>0.7868<\/td>\n<\/tr>\n<tr>\n<td>ESPN<\/td>\n<td>75<\/td>\n<td>3.7014<\/td>\n<td>0.5878<\/td>\n<td>0.7882<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>75<\/td>\n<td>3.5329<\/td>\n<td>0.5981<\/td>\n<td>0.7886<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>75<\/td>\n<td>3.8057<\/td>\n<td>0.6164<\/td>\n<td>0.7905<\/td>\n<\/tr>\n<tr>\n<td>RotoGuru<\/td>\n<td>75<\/td>\n<td>3.8089<\/td>\n<td>0.5713<\/td>\n<td>0.8011<\/td>\n<\/tr>\n<tr>\n<td>RV Pre-Australia<\/td>\n<td>75<\/td>\n<td>3.6518<\/td>\n<td>0.6038<\/td>\n<td>0.8037<\/td>\n<\/tr>\n<tr>\n<td>Fans<\/td>\n<td>75<\/td>\n<td>3.4440<\/td>\n<td>0.6275<\/td>\n<td>0.8047<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>75<\/td>\n<td>3.6512<\/td>\n<td>0.6052<\/td>\n<td>0.8049<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>75<\/td>\n<td>3.5315<\/td>\n<td>0.6179<\/td>\n<td>0.8066<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>75<\/td>\n<td>3.5351<\/td>\n<td>0.6159<\/td>\n<td>0.8222<\/td>\n<\/tr>\n<tr>\n<td>Razzball<\/td>\n<td>75<\/td>\n<td>3.4998<\/td>\n<td>0.6469<\/td>\n<td>0.8231<\/td>\n<\/tr>\n<tr>\n<td>Davenport<\/td>\n<td>75<\/td>\n<td>3.5964<\/td>\n<td>0.6320<\/td>\n<td>0.8270<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>75<\/td>\n<td>3.6321<\/td>\n<td>0.6367<\/td>\n<td>0.8300<\/td>\n<\/tr>\n<tr>\n<td>AggPro<\/td>\n<td>75<\/td>\n<td>3.7284<\/td>\n<td>0.6879<\/td>\n<td>0.8411<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>75<\/td>\n<td>3.6513<\/td>\n<td>0.6742<\/td>\n<td>0.8525<\/td>\n<\/tr>\n<tr>\n<td>Bayesball<\/td>\n<td>75<\/td>\n<td>3.9120<\/td>\n<td>0.7358<\/td>\n<td>0.8876<\/td>\n<\/tr>\n<tr>\n<td>y2013<\/td>\n<td>75<\/td>\n<td>3.3790<\/td>\n<td>0.7074<\/td>\n<td>0.9554<\/td>\n<\/tr>\n<\/table>\n<p>Not surprisingly the errors, even as a percentage of the average, are much higher here than for wOBA, as pitching performance is more volatile than batting performance. Will Larson&#8217;s projections did best here, followed by the Consensus, CBS Sportsline, and Steamer. All the models handily beat using 2013 data, albeit not as decisively as with wOBA, but but seven lagged behind Tango&#8217;s Marcel system. The other notable thing to me is that the average ERA for these players of <em>every<\/em> system is higher than the actual. Indeed 2013 actual data was slightly lower, so this shows how dependent systems are on older historical data. Aggregate ERA has fallen sharply since 2012, and the projection models are still reflecting that higher run environment to a large degree.<br \/>\nWhen you&#8217;re preparing for a fantasy auction, you care much more about how a system rates players relative to each other than how it rates them compared to the actual data. For these 75 pitchers, Steamer projected an aggregate 3.81 ERA, while they actually had a 3.43 ERA (weighted by actual 2014 innings pitched). But Steamer still ranks lower in errors than Fangraphs Fans&#8217; projections, which came closest to the actual average of 3.44. This is the impact of bias adjustment: while the fans did better at projecting the actual league average ERA, Steamer tended to be closer on more individual players when adjusted for its league average.<br \/>\nTo underscore that point, here&#8217;s the same ERA table as above, but this time doing raw errors, i.e. not doing any bias adjustment at all:<\/p>\n<table>\n<tr>\n<th>Source<\/th>\n<th>Num<\/th>\n<th>Avg  ERA<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>75<\/td>\n<td>3.4261<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>75<\/td>\n<td>3.5056<\/td>\n<td>0.5900<\/td>\n<td>0.7739<\/td>\n<\/tr>\n<tr>\n<td>Zips<\/td>\n<td>75<\/td>\n<td>3.5525<\/td>\n<td>0.6152<\/td>\n<td>0.7820<\/td>\n<\/tr>\n<tr>\n<td>Fans<\/td>\n<td>75<\/td>\n<td>3.4440<\/td>\n<td>0.5901<\/td>\n<td>0.7823<\/td>\n<\/tr>\n<tr>\n<td>Razzball<\/td>\n<td>75<\/td>\n<td>3.4998<\/td>\n<td>0.5887<\/td>\n<td>0.7834<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>75<\/td>\n<td>3.5329<\/td>\n<td>0.6043<\/td>\n<td>0.7923<\/td>\n<\/tr>\n<tr>\n<td>All Consensus<\/td>\n<td>75<\/td>\n<td>3.6435<\/td>\n<td>0.6250<\/td>\n<td>0.7993<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>75<\/td>\n<td>3.5315<\/td>\n<td>0.6184<\/td>\n<td>0.8069<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>75<\/td>\n<td>3.5351<\/td>\n<td>0.6375<\/td>\n<td>0.8271<\/td>\n<\/tr>\n<tr>\n<td>AggPro<\/td>\n<td>75<\/td>\n<td>3.7284<\/td>\n<td>0.6699<\/td>\n<td>0.8273<\/td>\n<\/tr>\n<tr>\n<td>ESPN<\/td>\n<td>75<\/td>\n<td>3.7014<\/td>\n<td>0.6477<\/td>\n<td>0.8278<\/td>\n<\/tr>\n<tr>\n<td>RV Pre-Australia<\/td>\n<td>75<\/td>\n<td>3.6518<\/td>\n<td>0.6385<\/td>\n<td>0.8314<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>75<\/td>\n<td>3.6512<\/td>\n<td>0.6384<\/td>\n<td>0.8317<\/td>\n<\/tr>\n<tr>\n<td>Larson<\/td>\n<td>75<\/td>\n<td>3.7608<\/td>\n<td>0.6769<\/td>\n<td>0.8336<\/td>\n<\/tr>\n<tr>\n<td>Davenport<\/td>\n<td>75<\/td>\n<td>3.5964<\/td>\n<td>0.6460<\/td>\n<td>0.8394<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>75<\/td>\n<td>3.6321<\/td>\n<td>0.6812<\/td>\n<td>0.8546<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>75<\/td>\n<td>3.8057<\/td>\n<td>0.7031<\/td>\n<td>0.8577<\/td>\n<\/tr>\n<tr>\n<td>Steamer<\/td>\n<td>75<\/td>\n<td>3.8116<\/td>\n<td>0.7054<\/td>\n<td>0.8592<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>75<\/td>\n<td>3.6513<\/td>\n<td>0.6995<\/td>\n<td>0.8711<\/td>\n<\/tr>\n<tr>\n<td>RotoGuru<\/td>\n<td>75<\/td>\n<td>3.8089<\/td>\n<td>0.7216<\/td>\n<td>0.8758<\/td>\n<\/tr>\n<tr>\n<td>y2013<\/td>\n<td>75<\/td>\n<td>3.3790<\/td>\n<td>0.7054<\/td>\n<td>0.9537<\/td>\n<\/tr>\n<tr>\n<td>Bayesball<\/td>\n<td>75<\/td>\n<td>3.9120<\/td>\n<td>0.8315<\/td>\n<td>0.9704<\/td>\n<\/tr>\n<\/table>\n<p>Now Fans ranks well above Steamer, which is near the bottom in this test. The lower errors here tend to come from systems that come closer to matching the actual league ERA. Even the naive 2013 data as a forecast now is no longer dead last by a wide margin, while CBS, Zips, and Fans are the best performing systems here. Comparing these two tables shows the impact of bias adjustment. What matters most for fantasy valuation is how players compare relative to each other, and not how well some system predicts the actual run environment players are in. So the first table is a better comparison of projections for fantasy purposes.<br \/>\nBoth these tables are &#8220;apples to apples&#8221;, comparing only those players that each system projected. And it&#8217;s a small set of overall better than average pitchers, a group which overall is easier to project than a deeper set of MLB pitchers.<br \/>\nBut of course if you&#8217;re in a fantasy league, it doesn&#8217;t help when a system doesn&#8217;t project someone you might care about. So this next table will use an ERA of 0.50 worse than the system&#8217;s league average for anybody not projected, and compare against a set of almost 700 pitchers:<\/p>\n<table>\n<tr>\n<th>Source<\/th>\n<th>MLB<\/th>\n<th> ERA<\/th>\n<th>StdDev<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<th>Missing<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>672<\/td>\n<td>3.7395<\/td>\n<td>1.4707<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>672<\/td>\n<td>4.0077<\/td>\n<td>0.5245<\/td>\n<td>0.8759<\/td>\n<td>1.4168<\/td>\n<td>215<\/td>\n<\/tr>\n<tr>\n<td>All Consensus<\/td>\n<td>672<\/td>\n<td>3.9776<\/td>\n<td>0.5452<\/td>\n<td>0.8773<\/td>\n<td>1.4251<\/td>\n<td>4<\/td>\n<\/tr>\n<tr>\n<td>Davenport<\/td>\n<td>672<\/td>\n<td>3.8822<\/td>\n<td>0.4985<\/td>\n<td>0.8856<\/td>\n<td>1.4270<\/td>\n<td>198<\/td>\n<\/tr>\n<tr>\n<td>Steamer<\/td>\n<td>672<\/td>\n<td>4.0087<\/td>\n<td>0.5731<\/td>\n<td>0.8883<\/td>\n<td>1.4280<\/td>\n<td>37<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>672<\/td>\n<td>3.9382<\/td>\n<td>0.6424<\/td>\n<td>0.9090<\/td>\n<td>1.4418<\/td>\n<td>31<\/td>\n<\/tr>\n<tr>\n<td>Bayesball<\/td>\n<td>672<\/td>\n<td>4.0215<\/td>\n<td>0.5219<\/td>\n<td>0.9252<\/td>\n<td>1.4427<\/td>\n<td>222<\/td>\n<\/tr>\n<tr>\n<td>Fans<\/td>\n<td>672<\/td>\n<td>3.7886<\/td>\n<td>0.4485<\/td>\n<td>0.9017<\/td>\n<td>1.4473<\/td>\n<td>465<\/td>\n<\/tr>\n<tr>\n<td>AggPro<\/td>\n<td>672<\/td>\n<td>4.0252<\/td>\n<td>0.3698<\/td>\n<td>0.9071<\/td>\n<td>1.4483<\/td>\n<td>557<\/td>\n<\/tr>\n<tr>\n<td>Razzball<\/td>\n<td>672<\/td>\n<td>3.8558<\/td>\n<td>0.3791<\/td>\n<td>0.9045<\/td>\n<td>1.4485<\/td>\n<td>524<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>672<\/td>\n<td>3.8888<\/td>\n<td>0.4810<\/td>\n<td>0.9093<\/td>\n<td>1.4488<\/td>\n<td>32<\/td>\n<\/tr>\n<tr>\n<td>RV Pre-Australia<\/td>\n<td>672<\/td>\n<td>3.9020<\/td>\n<td>0.4852<\/td>\n<td>0.9080<\/td>\n<td>1.4505<\/td>\n<td>42<\/td>\n<\/tr>\n<tr>\n<td>Larson<\/td>\n<td>672<\/td>\n<td>4.2586<\/td>\n<td>0.4348<\/td>\n<td>0.9232<\/td>\n<td>1.4540<\/td>\n<td>463<\/td>\n<\/tr>\n<tr>\n<td>ESPN<\/td>\n<td>672<\/td>\n<td>4.0331<\/td>\n<td>0.5404<\/td>\n<td>0.9056<\/td>\n<td>1.4563<\/td>\n<td>336<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>672<\/td>\n<td>3.7990<\/td>\n<td>0.4901<\/td>\n<td>0.9103<\/td>\n<td>1.4587<\/td>\n<td>136<\/td>\n<\/tr>\n<tr>\n<td>Zips<\/td>\n<td>672<\/td>\n<td>4.0703<\/td>\n<td>0.7450<\/td>\n<td>0.9207<\/td>\n<td>1.4640<\/td>\n<td>69<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>672<\/td>\n<td>4.0489<\/td>\n<td>0.6991<\/td>\n<td>0.9369<\/td>\n<td>1.4758<\/td>\n<td>94<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>672<\/td>\n<td>4.0622<\/td>\n<td>0.4768<\/td>\n<td>0.9404<\/td>\n<td>1.4799<\/td>\n<td>462<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>672<\/td>\n<td>3.8850<\/td>\n<td>0.6547<\/td>\n<td>0.9492<\/td>\n<td>1.4907<\/td>\n<td>143<\/td>\n<\/tr>\n<tr>\n<td>RotoGuru<\/td>\n<td>672<\/td>\n<td>4.3407<\/td>\n<td>0.6741<\/td>\n<td>0.9486<\/td>\n<td>1.5056<\/td>\n<td>150<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>672<\/td>\n<td>3.8623<\/td>\n<td>0.6814<\/td>\n<td>0.9398<\/td>\n<td>1.5199<\/td>\n<td>249<\/td>\n<\/tr>\n<tr>\n<td>y2013<\/td>\n<td>672<\/td>\n<td>3.8355<\/td>\n<td>1.7932<\/td>\n<td>1.2610<\/td>\n<td>2.2525<\/td>\n<td>169<\/td>\n<\/tr>\n<\/table>\n<p>Here systems get more credit for projecting more players, so long as those projections are better than the default 0.50 worse than average. This shakes up the order quite a bit. Now Steamer\/Razzball does best, followed by the Consensus, Clay Davenport, and Steamer. Will Larson, the winner of the test of fewer (and overall better) players drops to the middle of the pack, while CBS Sportsline and Zips now slip behind Marcel. Bayesball, which ranked worst in the earlier test, improves markedly. The overall errors increase quite a bit, as we&#8217;re now comparing projections for a much deeper set of players, many of whom have much less of a track record.<br \/>\nFinally, let&#8217;s take a look at WHIP. First, the &#8220;apples-to-apples&#8221; table comparing only the 75 pitchers projected by all systems:<\/p>\n<table>\n<tr>\n<th>Source<\/th>\n<th>Num<\/th>\n<th>Avg WHIP<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>75<\/td>\n<td>1.2036<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>75<\/td>\n<td>1.2077<\/td>\n<td>0.0874<\/td>\n<td>0.1175<\/td>\n<\/tr>\n<tr>\n<td>All Consensus<\/td>\n<td>75<\/td>\n<td>1.2360<\/td>\n<td>0.0899<\/td>\n<td>0.1205<\/td>\n<\/tr>\n<tr>\n<td>Zips<\/td>\n<td>75<\/td>\n<td>1.2214<\/td>\n<td>0.0943<\/td>\n<td>0.1229<\/td>\n<\/tr>\n<tr>\n<td>ESPN<\/td>\n<td>75<\/td>\n<td>1.2397<\/td>\n<td>0.0963<\/td>\n<td>0.1231<\/td>\n<\/tr>\n<tr>\n<td>Larson<\/td>\n<td>75<\/td>\n<td>1.2562<\/td>\n<td>0.0954<\/td>\n<td>0.1239<\/td>\n<\/tr>\n<tr>\n<td>Steamer<\/td>\n<td>75<\/td>\n<td>1.2616<\/td>\n<td>0.0963<\/td>\n<td>0.1246<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>75<\/td>\n<td>1.2238<\/td>\n<td>0.0970<\/td>\n<td>0.1254<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>75<\/td>\n<td>1.2608<\/td>\n<td>0.0996<\/td>\n<td>0.1271<\/td>\n<\/tr>\n<tr>\n<td>Fans<\/td>\n<td>75<\/td>\n<td>1.2030<\/td>\n<td>0.0993<\/td>\n<td>0.1271<\/td>\n<\/tr>\n<tr>\n<td>RotoGuru<\/td>\n<td>75<\/td>\n<td>1.2364<\/td>\n<td>0.0967<\/td>\n<td>0.1272<\/td>\n<\/tr>\n<tr>\n<td>Davenport<\/td>\n<td>75<\/td>\n<td>1.2487<\/td>\n<td>0.0981<\/td>\n<td>0.1284<\/td>\n<\/tr>\n<tr>\n<td>RV Pre-Australia<\/td>\n<td>75<\/td>\n<td>1.2276<\/td>\n<td>0.0977<\/td>\n<td>0.1286<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>75<\/td>\n<td>1.2275<\/td>\n<td>0.0979<\/td>\n<td>0.1288<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>75<\/td>\n<td>1.2456<\/td>\n<td>0.0978<\/td>\n<td>0.1303<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>75<\/td>\n<td>1.2090<\/td>\n<td>0.1069<\/td>\n<td>0.1335<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>75<\/td>\n<td>1.2464<\/td>\n<td>0.1047<\/td>\n<td>0.1336<\/td>\n<\/tr>\n<tr>\n<td>Razzball<\/td>\n<td>75<\/td>\n<td>1.2012<\/td>\n<td>0.1108<\/td>\n<td>0.1388<\/td>\n<\/tr>\n<tr>\n<td>AggPro<\/td>\n<td>75<\/td>\n<td>1.2412<\/td>\n<td>0.1170<\/td>\n<td>0.1415<\/td>\n<\/tr>\n<tr>\n<td>Bayesball<\/td>\n<td>75<\/td>\n<td>1.2794<\/td>\n<td>0.1134<\/td>\n<td>0.1431<\/td>\n<\/tr>\n<tr>\n<td>y2013<\/td>\n<td>75<\/td>\n<td>1.1895<\/td>\n<td>0.1097<\/td>\n<td>0.1469<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>75<\/td>\n<td>1.2603<\/td>\n<td>0.1203<\/td>\n<td>0.1495<\/td>\n<\/tr>\n<\/table>\n<p>This time the CBS Sportsline projections wind up with the lowest errors, followed by the consensus and Zips. Marcel actually beats most systems in this test. As a percentage of the projected average, the errors are smaller for WHIP than ERA, which makes sense since WHIP stabilizes more quickly, but the spread in errors of WHIP between systems is much wider, so systems vary more in their projections of WHIP than ERA for this sample of pitchers.<br \/>\nFinally, here&#8217;s the table using a WHIP of 0.10 worse than the projected league average for missing players:<\/p>\n<table>\n<tr>\n<th>Source<\/th>\n<th>MLB<\/th>\n<th>WHIP<\/th>\n<th>StdDev<\/th>\n<th>MAE<\/th>\n<th>RMSE<\/th>\n<th>Missing<\/th>\n<\/tr>\n<tr>\n<td>Actual<\/td>\n<td>672<\/td>\n<td>1.2746<\/td>\n<td>0.2381<\/td>\n<td>0.0000<\/td>\n<td>0.0000<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>Davenport<\/td>\n<td>672<\/td>\n<td>1.3088<\/td>\n<td>0.0951<\/td>\n<td>0.1457<\/td>\n<td>0.2253<\/td>\n<td>198<\/td>\n<\/tr>\n<tr>\n<td>Steamer\/Razzball<\/td>\n<td>672<\/td>\n<td>1.3155<\/td>\n<td>0.0955<\/td>\n<td>0.1472<\/td>\n<td>0.2256<\/td>\n<td>215<\/td>\n<\/tr>\n<tr>\n<td>Steamer<\/td>\n<td>672<\/td>\n<td>1.3148<\/td>\n<td>0.1027<\/td>\n<td>0.1480<\/td>\n<td>0.2264<\/td>\n<td>37<\/td>\n<\/tr>\n<tr>\n<td>Fans<\/td>\n<td>672<\/td>\n<td>1.2716<\/td>\n<td>0.0829<\/td>\n<td>0.1462<\/td>\n<td>0.2281<\/td>\n<td>465<\/td>\n<\/tr>\n<tr>\n<td>AggPro<\/td>\n<td>672<\/td>\n<td>1.3036<\/td>\n<td>0.0693<\/td>\n<td>0.1478<\/td>\n<td>0.2286<\/td>\n<td>557<\/td>\n<\/tr>\n<tr>\n<td>ESPN<\/td>\n<td>672<\/td>\n<td>1.3082<\/td>\n<td>0.1080<\/td>\n<td>0.1467<\/td>\n<td>0.2290<\/td>\n<td>336<\/td>\n<\/tr>\n<tr>\n<td>Razzball<\/td>\n<td>672<\/td>\n<td>1.2770<\/td>\n<td>0.0846<\/td>\n<td>0.1495<\/td>\n<td>0.2304<\/td>\n<td>524<\/td>\n<\/tr>\n<tr>\n<td>Marcel<\/td>\n<td>672<\/td>\n<td>1.2838<\/td>\n<td>0.0888<\/td>\n<td>0.1504<\/td>\n<td>0.2311<\/td>\n<td>136<\/td>\n<\/tr>\n<tr>\n<td>Bayesball<\/td>\n<td>672<\/td>\n<td>1.3262<\/td>\n<td>0.0928<\/td>\n<td>0.1537<\/td>\n<td>0.2317<\/td>\n<td>222<\/td>\n<\/tr>\n<tr>\n<td>All Consensus<\/td>\n<td>672<\/td>\n<td>1.3163<\/td>\n<td>0.1202<\/td>\n<td>0.1487<\/td>\n<td>0.2318<\/td>\n<td>4<\/td>\n<\/tr>\n<tr>\n<td>Oliver<\/td>\n<td>672<\/td>\n<td>1.3324<\/td>\n<td>0.1364<\/td>\n<td>0.1507<\/td>\n<td>0.2330<\/td>\n<td>31<\/td>\n<\/tr>\n<tr>\n<td>Larson<\/td>\n<td>672<\/td>\n<td>1.3506<\/td>\n<td>0.0857<\/td>\n<td>0.1520<\/td>\n<td>0.2332<\/td>\n<td>463<\/td>\n<\/tr>\n<tr>\n<td>RotoValue<\/td>\n<td>672<\/td>\n<td>1.2871<\/td>\n<td>0.0968<\/td>\n<td>0.1525<\/td>\n<td>0.2335<\/td>\n<td>32<\/td>\n<\/tr>\n<tr>\n<td>RV Pre-Australia<\/td>\n<td>672<\/td>\n<td>1.2891<\/td>\n<td>0.0974<\/td>\n<td>0.1523<\/td>\n<td>0.2336<\/td>\n<td>42<\/td>\n<\/tr>\n<tr>\n<td>Zips<\/td>\n<td>672<\/td>\n<td>1.3289<\/td>\n<td>0.1501<\/td>\n<td>0.1562<\/td>\n<td>0.2379<\/td>\n<td>69<\/td>\n<\/tr>\n<tr>\n<td>CBS<\/td>\n<td>672<\/td>\n<td>1.3189<\/td>\n<td>0.0962<\/td>\n<td>0.1574<\/td>\n<td>0.2380<\/td>\n<td>462<\/td>\n<\/tr>\n<tr>\n<td>RotoGuru<\/td>\n<td>672<\/td>\n<td>1.3139<\/td>\n<td>0.1231<\/td>\n<td>0.1576<\/td>\n<td>0.2405<\/td>\n<td>150<\/td>\n<\/tr>\n<tr>\n<td>MORPS<\/td>\n<td>672<\/td>\n<td>1.3036<\/td>\n<td>0.1243<\/td>\n<td>0.1583<\/td>\n<td>0.2406<\/td>\n<td>143<\/td>\n<\/tr>\n<tr>\n<td>CAIRO<\/td>\n<td>672<\/td>\n<td>1.3809<\/td>\n<td>0.1890<\/td>\n<td>0.1805<\/td>\n<td>0.2593<\/td>\n<td>94<\/td>\n<\/tr>\n<tr>\n<td>RotoChamp<\/td>\n<td>672<\/td>\n<td>1.2640<\/td>\n<td>0.1981<\/td>\n<td>0.1757<\/td>\n<td>0.2892<\/td>\n<td>249<\/td>\n<\/tr>\n<tr>\n<td>y2013<\/td>\n<td>672<\/td>\n<td>1.2950<\/td>\n<td>0.2538<\/td>\n<td>0.2035<\/td>\n<td>0.3252<\/td>\n<td>169<\/td>\n<\/tr>\n<\/table>\n<p>In this test Clay Davenport&#8217;s model edges out Steamer\/Razzball and Steamer for lowest RMSE, with Fangraphs Fans, AggPro, and ESPN close behind. Marcel beats more than half the models, as in the test of the 75 players projected by all, but now for the first time in these tests it also beats out the consensus, which is usually among the best performing systems. But here, the wider spread among systems in errors might work against a crowd-sourcing approach which usually does quite well with other stats.<br \/>\nProjections systems vary much more on WHIP than they do on ERA (or wOBA), but in general they all perform much better than 2013 data. Yet while for wOBA, most systems usually beat the benchmark of Tom Tango&#8217;s Marcel system, for these pitching stats, Marcel is still quite good. Projecting pitching is harder than projecting hitting. Fantasy veterans know that already, of course, but these numbers support that conclusion also.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yesterday I ran comparisons of several projections systems for an all-inclusive batting statistic, wOBA. Today I&#8217;m running the same tests, computing\u00a0root mean square error\u00a0(RMSE) and\u00a0mean absolute error\u00a0(MAE), for two commonly used fantasy statistics, ERA and WHIP. These tests are bias-adjusted, so what matters is a player&#8217;s\u00a0ERA or WHIP relative to the overall average of that&hellip; <a class=\"more-link\" href=\"https:\/\/blog.rotovalue.com\/index.php\/2015\/02\/14\/comparing-2014-projections-era-and-whip\/\">Continue reading <span class=\"screen-reader-text\">Comparing 2014 Projections &#8211; ERA and WHIP<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,11,14],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts\/1342"}],"collection":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/comments?post=1342"}],"version-history":[{"count":0,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/posts\/1342\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/media?parent=1342"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/categories?post=1342"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.rotovalue.com\/index.php\/wp-json\/wp\/v2\/tags?post=1342"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}