Monday, November 21, 2011

Final Thoughts on Sabermetrics

"I thought you were gonna ask me about JD Drew having the second-highest OPS of all AL outfielders."

With that, it was over for me.  Over the course of our five years at the helm of How Youz Doin Baseball, Pat and I (and all of you, I'm sure) have had a complicated relationship with sabermetrics, their value, and how important they are when trying to put together, evaluate, and manage a baseball team.  I think somewhere around 2009, when Theo Epstein said the line above to justify another painfully mediocre season by JD Drew that you had to start calling everything into question.  I'm gonna try to boil an intricate argument into a few bullet points.

1.  There is value in sabermetrics and in the stuff contained within the book Moneyball.  I still fully support the Moneyball philosophy of preferring college players in the draft because their statistics are more reliable than high school statistics because you know how good the competition is in college whereas high school competition has many other variables.  It also provides value within managing games:  At times, when it's a toss-up, go with the numbers.  Proving the indisputable correlation between getting on base and scoring runs (which is intuitive without looking at the numbers) definitely enabled managers to abandon some "cute" tactics that really didn't provide much value.  It's also very interesting to see what happens on the aggregate level. 

I also do believe strongly in some of the pitching metrics.  FIP is a great statistic in particular.  The offensive equivalent, BABIP (I know they're not perfect mirror images, go back under your bridge), is not really as valuable in my book, but I am impressed with the intuitive nature of FIP and the important stories it tells.

2.  The problem I have with a lot of statistics, including BABIP many aggregate stats (including Pythagorean winning percentage, and virtually every defensive metric ever invented, is that they force you into a ceteris paribus (all else being equal) assumption.  They make you do this stuff in economics classes and science classes alike.  The experiments you read about in a textbook are performed in a vacuum, and UZR150 doesn't take anything into consideration, such as where a guy is positioned, what the situational strategy is, whether there's a shift on a lefty hitter, or anything like that.  We're told in economics class or science class that in real life a feather and a bowling ball won't fall off the Leaning Tower of Pisa at the same speed or that markets are never actually 100% perfectly competitive.  In sabermetrics class, we are not told that a lot of these statistics are only scientific in a vacuum.

2.5.  As I wrote in a previous post, OBP was the last frontier of labor market inefficiencies in baseball.  Trying to find it in flawed vacuum stats like the defensive ones results in failure.  Several Oakland teams, the 2010 Mariners, the 2010 Red Sox.  Period.  Run prevention does not work.

3.  Closely related, these statistics come from a very high volume of data played out over a 162-game season (lest you summons the Sample Size Police).  However, baseball is a situational game.  It is foolish to play the same style of station-to-station baseball at all situations throughout the game.  I always cite the same situation of Drew drawing a walk with a runner on third, one out, and Varitek on deck.  On the aggregate, the walk makes sense.  In this very specific situation, it does not.  While formulating this post in my head, I was consdering the shunning of the bunt.  But if you have Mark Reynolds up, say he's hitting .250, hits a lot of home runs, strikes out 1/3 of the time, and you have a guy on second, no outs, and a fly ball hitter at the plate.  You gotta think about bunting him.  You're taking the bat out of his hands, but he might do the most inefficient thing you could possibly do in baseball, and that's strike out.

4.  Some of the math is fuzzy.  I have read more books than I care to admit about sabermetrics and the value of a player in a situation.  There have been interesting situations about the value of a win, the marginal value of a win, and so forth.  Vince Gennaro and JC Bradbury especially write some interesting stuff about it.  But (as much as it's a punch line regarding certain terrible catchers with extra letters on their jerseys) intangibles do count, both on the field (sorry, I believe in clutch) and to fans.  Even if Jeter hits .260 in his final season in New York, losing him would subtract value from the Yankee franchise.  And this is something that cannot be calculated in a lab.

5.  Closely related to point #4, it's a human game.  We're not playing on the Strat-O-Matic machine, and we're not even playing Baseball Mogul here.  Sometimes you need the scouts "selling jeans" (a Moneyball term) to see that despite his high OBP, a player does not give an F about playing the game.  Either that or you look at the way that player is viewed by former managers, former players, and each fan base he's ever played for (and one he spurned, of course).  Sometimes you need someone selling jeans to tell you that a reliever vomits on the mound during high-leverage situations.  Sometimes you need to look at traditional statistics, such as RBIs or wins, to see the way a player actually goes about playing the game.  As much as I hate 46, he was aggressive in all the right times (except for one particular Saturday afternoon), and his RBI total reflected that.  Spreadsheets do not indicate whether a player has balls.

In closing, I treat hard-core sabermetricians like sky-is-falling global warming advocates.  Just because they have a high volume of data they've never had available before, they cannot accept the fact that some of the data is exaggerated, imperfect, or just plain irrelevant.  To justify their existence on Planet Earth, they have to exaggerate the importance of the data they have and make a big story about it.  As I said earlier, sabermetric statistics have their place in baseball, and when used responsibly they give a club an edge over a team that completely disregards statistics.  They also help explain a lot about the nature of the game we watch.  However, as Pat and I (and long-lost commenter Craig) have proven over the past five years, if you dig deep enough, you can use a statistic to support any argument. 

9 comments:

Anonymous said...

DV

One of the great points about JD Drew having the second highest OPS of AL outfielders in 2009 is that the Sox also had the guy with THE highest OPS of AL outfielders in 2009. Jason Bay. And they didn't re-sign him. He was younger, better, tougher, and gave a shit about baseball. Clearly, that comment by Theo Epstein was not one rooted in a deep belief in JD Drew and the value of OPS, but rather as an attempt to justify signing a guy who never drove in 70 runs in a season for the Sox.

Also, I think there is something to run prevention. Unfortunately, it has nothing to do with the SABR communities' definition of run prevention. If you go out and put together a great pitching staff you won't give up that many runs. And you'll win games. Just don't think that signing Mike Cameron will have the same impact.

--the Gunn

TimC said...

Really good wrap up DV. It saddens me to see some of the flawed stuff seep into usage in other sports- basketball first, more recently football and soccer. Sabermetrics is one of the many tools that should be used by GMs, managers, and (perhaps most significantly) fans to assess what has happened on the field.

I think the biggest failure of the advanced metrics, though, is that it has distorted what Moneyball is really about. Anyone who has not read the book assumes it is some kind of treatise on the use of multiple linear regression and how that can solve various baseball situations. Instead, like most Michael Lewis books, it is about the dangers of a lemming mentality and how thinking around that can lead to success- or failure. I'm sure the movie covered that to its fullest extent.

TimC

the gm at work said...

Gunn,

Jason Bay had a bad, what was it, knee? JD Drew was very durable - during the very same interview, Epstein said "he's played a lot, though. He's played a lot since he's been here." Obviously, I agree with you.

If we're talking about improving a team one run per game due to elite defense, fine. I feel like a lot of these "run prevention" upgrades were worth about one run every forty games. Four runs per season! Whee! The parallels to global warming statistics, without turning this into a political battle, are staggering.

Tim,

If you think Moneyball is about on-base percentage, you read the Cliffs Notes version. It's about what you said, overcoming odds. The way the A's did it was by exploiting the last great market inefficiency. At some point, maybe when they hit the Pacific Ocean, people realized there wasn't any more to discover. That's where we are now. Perhaps UZR is Hawaii or Guam.

Anonymous said...

Great stuff all around here, and I'm glad we now have this post summarizing what has been the biggest topic on this blog outside of the Yankees and Red Sox themselves. When you consider how sabermetrics factored into most of the debates we had here, they were involved in almost every conversation we had regarding baseball analytsis, whether directly or indirectly.

I think the biggest thing for me here is that you want the most information available. That's what really drove sabermetrics to the top in the first place - this entire wealth of information was being ignored. Interestingly enough, where sabermetrics went wrong was that, at least in some circles, it became a positional thing. You couldn't be a "stats" guy and a "subjective" guy at the same time. You were either on the sabermetrics bandwagon, turning your nose up to certain "traditional" approaches, or you weren't (and I ws definitely a semi-part of that movement in the early stages of this website). In doing so, however, they were doing exactly what had preciously been done to the - ignoring a huge wealth of knowledge by way of scouting/subjective/intangible analysis.

DV crushes it with this point in his post. It's great that we are now, through statistical analysis, identifying the flaws of a 6'5"/240 lb. Guy who can hit the ball 450 and has all the physical tools but can't crack a .330 OBP. But that doesn't mean we also shouldn't identify the flaws of the .380 OBP guy who can't hit major league sliders which means he may be doomed to walking, hitting fastballs, or doing nothing at all. Although statistical analysis aids us in this analysis, we still get this type of information promarily through scouting. And that's to say nothing of the many, even more subjective things that can in fact only be found this way (DV cites a few grat examples in this post).

TimC brings up a great point regarding Moneyball, and that it's not, and never was, about any particular set of stats (and we have touched on this here before). Rather it was and continues to be, like a lot of Michael Lewis' - who might be my favorite author - work, about value, and a search for value, and what's overvalued and undervalued and how people can profit from the relationship between thos two things.

I don't know exactly where the value is in baseball at this very moment, but it certainly no longer is strictly tied to sabermetrics. If anything, they may now be overvalued and value searching teams may actually be looking away from sabermetrics as a result, at least towards a more balanced approach, using them as just one part of the equation. I think balance is really what we've been pushing here, and that's what this is all about. You want the most information available in any analysis. For me personally, it's right down to what that person eats for breakfast if there are wins, losses, and millions of dollars on the line. I can't get that info in a statistical formula. I need both types of information, subjective and statistical.

PF

Ross Kaplan said...

I'm going to be the crusty old commenter and just say how much I hate Sabermetrics. Simply for me, there are just too many acronyms and formulas to remember. I know what OBP is, I'm pretty sure I know what OPS and BABIP are, the rest may as well be in Russian. I do appreciate seeing some applicable Sabermetric stat flash on the screen during the game, but most of the time when I'm not watching a game I'm really not thinking of a player's UZR, VORP, WAR, PECOTA and I really don't want to have to think of those things. I just want to watch the damn game without have Sabermetrics shoved down my throat.

the gm at work said...

Pat,

Thanks for the comment, and I'm glad I took it out of spam quarantine. The fact that there was going to be a hybrid spreadsheet/baseball guy taking over the Red Sox baseball operations was pretty exciting. Unfortunately, he has been stripped of all of his power in the interest of exciting postgame press conferences on NESN. Thanks, Larry.

I feel you about how knowledge is power, but some of the stuff that sabermetricians are coming out with is minutia.

Tank,

It's too bad you weren't in Boston in September. While the team was crashing and burning, NESN was touting Bill James's new stat, which was something like Heat Index or Temperature Gauge or something like that. It crunched a certain cocktail of numbers to indicate how hot a hitter was. All you need to do to see how hot a hitter is, is watch a couple of games or look at a box score. We don't need 72 degrees or 96 degrees or 54 degrees. It's people coming up with numbers just for the fun of it. Completely silly.

The only number that matters today is 2, as in 46 came in second in MVP voting. This is especially good because if he had won I would have to throw another post up featuring the image of him getting thrown out at third against Tampa.

ZWeiss said...

Awesome post. Just one last thing about JD Drew, and Theo's infamous comment: What was Theo 'supposed' to say? He obviously wasn't going to come out and say "Yeah, this guy sucks." He had to stand behind his decision and, more importantly, his player, so he pulled out probably the only somewhat positive statistic to JD Drew's name. It's not like JD really gives a shit about baseball, but imagine how little he'd care if he had the GM telling the media how freaking terrible he is.

the gm at work said...

ZWeiss,

Regarding what Theo was "supposed to say," I encourage you to listen to the whole interview. It's about a 40-minute commitment, but it's actually quite good from front to back (not to be confused with "front AND back," who finished a solid not-first in MVP voting today). It was only supposed to be a 30-minute interview, Felger and Mazz were pretty much wrapping it up, and instead of saying goodbye like everyone expected, Theo drops the line perched at the top of this post. The interviewers did not bring it up; the interviewee did because, in fact, he knew they weren't going to ask him about JD Drew having the second-highest OPS of all AL outfielders.

I'm pretty sure Drew didn't get motivated or anti-motivated by any kind of negative press. He wasn't really moved by Philadelphia fans throwing batteries at him for God's sake. He was one of maybe two or three players Francona publicly criticized over the course of eight seasons as manager (and with the F-word at that!). But that didn't really change things.

Rocci said...

Why you gotta hate on the environment?