Carlos Boozer: Another Example of How Wins Produced Fails

Introduction

In an earlier article I tested the robustness of the Wins Produced model by looking at how the Raptors did when Calderon was playing in a game versus how they did when he was not playing. Since he regularly got injured, there were enough games for a small study. Why Wins Produced, and what is it? It's a metric that's intended to capture how many wins a player adds to a team by assigning box score stats particular values, and its creators and backers purport the method to be exceedingly accurate, regularly disparage other basketball analysts and publish daily articles that amount to little more than posting their stats. If you want to know how the stat was created and what goes into it, I've discussed that here. The website is leaning toward Armond White territory in terms of its commentary and insular nature. That's why I'm going to focus more on looking at their model, pointing out its weaknesses and bringing attention to a popular basketball site that is advertising something it is not.

I don't think Wins Produced is a terrible metric. There are flaws with all basketball metrics, but no one is realistically expecting them to be perfect; acknowledging those flaws and being your own toughest critic is an important part of analysis. The team at Wins Produced does none of that, and makes decisions based only on its numbers. When mainstream NBA writers and fans criticize the stats movement, it's for exactly what they are doing. In a recent article they argued Jared Dudley and Jason Kidd should be invited to the all-star game. Anyone reasonable would look at that, take a step back and use his or her judgement, not blindly follow a basic model to the ends of the earth. Making decisions purely on numbers without any human oversight is the sort of thing that leads to a financial catastrophe. I'm not bringing up that point because I believe basketball stats have as much weight as the stock market, but as a reminder these arguments, these battles -- human judgement versus computerized control -- will be the theme of our next century in humanity. Explaining the limitations of analytical models is good practice for when it actually matters.

The High Production, Low Intangibles Power Forward

There's a specific archetype that's cropped up the past few years, highlighted by guys like Hollinger who have found their per minute production insatiable. One of the best examples is Carl Landry, a power forward too small for the position who was passed over the draft because he didn't fit the typical mold. However, he was extremely productive in college, pulling down rebounds at high rates and scoring well at a high percentage. He carried this over to the NBA as a draft day steal, collecting per minute numbers with such velocity that in his rookie year Wins Produced had only a handful of players with a higher Wins/48 minutes, like Chris Paul and Kevin Garnett, and ahead of someone like LeBron James. So was Carl Landry a draft day mistake? Well, yes, but not precisely: since he's short for his position, he struggles on defense, and this doesn't show up in any box score metrics like PER or Wins Produced because he's a decent rebounder. He also feasts on put-backs and offensive rebounds, scavenging to bring up his numbers in a cheap sense. Some players have to create a play at the top of the key, throwing up a jumper near the buzzer; this responsibility is lost on someone like Landry, who just waits for a miss near the basket to bring his field-goal percentage up.

This leads us to Carlos Boozer. He's a 20-10 frontcourt player with a nice low post game, a jumper, money from the line and a strong build. He's not as small as Landry, but he's another high producing power forward who doesn't defend and leaves the basket open to intruders, something that's sometimes hard to quantify. There's a reason why fans of his teams hold him in contempt. Even with a strong defensive culture, some fans are pushing for him to receive the amnesty because of his blown rotations and short-armed rim protection; they'd want to pay him not to play for the team. Why is this? Aren't the fans aware of his nice tidy stats and how he posted a 0.151 Wins Produced per 48 minutes number last season as a well above average player, adding 6 wins to the team in limited minutes? The fans are seeing something else.

Boozer's Best Years

Boozer's career is wildly uneven due to injuries and a wavering impact on the boards with shooting efficiency numbers that expand and contract like Florida real estate. For that reason, I have focused on his best years according to Wins Produced: 2003-04, 2005-06, 2006-07 and 2007-08. That way I'm testing the player Wins Produced thinks is a near-elite player because in other seasons he's at or near below average. I've posted a basic summary of those seasons below, and then I'll get into how his team performed when he was on and off the court.

Season	Points/game	Rebounds/game	Games played	Wins Produced	Team record
2003-04	15.5	11.4	75	12.7	35-47
2005-06	16.3	8.6	33	4	41-41
2006-07	20.9	11.7	74	12.7	51-31
2009-10	19.5	11.2	78	11.4	53-29

2004 was his sophomore year on the Cavaliers, before he bolted for Utah and left LeBron with such flotsam as Darius Miles. He missed the first half of the 2005-06 season recovering from a hamstring injury, but still posted sterling numbers after he returned. In the next season, Boozer received an all-star berth but missed the game due to a hairline fracture in his leg, yet barely missed any games. During his last season in Utah, he posted a near 20-10, and then signed with the Bulls in the following summer. It's over this span that Carlos Boozer has become the player people think of: he's frustrating, injury-plagued, puts up good numbers, stabbed Cleveland in the back on a verbal deal and never played defense.

Boozer's Missed Game Analysis

In the chart below, I've included a team of statistics during the games in which Boozer played and when he didn't. Except for the record and win percentage, the total rows are weighted by the number of missed games. What this means is because Boozer missed far more games in 2005-06, the total rows are more influenced by that season. Margin of victory is pretty self-explanatory, and the offensive and defensive ratings are a measure of points scored and allowed per 100 possessions from basketball-reference. Also from basketball reference is SRS, which is the simple rating system of margin of victory adjusted for home-court advantage and strength of schedule. Another note is that the win record with Boozer versus without is distorted by when most of Boozer's missed games occurred: 2006 when they were terrible with or without him, not 2010 when they were one of the best teams in the league.

Season		Record	Win%	Margin ofvictory	Off. rating	Def. rating	SRS	Wins Produced
Total	With Boozer	146-114	0.562	-1.5	105.7	107.3	-1.76	40.8
	Without Boozer	34-34	0.500	-2.1	104.6	107.3	-1.69
2003-04	With Boozer	34-41	0.453	-2.2	101.8	103.7	-2.68	12.7
	Without Boozer	1-6	0.143	-7.7	97.1	110.1	-7.22
2005-06	With Boozer	17-16	0.515	-2.3	105.9	108.3	-2.58	4
	Without Boozer	24-25	0.490	-2.9	102.9	106.1	-2.43
2006-07	With Boozer	45-29	0.608	0.2	105.7	105.5	0.52	12.7
	Without Boozer	6-2	0.750	4.6	116.9	111.5	4.59
2009-10	With Boozer	50-28	0.641	5.4	110.6	104.8	5.37	11.4
	Without Boozer	3-1	0.750	4.5	113.6	108.3	4.52

The biggest assumption made is the WP from his replacements when Boozer misses a game. I assumed this was 0.100 per 48 minutes, which is average and a pretty conservative estimate. Yes, valuable backups exist and can pick up more playing time, but the backup's backup is who you actually worry about; they're usually terrible. For the actual statistical tests, I'm using the weighted numbers over 68 games that Boozer missed. This simplifies the analysis.

With Boozer, his teams based on their SRS should have won 44.8% of every game. If you contend predicting win percentage based on point differential is problematic, then I'll counter that the Wins Produced sages hold that method in high esteem and in fact the entire method behind the statistic uses point differential as a fundamental underpinning. In the games that Boozer missed, with Boozer this would have meant by SRS they would have won 30.1 games. Wins Produced says the team would have lost 4.6 wins with Boozer gone. In reality, when Boozer missed games their SRS improved and their projected win total increased to 30.2. This means that in only 68 games Wins Produced was incorrect by 4.7 wins. If you want the actual win differences instead of predictions based on point differential, I included that too and a gap still exists. I used a basic t-test and a standard deviation of wins produced I found (which is actually for 48 minutes of play, so this is another safe conservative measure.) A p-value that small means there's a tiny chance based on the given information Wins Produced was accurate in assessing Boozer's value.

	Actual wins without Boozer	Wins Produced predicted wins	Standard deviation of wins produced	t-test p-value
Point margin wins	30.2	25.6	0.24	1.98E-29
Actual wins	34	31.2	0.24	1.78E-17

Conclusion

I think a WP-follower will immediately point to Paul Millsap as an explanation for the Jazz outplaying expectations with Boozer injured, but I don't think that is satisfactory. First of all, during his first year he was in Cleveland, although critics would point out the team got noticeably worse with him off the court, but his backups there were truly horrific (Michael Stewart?) I will say, however, that Boozer's problems, defensive effort and athleticism, got worse with age, and there's a possibility young Boozer was actually as valuable as his box score stats suggested. That's a good argument, but most of the missed games were during his Utah tenure and nearly three-quarters of his missed games occurred in 2005-06, so that's practically speaking nearly all that matters. His frontcourt backups that season? Okur with his WP/48 of 0.121, Jason Collins with 0.055, Kris Humphries with -0.016 and Ostertag with 0.121. Boozer being out meant significantly more time for Okur and Humphries since they both played as power forwards that year, and to some extent the centers Collins and Ostertag. Notice that the average WP of 0.100 is an optimistic average of his backups. (If you aren't convinced Collins would increase his playing time with Boozer out, look at these lineup combinations.) Millsap's rookie year was the 2007 season, and he was indeed productive but Boozer had a higher WP/48 number and the highest on the team; this is incongruent with how much better Utah was when he missed games. And in 2010 Boozer was still indeed "better" than Millsap, so the backup effect isn't enough to explain the disparity, especially since Millsap was only in the league for 12 of those games over the analysis period.

Another point one can make is that the analysis misses too many factors like who else was missing games, tanking and any other miscellaneous factors. And you know what? I agree. This is a good first step, but there's more to this in the Boozer problem. His play was too inconsistent and he didn't miss games at a nice even rate. For best representation, he should have missed half his games, or at least an constant number, each season, and for that I will again point to my Jose Calderon article because he consistently missed games. I will contend, however, that it's not looking well for Wins Produced. 

The next step in the analysis is to look closely at lineup combinations to see how the team responds when Boozer's on the court and when he's off the court. This is a play-by-play analysis, instead of game-by-game whenever he misses a game. I looked at missed games because it was clear to even the least math-savvy individual. 

Analysis never ends. At the game level during his best seasons, Wins Produced is incorrectly stating his value, but there's more work to be done.