The best example of how the metric fails is with Ryan Anderson. Traded from the Magic, who with Howard were a perennial defensive team, to the Hornets, still reeling from tanking and poor decisions, what changed was his scenery, not his defensive skill. Obviously, motivation is important with defense, as is coaching, but his defensive rating tracks closely to this team's rating. 2012 was his breakout season, yet his defensive rating plummeted from 19th in the league, or nearly a 95th percentile, to almost exactly average. The table below has the full details where percentile is based on players that season with at least 500 minutes.
Ryan Anderson's defensive rating (win shares)
|
Season
|
Defensive rating
|
Defensive rating
percentile
|
Team rating
|
Team ranking
|
|
2011
|
101
|
94.4
|
101.8
|
3
|
|
2012
|
105
|
51.1
|
104.1
|
12
|
|
2013
|
112
|
7.0
|
110.3
|
28
|
When people use defensive win shares, or win shares in general, they may not fully understand how it's calculated. This is the clearest way to illustrate that.
Need another example? Omer Asik was traded from the defensively dominant Chicago Bulls to the Houston Rockets, where he has to adjust for the mistakes from James Harden and a roster of rookies. Few people watching Asik this year would say he has dropped off significantly in defensive intensity or skill. However, the previous year his rating was 2nd and before that, his rookie season, he was 3rd. This season? 68th. The Rockets are a league average defensive team because of him; when he's off the court they're terrifyingly bad, yet win shares only sees a player who's a great defensive rebounder with some blocked shots on a mediocre defensive team.
|
Season
|
Defensive rating
|
Defensive rating
percentile
|
Team rating
|
Team ranking
|
|
2011
|
97
|
99.1
|
100.3
|
2
|
|
2012
|
92
|
99.4
|
98.3
|
2
|
|
2013
|
103
|
80.0
|
106.1
|
16
|
You can see this with players traded midseason too: Tayshaun Prince went from a defensive rating of 111 to 103, effectively going from roughly one of the worst defenders in the league, roughly 10th percentile, to significantly above average. (And no, it's not effort: win shares only sees box score stats, as his defensive rebounding leveled off but his steals and blocks went up, but the major culprit to the change was Memphis' suffocating defense compared to Detroit's mess.)
Need a quick way to discredit the stat to someone? In 2013, DeJuan Blair was 7th in the league in defensive rating.
I agree with your criticism, but one thing I like doing is intra-team comparisons with DRtg. Omer Asik may have dropped significantly, but he remained the best on his team.
ReplyDeleteHe's remained the leader in DRtg because he's an incredible defensive rebounder and picks up many steals and blocks. It's not purely because he's a better defender.
DeleteHe only gets 0.6 steals and 1.1 blocks a game. Neither one is particularly high. Being a good defensive rebounder is an important part of defense.
DeleteEd Davis and Speights have better defensive ratings than Tony Allen and Marc Gasol (newly minted DPOTY) because they pick up a lot of steals and blocks. No one in his or her right mind would say davis and Speights are better defensively, but that's what Win Shares says.
DeleteLast year per 36 minutes, Asik grabbed 1.1 steals and blocked 2.5 blocks, which helped put him above Bulls players. This year his defensive rebounds put him over the top because there's no other competition, really. No other Houston players grab a lot of rebounds and blocks and steals. However, Thomas Robinson and Terrence Jones have better defensive ratings, albeit in limited minutes, and this is solely because they grab lots of steals and blocks.
I'd like to reiterate this: defensive rating is just the team's defensive rating along with steals, blocks, and defensive rebounds. Looking at the players with the best rating on a team will not determine the best defender; it will determine who picks up the most steals, blocks, and rebounds.
I hope that helps.
From your own post: "If a person is against TS% as a stat, he or she will probably use the example of Anderson and Paul to say the stat is broken"
DeleteI definitely agree with you there, and so I don't think you can use these examples to dismiss the use of defensive win shares, especially on an intra-team level. Why can't Ed Davis be a better defender than Marc Gasol? Simply because a bunch of journalists voted him DPOY? Those same journalists thought Carmelo Anthony was the third most valuable player in the league... And similarly, why can't Thomas Robinson be a better defender than Omer Asik? We haven't seen much of him and he plays against back ups, so a comparison in this case isn't conclusive anyway.
A much better way to discredit the statistic would be to do a multiple regression of team defensive rating to rebounds, fouls, steals, and blocks, to see how much of defense this really explains. Then you can do a similar analysis to see how much of DRtg is explained by these stats. If the stats explain very little of defense, and if DRtg is largely a combination of these stats, then we can say with certainty that the stat is useless. It may even be the case that these stats explain a lot of defensive efficiency, and if that is the case, then DRtg may indeed be a good metric. But your current anecdotal analysis is not conclusive at all.
Ed Davis v. Marc Gasol:
DeleteNo, it's not about the same journalists who thought Carmelo was third. The most respected NBA-stat guys agreed he had immense defensive value.
Thomas Robinson v. Asik:
Okay, this one is silly. Thomas Robinson is a slightly undersized power forward and a young rookie. But Asik is clearly a very valuable defender. I'm fine with being a skeptic and looking at different stats to change the status quo on some issues, but this one is bogus. This has nothing to do with terrible mainstream issues on defense. Asik is clearly a much, much better defender than Robinson, and it isn't even close. This isn't a stubborn position where logical failings exist. Ask any scout, their coach, the players on the team, Thomas Robinson, Hollinger, etc. If a defensive statistic says Robinson is a better than Asik, it is wrong and has something wrong with it.
This isn't a comprehensive analysis, no. It was meant as a simple way of showing how Drtg works and what its flaws are. Obviously, only relying on rebounds/blocks/steals for defense is problematic, and I found a few good examples to illustrate this point.
This seems to go against exactly what you espoused in your contextualizing basketball stats post. You are simply saying the stat is wrong because it disagrees with what the "experts" say. That sounds like an argument from authority. Ultimately, we have to use some benchmark, or we'll fall into the same trap as the those same scouts coaches, players, etc, who believe Melo is one of the best offensive players in the game, mostly because of his "silky smooth jumper". Just because these people agree on something doesn't make it true.
DeletexRAPM ranks Josh Smith above Kevin Durant, Manu Ginobili above James Harden, and Chris Andersen above Joakim Noah. I disagree with these assessments and I think most "experts" would as well. RAPM had similar "atrocities" in its rankings when it was around. But I wouldn't use these examples as my reasons for why either metric is flawed.
I agree that only relying on rebounds/blocks/steals for defense is problematic, but how do we know that this isn't a big part of the story? And even if it isn't (very likely), how do we know that players like Marc Gasol and Omer Asik are great defenders? Again, we ultimately need an objective measure. DRtg may not be that measure, and it is clearly heavily influenced by teammates as shown in your post. It's not perfect, but it seems passable for intra-team analysis. Defensive xRAPM is another decent alternative, but it has its own set of issues and that's a different argument altogether.
One of the central tenets of the early metrics in basketball was the "laugh test." If the results were clearly completely crazy, then you need to retool the metric. This is not ignorance or stupidity. It is a basic law that guided much of the early basketball analysis, and all the great analysts used it. It is also a part of statistics in general. You have to judge your results.
DeleteSorry, but if you can't see a problem with a metric that says Thomas Robinson is better defensively than Asik than there is nothing more I can say. I looked for the most obvious cases that I thought no one would ever argue against.
You're pulling out the same tired arguments I see on WagesOfWins all the time. You guys are really enamored with things against the conventional wisdom, but this is a time when you have to take a step back and see what you're really talking about.
Thomas Robinson versus Asik.
Oh by the way I missed these:
Delete"A much better way to discredit the statistic would be to do a multiple regression of team defensive rating to rebounds, fouls, steals, and blocks, to see how much of defense this really explains."
That is a completely incorrect approach. The problem is that these variables are tangled. Team defensive rating is based on points allowed per possession, so of course possession stats can explain that, but it doesn't explain how important they actually are. Simple example: you can defend very well in isolation and force a missed shot. How does the box score see that? As a defensive rebound for someone else.
"This seems to go against exactly what you espoused in your contextualizing basketball stats post."
It's about using stats to tell the complete picture instead of completely relying on one metric or number. The problem I have with your replies is that you're using the same old tired cliches I always hear from Wages of Wins folks, but you have no content to your message. I noticed how you're not actually talking about basketball, but numbers. The stats will help tell what happened on the court, but if you are a complete slave to them you will see nothing.
Contextualize stats to understand what's going on in the game. Why aren't we actually talking about how Houston plays? When you watch Houston, Asik is the active defender who is deceptively quick for his size and snuffs out pick and rolls. Houston does not have good defensive schemes or many good defensive players. Asik basically is their entire defense, and at the end of games he's a sweaty, red-cheeked mess from giving it 100%. He also controls the boards. When he's off the court, Houston's defense completely collapses. He has one of the highest on/off defensive numbers in the league. Raw on/off can be terrible, but that's because a player may go off the court at the same time as other elite defenders. Not the case here. Houston doesn't have anyone else of Asik's caliber. Asik spent his first seasons with the Bulls, an elite defensive club, and even though he came off the bench the defense was often better when he was on the court. That is impressive. He was backing up the defensively stellar Noah. What you see on the court with Asik in the stats is an active player who controls the glass, and his defensive value is reflected in great defensive adjusted +/- numbers, which is not a surprise. Stats don't tell you everything, but they help form the picture.
That's funny because I don't like wages of wins much either. If you have actually read what I wrote on their blog, you would have seen that I am typically there to disagree with them or point out problems with the metric. I agree with you that they are very set in their ways, and I'm often frustrated by their refusal to consider any changes to their metric. Please don't label me as one of them.
DeleteI agree with both your examples (Asik and Gasol), but that agreement just comes from my own subjective opinion from watching games. I'm simply saying that we shouldn't place too much weight on these opinions. As I mentioned previously, RAPM provides several valuations that fail the "laugh test" as well (Danilo over Melo?), but that ultimately tells you very little about the efficacy of the metric. ASPM, IPV, WP, WS, etc. all have this same "problem" of providing several counter-intuitive results. But so what? PER has the fewest contrarian results; are we to take it to be the best metric available?
I think you've effectively shown why we cannot use the metric to compare players on different teams, but I haven't really heard anything from you that would serve to dismiss DRtg on an intra-team level. I understand it is flawed, but what measure is not? Along with defensive RAPM, DRtg is currently one of our best estimates for defense. I wouldn't be so quick to dismiss either stat, or any stat really, just due to a few results that you don't agree with.
Besides, Jones and Robinson are clearly the artifacts of small sample size, and no metric can protect against that. Of all the Rockets players with 500+ minutes, Asik is by far the best by DRtg. But again, this doesn't matter all that much.
I understand your point that every metric has some results that might not pass the "laugh test" but I think the most important thing is to ask "why?".
ReplyDeleteDRting isn't being dismissed simply because it disagrees with a few of his opinions. It's being dismissed because the way it tries to quantify defense is very flawed. Some of the most important aspects of defense whether it's hedging on a screen, rotating properly, or simply playing great on-ball defense are all things that are being ignored. I'm not sure how you can defend a metric's validity when it doesn't attempt to quantify some of the most important aspect's of what it is suppose to measure.
I'd compare it to APG and how it is in no way a direct measurement of someone's passing ability.There's much more to passing than just giving the ball to a teammate and having them make a field goal. Same way there is much more to defense than your opponent ending a possession without scoring while you're on the floor (which is all Drating measures).
Ok, those are very legitimate concerns. I agree that DRtg is flawed, I just think that, given the dearth of defensive metrics available, it does deserve some consideration when evaluating defense. What other alternatives are there? Defensive plus minus stats are much better in theory, but they will be essentially half as significant as overall plus minus stats, so I'm not sure how much better it actually is in practice.
ReplyDeleteFacing Share Problems for NRI. Capital Experts have many of consultants that will help you in Shares Inheritance for NRI . Contact Us Now and get the perfect solution you want.
ReplyDelete