A Consistency Metric Needs to be Readily Available for the NBA

Everything from counting statistics to the number of bank shots a player shoots is readily available for all NBA fans. Why isn’t consistency measured?

Player A and Player B both are shooting 50 percent from beyond the 3-point arc on 100 shots over the course of 10 games, each taking 10 shots per game. Just from these numbers provided, it would appear that both players are equal & elite 3-point shooters, right? Now, let’s present the data in a different fashion. Here are the shots made and attempted breakdown over those 10 games for each player:

Chart 1

Both Player A and Player B made 50 out of 100 3-point shots. However, Player A was wildly inconsistent over this 10-game stretch compared to Player B. Without looking at a player’s game log in detail or having a great memory from watching these games, you would think that both players were identical shooters over this stretch. When we look at a player’s season statistics either on Basketball-Reference.com or NBA.com, we are readily provided the averages and totals, yet have to go the extra mile to determine whether or not said player is consistent. Even then, it is a rather vague understanding of consistency, as in there is not specific metric provided to measure consistency.

Need to know how efficient a player is? There are stats that measure efficiency. Need to know what a player is shooting on pull-up jump shots? There are stats that measure different shot types. Need to know the number of miles per game a player runs on offense? There are stats for that. Need to know if a player is consistently putting up his average rebounds per game? Sorry, those consistency statistics are not available.

If you type into the Google Machine “NBA consistency stat,” you will find a number of daily fantasy sports websites providing consistency percentages, for what I am guessing is putting up their mean fantasy points per game. I do not play DFS, so I am not certain on this, but what I am certain about is that the metrics provided on those websites do not address the issue of the absence of a mainstream consistency metric.

Where are the Top NBA Players Drafted?

What is quite frustrating about this issue is that coming up with a method to calculate consistency isn’t a difficult task at all. The standard deviation (the square root of the variance) is a basic mathematical measure that calculates variation within the sample of the mean; in other words, how spread out a set of numbers are. The standard deviation effectively measures the consistency of a sample, and calculating it is even easier than before given that programs such as Google Sheets and Microsoft Excel can perform the task within seconds.

So, if it is so simple to calculate, why isn’t providing a standard deviation, or some variation of a standard deviation that measures consistency, of specific player statistics provided? Remember when I told you to Google “NBA consistency stat” two paragraphs ago? Well, in the midst of all of these DFS websites there is a 2014 Nylon Calculus article titled “Introducing: Player Consistency” written by Tim “Hal” Brown.

Brown, like myself, set out and asked the same larger question of why isn’t there a metric that allows individuals not working for a team’s analytics department to know if a player is consistency putting up his averages?

What makes Brown’s article particularly important is that went out of his way to actually create a measurement system for player consistency and specific values for a set of players in 2014! Brown explained his methodology behind his consistency metric, yet did not provide the formula in which he used to calculate the consistency values, and unfortunately has not been able to maintain and expand the data over time. I reached out to Nylon Calculus via Twitter to ask them about the data, to which they responded they no longer had the data.

I then reached out to Brown himself about the topic, to which he was excited that someone truly appreciated the work he put into calculating a consistency statistic. I asked him for his thoughts on the lack of having a variance-like statistic to measure, the issue of using a pure standard deviation to measure consistency, and the formula he used to calculate consistency.

“There’s more to basketball — and really social science in general — than talking about averages and absolutes,” Brown wrote to me in an email about the topic at hand. “I felt like consistency was a good way to introduce variance into the conversations people have about who is and isn’t valuable on an NBA court and why.” When I asked him why not just use a pure standard deviation, Brown wrote, “The problem with using raw variance as a measure of players’ consistency, though, is variance is going to naturally rely heavily on the player’s role, where theoretically a player’s consistency should be comparable across all players.”

This is of course completely true given that players who have larger roles will have larger standard deviation values. For example, if Player A averages 28 points per game with a standard deviation of 7 and Player B averages 10 points per game with a standard deviation of 5, it would appear that Player B is more “consistent.” But, if Player B is a role player whose standard deviation is 50% of his average versus 40% for Player A, who is a starter, can we really concluded that Player B is more consistent despite the smaller standard deviation?

Brown wanted to avoid these type of issues, so he and Krishna Narsu, a writer at Nylon Calculus at the time, came up with a way to normalize a player’s variance in every category “so that what we’re really measuring with consistency is how severely a player regularly deviates from his per-game averages.” The formula Brown and Narsu came up with is both simple and brilliant. It comes in two parts:

My attempted at writing Brown & Narsu’s formula in mathematical terms

For those who have no clue what I wrote out, the translation is the following. I would strongly suggest potentially pulling up a player’s game log to follow along:

1. I take the value of a specific statistical category for each game and subtract that value from its corresponding season average value. For example, Kristaps Porzingis scored 19 points against the Brooklyn Nets on Monday. He is averaging 18.1 points per game this season, so the result would be 0.9.
2. Take the absolute value (the positive of a number) of the result of that subtraction and divide it by the standard deviation of the results for each game. You will need to calculate the standard deviation beforehand. Following the previous example, you would take the absolute value of 0.9, which is 0.9 (if the result of Step 1 happened to be -0.9, the absolute value would be 0.9), and divide it by the standard deviation of the points sample, which are all the points scored in each game by Kristaps Porzingis.
3. You perform Steps 1 & 2 for every single value in the numeric sample and find the sum of all of those results. Regarding Porzingis, you would would perform the previous two steps for every single game Kristaps Porzingis played in and add all of those results together.
4. Calculate the mean of the results found in Step 3. Sticking with the example, after you take each and every points total for Porzingis and subtract each one with his PPG stat, you then calculate the average of those results.
5. Once you have that mean consistency figure in Step 4, you then subtract it from 1 and multiply that result by 10, which then gives you a normalized consistency metric for whatever statistical category you want to measure consistency for.

This is how Brown describes what the statistic measures in his 2014 article:

The result is a measure of how likely a player is to perform along the lines of his averages on any given game. A higher value means that the player has something approaching his averages for every game, and a lower value means that you can’t actually expect any specific level of production from him on any given night.

The scale of this system roughly works out as 0–5 with 0 being completely inconsistent and 5 being most consistent.

Brown’s article provides consistency statistics for points, true shooting percentage, assists, and rebounds on a per game basis, doing this for 67 different players. Here are his baselines for his 67 player sample:

For the pure production measures (points, assists, and rebounds), a consistency of 1.85 or below can be considered “damagingly inconsistent” for all players. Similarly, a consistency above 2.4 is “astoundingly consistent,” which could actually be a problem for some star-level players, since it means they’re unlikely to be going on a scoring binge on any given night.

For True Shooting Percentage, a consistency of 1.91 and 2.55 are more safe low and high baselines.

So, if there is a quality formula to use, or at the very least a basis for a formula that can be improved upon to create a more accurate consistency statistic, then why hasn’t it caught on? Why don’t either Basketball-Reference or the NBA use this to measure consistency? They have the manpower and resources to track this, right? Unfortunately, I do not have the answers to why that is. However, what I do know is that Brown and Narsu were on the right path and the NBA needs to make an effort to make consistency a statistical norm. It’s extremely difficult for 2 people to regularly upkeep this sort of data, especially for every player in the league with so many different statistical categories. The purpose of this article is an attempt to spread awareness of this issue, hoping to catch the eye of either Basketball-Reference or the NBA to begin keeping track of consistency, since Brown, Narsu, or myself don’t have the time or resources to perform and maintain the data.

The foundation has already been laid out. Now, let’s work to make consistency a mainstay in the NBA statistical community!

Since this is an article published on Gotham Sports Network, it is only necessary to tie in the New York Knicks into this discussion about consistency. Instead of going on a huge spiel and soapboxing about how the Knicks gave up 120 points to the worst team in the league, I thought I would provide some key consistency figures for key Knicks players.

The chart below details the field goal percentage consistency metric (I should have calculated true shooting percentage, but I did not and I apologize for that) and the game score consistency metric for five Knicks players this season: Carmelo Anthony, Guillermo Hernangomez, Courtney Lee, Kristaps Porzingis, and Derrick Rose.

Chart 2

According to the results, Derrick Rose in the most consistent shooter and performer of the five players, in terms true shooting percentage, and the least consistent of this small group is Carmelo Anthony. Keep in mind that the established baselines in the previous section DO NOT HAVE ANY BEARING on the results in Chart 2. Since I did not calculate consistency figures for at least 67 players, I do not know what values are considered “great” or “poor” relative to the rest of the league. What can be concluded is that if the scale is essentially 0–5, only Derrick Rose is above the median figure of 2.50 for TS% Consistency.

A Stoic’s Guide to Rooting for the Knicks (and other terrible teams)

If the 2013–14 season results have any similarities to the results in Chart 2, Derrick Rose is consistently a true shooting percentage of 52.4%, which is below the league average of 55.2%, per Basketball-Reference.com, whereas Carmelo Anthony is flirting with the “damagingly inconsistent” threshold. Remember, take these comparisons to the Nylon Calculus results with a grain of salt.

When the Knicks seasons comes to its technical end (the season has been essentially over for about a month now), I will revisit the consistency metric for key statistical categories for all Knicks players. Until then, advocate for consistency to become mainstream.

Note: A previous version of this article had a different Chart 2, which detailed field goal percentage and game score consistency figures. It was updated to detail just true shooting percentage consistency, as TS% is a better metric to measure a player’s shooting than FG%.