Comp The Player: Classifying NBA Players

Article header for player comparison project.

With the expansion in popularity of NBA basketball over the years, an enormous amount of data is now being produced from game to game. This no longer is limited to simple counting stats like points, rebounds, and assists -- we can now ask questions like: "How many times did Klay Thompson dribble the ball before shooting in the 3rd quarter? How many miles did he run on defense? How many times did he shoot from beyond 24 feet?"

I recently spent some time scraping such data from the web in order to do some exploratory analysis on NBA basketball players (see my Github for code). One of the simplest questions we can ask is: what defines a player? Not in terms of production or value, but stylistically, how does a player play?

In my explorations, I have found that it is difficult to use unsupervised methods such as k-means clustering to define style of play, because the number of features and groups to use becomes highly arbitrary with so many different ways to play.

Therefore, I constructed a player classification model, with the target labels being the veteran players themselves. Such a model allows us to find the best "comp" for, say, rookie players who are frequently scouted by NBA front offices searching for cheap talent. With this motivation in mind, let's see how such a model can be built.

A Word About Random Forest Classification

A common tool used for regression and classification is the decision tree, which is a tree-like structure beginning with a root. From this starting node, which includes the entire sample, the algorithm decides which of the input features provides the most information for differentiating the sample data (i.e. player-seasons). The algorithm then splits the data on this feature.

The formula for information gain provided by a new feature is given by the difference between the entropy for the current node, $E(X)$, and weighted sum of the entropies of the nodes resulting from splitting the data on the new feature, $E(X | Y)$:

$$ IG(X, Y) = E(X) - E(X | Y) $$

The entropy of $E(X)$ is simply calculated from each $p_i$ (the probability/proportion of the $i$th class):

$$ E(X) = - \sum_i p_i \log{p_i} $$

The entropy $E(X | Y)$ is weighted by number of samples in each resultant node, preventing an insignificant node from dominating the entropy.

The algorithm repeats this process for each node, each time computing the information gain for the remaining input features. In this way, a tree is "grown" in descending order of information gain. For a nice, detailed explanation broken down using examples, see this article.

Now, if we can grow a tree, why not grow a forest? A random forest is a collection of many decision trees, which is highly advantageous for many problems. Each tree is a random sub-sample of the input training data (a sampling technique known as bootstrapping), and each tree uses a subset of input features. A vote is tallied from each tree's prediction in order to determine the overall prediction, essentially averaging over all estimators. This entire process is known as bagging, short for bootstrap aggregation.

Below is a cartoon from this article by Rosaria Silipo and Kathrin Melcher.

Image of random decision tree forest.
A random forest of decision trees. At the bottom is a simple example of data bootstrapping and random feature selection for various trees.

The strengths of this algorithm are that it can handle missing values, it automatically accounts for relative feature importance, and it avoids overfitting by cancelling out the large variance associated with each tree.

As we will see, our model will consist of 181 output labels, which is the number of veteran players who played for the last 4 complete NBA seasons (2016-2019). Not only are there many labels, but there are also many input features to differentiate style of play. What's more, we only have 3 samples per label to work with if we leave out one season as a test dataset.

For all these reasons, the Random Forest model is the ideal candidate for performing player classification.

Feature Selection

Through some exploratory data analysis which I will not thoroughly detail here, combined with some basketball logic and intuition, the input features below were selected for the random forest classification (RFC).

Shown is a Python snippet which displays the relevant features in a Pandas dataframe, which was the result of data scraping from a variety of NBA data sources.

# Specify which dataframe columns to use as model features
feature_list = ['height', 'weight', 'FG_FREQ_05FT', 'FG_FREQ_59FT', 'FG_FREQ_1014FT', 'FG_FREQ_1519FT', 'FG_FREQ_2024FT', 'FG_FREQ_GT24FT', 'FG_FREQ_CANDS', 'FTR', 'ASTR', 'TOR', 'ORR', 'DRR', 'BLK_PH', 'STL_PH', 'DFGP_PG']

# Display the dataframe with selected features
pd.set_option('display.max_columns', 13)
df[['name'] + feature_list]
A Pandas dataframe showing the data used as classifier model input.

These stats can be summarized as follows:

Note that height and weight are hardly "play-style" attributes, but rather physical attributes. They are still important since strict lines between player positions, e.g. point guards, shooting guards, centers, etc., are becoming increasingly blurred in the modern NBA, but a player's size still determines how a player is utilized.

The offensive stats are relatively self-explanatory; we largely differentiate a player by where on the floor they take their shots, how often they draw fouls, and how often they make plays or rebound the ball.

A player's defense is more difficult to quantify, however the opponent's defensive field goal percentage is included to account for man-on-man defense in addition to a player's blocks and steals.

It is worth noting that this list is neither exhaustive nor completely optimized. These resulted in decent accuracy, however there are a variety of unused additional stats that would be interesting to leverage. For example, I have also scraped play-type stats: how often does a player score on pick-and-rolls? Put-backs? Isolation plays? However, this would greatly increase the number of features and statistical interactions have not been thoroughly studied.

Creating a Random Forest Model

To create the RFC model, we first separate the input data into veterans and rookies. Here, veterans are all players who played in each season from 2016-2019, while rookies only played in the 2019 season.

I then divide the veteran dataset into a training set (3 seasons) and a testing set (the 2019 season), as out-of-sample testing will be used to calculate the unbiased accuracy of our model. I also separate the training and testing datasets into input features and output labels (the player names).

# Create separate dataframes for established players and rookies
df_vets = df[df["name"].isin(df["name"].value_counts()[df["name"].value_counts() == 4].index)]
df_train = df_vets[df_vets.year < 2019]
df_test = df_vets[df_vets.year == 2019]

df_rooks = df[df["name"].isin(df["name"].value_counts()[df["name"].value_counts() == 1].index)]
df_rooks = df_rooks[df_rooks.year == 2019]

# Create input features and target labels (player names) for test data
test_labels = df_test[["name"]]
test_features = df_test[feature_list]

# Create matching training dataset using only veterans who are still in the NBA
# (and thus have available training data)
train_labels = df_train[["name"]]
train_features = df_train[feature_list]

In order to properly fit our input training data, it is also a good idea to scale the input features to unit variance and subtract their means. This is a good habit when training machine learning algorithms to aid in model convergence and avoid feature biases.

# Scale features which will be used in model fitting
# (for both training and testing datasets)
scale_train = StandardScaler().fit(train_features)
train_features = scale_train.transform(train_features)
test_features = scale_train.transform(test_features)

A comprehensive grid search is performed on the various hyperparameters of the RandomForestClassifier in scikit-learn, using the GridSearchCV k-fold cross validation method. This is actually quite time intensive, but worth it given the large number of features and output labels.

# Perform a comprehensive grid search to optimize parameters for Random Forest classification
n_estimators = [100, 500, 1000, 2000, 5000]
max_depth = [5, 15, 30]
min_samples_split = [2, 10, 100]
min_samples_leaf = [1, 5, 10]

param_grid = {'n_estimators': n_estimators, 'max_depth' : max_depth, 'min_samples_split' : min_samples_split, 'min_samples_leaf' : min_samples_leaf}
grid_search = GridSearchCV(RandomForestClassifier(random_state = 1), param_grid, cv=3)
grid_search.fit(train_features, train_labels.values.ravel())
grid_search.best_params_
print('Optimized hyperparameters:', grid_search.best_params_)

rfc = RandomForestClassifier(
    n_estimators=grid_search.best_params_['n_estimators'],
    max_depth=grid_search.best_params_['max_depth'],
    min_samples_split=grid_search.best_params_['min_samples_split'],
    min_samples_leaf=grid_search.best_params_['min_samples_leaf'] )

rfc.fit(train_features, train_labels.values.ravel())

Once the hyperparameters are optimized, we fit the training data to create our RFC model.

The model is then run on our test dataset (the 2019 dataset for each veteran player). The accuracy here is then simply the fraction of veterans that were properly identified with themselves (from previous seasons).

# Run the classifier model on the test data
y_pred = rfc.predict(test_features)
print("Classified veteran players with an accuracy of", np.sum(test_labels.name.values==y_pred)*100./(len(y_pred)*1.), "%")

Output:
Classified veteran players with an accuracy of 92.81767955801105 %

As we can see, we correctly predict that veteran players are themselves with great accuracy, despite season-to-season variation. This is not far-fetched, since we are providing the player's physical attributes, however given fluctuations in the other stats this is still a solid result.

Results for Rookies in the 2019 Season

Once we have a trained and tested model, we can ideally run the classifier on any player not used in the training data to find their best player "comp".

It is worth noting: this is not a comparison of production or value. Aside from defensive FGP, the features used in the model are rate or frequency stats meant to characterize play style rather than performance.

# Format rookie dataset for input into the classifier and
# attempt to classify rookies by their most similar veteran counterpart
pred_labels = df_rooks[["name"]]
pred_features = df_rooks[feature_list]

pred_features = scale_train.transform(pred_features)
rook_pred = rfc.predict(pred_features)

for i,pred in enumerate(rook_pred):
    print("Predicted that Rookie", pred_labels.name.values[i], "is similar to Player:", pred)

Output:
Predicted that Rookie Aaron Holiday is similar to Player: Isaiah Canaan
Predicted that Rookie Alfonzo McKinnie is similar to Player: JaMychal Green
Predicted that Rookie Boban Marjanovic is similar to Player: Jusuf Nurkic
... (Luka Doncic → James Harden, Trae Young → Jeff Teague, Landry Shamet → Marco Belinelli, etc.)

Shown above are the predictions for each 2019 rookie. Just to get a sense that these comparisons are sensible, let's dive into an example.

An Example Comparison

Our classifier algorithm found Landry Shamet and Marco Belinelli to have matching play styles. Shamet came into the league with low expectations, but was a pleasant surprise and made an early impact as a shooter on the L.A. Clippers. Belinelli is a veteran player also known for his exceptional shooting skills off the bench.

We can simply plot these players in a number of feature spaces and examine how similar they really are.

Comparison plots for Landry Shamet and Marco Belinelli.
A comparison of Landry Shamet (black) and Marco Belinelli's seasons (red).

As we can see, they are indeed very similar. They are both lanky shooting guards who like to shoot deep 3-pointers. They are both limited play-makers who get around half their shots from catch-and-shoot action (i.e. without dribbling before the shot). Belinelli tended to shoot more free throws over the last few years, but this could merely indicate that he is older and more experienced with drawing fouls.

Why is this useful?

NBA team front offices and scouting teams spend an enormous amount of time scouting and evaluating players. Scouts fly all over the world to watch players live, they organize workouts to perform the "eye test", and the media often portrays a player in a light that effectively skews general perception.

In short, biases are a part of the evaluation system. This algorithm is a method for providing a direct comparison with veteran players who are successful enough to have found a role in the league.

Not only does this classification tool provide an unbiased label that may not have been considered by glancing at a box score, it also allows front offices to seek out cheap talent to fill a specific role. If a rebuilding team is looking for a young backup catch-and-shoot guard who plays tenacious defense, this algorithm may indicate which players are closer to Klay Thompson than James Harden.