Ranker Methodology

I started using The Ranker about mid-way through the 2016 college football season.  I was encouraged by the initial results, so I transitioned after the football season to basketball and continued to develop, test, and train The Ranker.  I’ve been quite pleased with the results from working on our first basketball season.  After all, The Ranker was better at picking against the spread than ESPN’s BPI predictions.

To be clear, The Ranker is a computer model that I created and continue to test and tweak.  Inspired by Jeff Sagarin’s ratings, the goal was to assign ratings to every team that can be used to predict the scoring margin of any given game.  Basically, the team with the higher rating should win by a margin equal to the difference between the two teams’ ratings.

So I initially worked to figure out a good way to assign ratings to teams.  I came across an approach I really liked based on a simple linear algebraic formulation where you use a matrix representing every game from the season, a vector containing the actual scoring margin of each of those games, and a vector containing the unknown team ratings for each team.  Then you solve the formula to find the set of team ratings that minimizes the error between the actual and predicted scoring margins.

I wanted to have team ratings that are based on more than just the scoring margin of each game.  I wanted a team’s rating to be influenced by other measures of how good a team is, as well, such as how good they are at shooting or how good they are at creating turnovers, for example.  So I decided to create a simple function that uses a team’s stats (FG%, defensive FG%, scoring, rebounds, etc) to calculate it’s rating.  I just add a team’s statistical scores together to get their rating, but each stat would be weighted.  Assigning a weight to each statistical category allows the model to decide which stats are more important than others in calculating a team’s rating.  This left me with two big questions:  1) How do I represent a team’s stats?   and 2)  How do I determine what the weights should be?

First, I was weary about using a team’s average statistics, because it seemed unfair to compare a team that plays tough opponents with a team that has a lighter strength of schedule.  For example, let’s say West Virginia is ranked 20th in points per game, behind teams like IPFW and Savannah State.  I wanted a way of quantifying how good a team is at scoring points.  No offense, but are IPFW and Savannah State actually better at scoring points than West Virginia, who plays in the tough Big 12 Conference?  So, long story short, I developed a method to account for who a team plays against.  Using points per game as an example, my method rewards a team for scoring more points than their opponent allows on average and penalizes a team for scoring less than their opponent’s average points allowed.  Adjusting for opponent strength results in West Virginia moving up a few ranks to 13th, as I expected.  Savannah State dropped from to 10th to 16th, and IPFW dropped from 7th to 39th. I adjust every team’s stats likewise to end up with what you could call scores for every statistic that I used.  Next, I needed to develop weights for each statistical category, and then I would have what I need to calculate team ratings.

There are lots of ways to calculate the weights for each team stat.  Actually, testing different methods is something I want to do in the off-season to look for ways to improve The Ranker.  I had always wanted to use a genetic algorithm for some analysis, and I thought it was a viable option for this effort, so I decided to use that method.  I know – not the best way to pick an approach, but this is a hobby, and I wanted to have fun, so I went for it.  In a genetic algorithm, you treat a potential solution (in this case,  a set of weights) as an individual, and you measure the “health” of that individual (in this case, how well the weights work out in calculating the team ratings for predicting scoring margins of games).  Next, you set up a “population” of individuals, and let them breed and mutate.  In each generation, you keep the healthiest individuals, discard the weakest ones, and let them breed and mutate again.  So from generation to generation, your population gets stronger, which means your solution improves.  Through “evolution,” you eventually wind up with a pretty good solution.  After a set number of generations, I take the best “individual” and use it as my weights in the team rating calculation.

Finally, to calculate a team’s rating, I multiply each of the team’s statistical scores by that statistic’s weight and add it all up.  One last tidbit:  I also threw in a home team advantage factor.  Once again, inspired by Jeff Sagarin, I assumed the home team benefits from being the home team.  So when predicting the scoring margin of a game, I add the home team advantage factor to the home team’s rating.  I included this factor in the genetic algorithm to determine what it should be.

It’s been a fun effort.  I’ve learned a lot, and I have some ideas that could possibly (hopefully) improve The Ranker’s performance.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s