Saturday, April 21, 2012

What is the Mocktopus?

The Mocktopus is a collection of models designed to create mock drafts using statistics.  

Here is how it works.  The Mocktopus first estimates the probability that a given team "on the clock" will select a player from one of eleven position groups--quarterbacks, offensive tackles, running backs, wide receivers, tight ends, interior offensive linemen, interior defensive linemen, edge rushers (college defensive ends), linebackers who are not edge rushers, cornerbacks, and safeties.  A separate model then estimates the probability that the team on the clock will select the first, second, third, or fourth "most valuable" player available at the position.  Both of these estimates are conducted using a technique called logistic regression.  Using software that I specifically designed to execute the models, the Mocktopus simulates the NFL Draft 10,000 times to create a mock draft. 

The principal components in the Mocktopus are other mock drafts.  Although mock drafts are frequently criticized for missing on their picks more often than they hit, they are nevertheless significantly more accurate than a chimpanzee pulling random names out of a hat.  My research suggests that those who have had the most accurate mock drafts in the past can also be expected to produce above average mock drafts in the future, although their level of accuracy can be expected to drop somewhat.  Thus, as part of its calculations, the Mocktopus assigns weights to mock drafts from the most successful draftniks based on historical accuracy, but it also assumes that the accuracy of their mock drafts will regress towards the mean in the future. 

The second component is a metric that measures value by combining the "Top 100" overall prospects lists from the Draft Countdown and NFL Draft Scout websites.  Mock drafts tend to focus too much on team needs and not enough on prospect value, so including a metric for "value" helps control for a bias for pairing teams with low value players in order to address the teams' most critical needs.  Moreover, when a team drafts a player at a position that few expect, it often does so at a position where good value is available (for instance, St. Louis' selection of Robert Quinn and Detroit's selection of Nick Fairley in the 2011 Draft).  Interestingly, value was not a statistically significant factor as to quarterbacks and offensive tackles, which are positions highly-driven by team needs, and therefore the Mocktopus does not use value to predict whether a team will draft a quarterback or offensive tackle (although it does use value to help it predict which quarterback or offensive tackle will be drafted). 

Finally, the Mocktopus also incorporates team performance variables that have been statistically significant predictors of player choice in past drafts.  As to quarterbacks, the Mocktopus' predictor variables include the age of the team's projected starter (teams with old quarterbacks tend to draft new ones).  As to wide receivers, the predictor variables include the collective yards per catch of the team's projected starting wide receivers (teams tend to draft wide receivers when they lack down-field threats on the roster).  There are other team-based factors that barely missed the cut for inclusion in the Mocktopus because they approached, but were not, statistically significant (such as team sack rates for edge rushers) so there is definitely the potential to add additional team-based factors in the future.

The Mocktopus creates what I refer to as a "competitive mock."  I will have a little more on that later, but in short, it is a mock draft that is optimized to score as highly as possible under the scoring system developed by the website the Huddle Report: two points for a player correctly matched to the team that drafted him and one point for correctly slotting a player in the first round.  In short, this means that the Mocktopus sometimes does not match each team with the player it is most likely to draft, but rather, sometimes takes a team that lacks a clear favorite draftee and gives it a player that Mocktopus is confident will go in the first round, but cannot quite figure out where. 

I can only speculate on how accurate the mock drafts produced by the Mocktopus will be.  An early version of the Mocktopus would have scored a 51 on the Huddle Report's scoring system for the 2011 Draft (good enough to tie for second-best out of the hundred plus mock drafts submitted to the Huddle Report that year).  Its scores for previous years were okay, but not nearly that impressive.  I made some significant changes to the models since then that should make the Mocktopus more accurate, but I have no way of knowing until I take it for a test run on an independent set of data--the 2012 NFL Draft.  So, yes, this is a test.  This is only a test.  

I hope you enjoy the Mocktopus! 

No comments:

Post a Comment