Sports rating system

Last updated

A sports rating system is a system that analyzes the results of sports competitions to provide ratings for each team or player. Common systems include polls of expert voters, crowdsourcing non-expert voters, betting markets, and computer systems. Ratings, or power ratings, are numerical representations of competitive strength, often directly comparable so that the game outcome between any two teams can be predicted. Rankings, or power rankings, can be directly provided (e.g., by asking people to rank teams), or can be derived by sorting each team's ratings and assigning an ordinal rank to each team, so that the highest rated team earns the #1 rank. Rating systems provide an alternative to traditional sports standings which are based on win–loss–tie ratios.

Contents

College football players in the United States UCF at the Texas goal line.jpg
College football players in the United States

In the United States, the biggest use of sports ratings systems is to rate NCAA college football teams in Division I FBS, choosing teams to play in the College Football Playoff. Sports ratings systems are also used to help determine the field for the NCAA men's and women's basketball tournaments, men's professional golf tournaments, professional tennis tournaments, and NASCAR. They are often mentioned in discussions about the teams that could or should receive invitations to participate in certain contests, despite not earning the most direct entrance path (such as a league championship). [1]

Computer rating systems can tend toward objectivity, without specific player, team, regional, or style bias. Ken Massey writes that an advantage of computer rating systems is that they can "objectively track all" 351 college basketball teams, while human polls "have limited value". [2] Computer ratings are verifiable and repeatable, and are comprehensive, requiring assessment of all selected criteria. By comparison, rating systems relying on human polls include inherent human subjectivity; this may or may not be an attractive property depending on system needs.

History

Sports ratings systems have been around for almost 80 years, when ratings were calculated on paper rather than by computer, as most are today. Some older computer systems still in use today include: Jeff Sagarin's systems, the New York Times system, and the Dunkel Index, which dates back to 1929. Before the advent of the college football playoff, the Bowl Championship Series championship game participants were determined by a combination of expert polls and computer systems.

Theory

Sports ratings systems use a variety of methods for rating teams, but the most prevalent method is called a power rating. The power rating of a team is a calculation of the team's strength relative to other teams in the same league or division. The basic idea is to maximize the amount of transitive relations in a given data set due to game outcomes. For example, if A defeats B and B defeats C, then one can safely say that A>B>C.

There are obvious problems with basing a system solely on wins and losses. For example, if C defeats A, then an intransitive relation is established (A > B > C > A) and a ranking violation will occur if this is the only data available. Scenarios such as this happen fairly regularly in sports—for example, in the 2005 NCAA Division I-A football season, Penn State beat Ohio State, Ohio State beat Michigan, and Michigan beat Penn State. To address these logical breakdowns, rating systems usually consider other criteria such as the game's score and where the match was held (for example, to assess a home field advantage). In most cases though, each team plays a sufficient amount of other games during a given season, which lessens the overall effect of such violations.

From an academic perspective, the use of linear algebra and statistics are popular among many of the systems' authors to determine their ratings. Some academic work is published in forums like the MIT Sloan Sports Analytics Conference, others in traditional statistics, mathematics, psychology, and computer science journals.

If sufficient "inter-divisional" league play is not accomplished, teams in an isolated division may be artificially propped up or down in the overall ratings due to a lack of correlation to other teams in the overall league. This phenomenon is evident in systems that analyze historical college football seasons, such as when the top Ivy League teams of the 1970s, like Dartmouth, were calculated by some rating systems to be comparable with accomplished powerhouse teams of that era such as Nebraska, USC, and Ohio State. This conflicts with the subjective opinion that claims that while good in their own right, they were not nearly as good as those top programs. However, this may be considered a "pro" by non-BCS teams in Division I-A college football who point out that ratings systems have proven that their top teams belong in the same strata as the BCS teams. This is evidenced by the 2004 Utah team that went undefeated in the regular season and earned a BCS bowl bid due to the bump in their overall BCS ratings via the computer ratings component. They went on to play and defeat the Big East Conference champion Pittsburgh in the 2005 Fiesta Bowl by a score of 35-7. A related example occurred during the 2006 NCAA men's basketball tournament where George Mason were awarded an at-large tournament bid due to their regular season record and their RPI rating and rode that opportunity all the way to the Final Four.

Goals of some rating systems differ from one another. For example, systems may be crafted to provide a perfect retrodictive analysis of the games played to-date, while others are predictive and give more weight to future trends rather than past results. This results in the potential for misinterpretation of rating system results by people unfamiliar with these goals; for example, a rating system designed to give accurate point spread predictions for gamblers might be ill-suited for use in selecting teams most deserving to play in a championship game or tournament.

Rating considerations

Home advantage

France national basketball team fans France national team fans.jpg
France national basketball team fans

When two teams of equal quality play, the team at home tends to win more often. The size of the effect changes based on the era of play, game type, season length, sport, even number of time zones crossed. But across all conditions, "simply playing at home increases the chances of winning." [3] A win away from home is therefore seen more favorably than a win at home, because it was more challenging. Home advantage (which, for sports played on a pitch, is almost always called "home field advantage") is also based on the qualities of the individual stadium and crowd; the advantage in the NFL can be more than a 4-point difference from the stadium with the least advantage to the stadium with the most. [4]

Strength of schedule

Strength of schedule refers to the quality of a team's opponents. A win against an inferior opponent is usually seen less favorably than a win against a superior opponent. Often teams in the same league, who are compared against each other for championship or playoff consideration, have not played the same opponents. Therefore, judging their relative win–loss records is complicated.

We looked beyond the record. The committee placed significant value on Oregon's quality of wins.

—  College football playoff committee chairman Jeff Long, press conference, week 12 of the 2014 season, [5] after ranking 9–1 Oregon above 9–0 Florida State

The college football playoff committee uses a limited strength-of-schedule algorithm that only considers opponents' records and opponents' opponents' records [6] (much like RPI).

Points versus wins

A key dichotomy among sports rating systems lies in the representation of game outcomes. Some systems store final scores as ternary discrete events: wins, draws, and losses. Other systems record the exact final game score, then judge teams based on margin of victory. Rating teams based on margin of victory is often criticized as creating an incentive for coaches to run up the score, an "unsportsmanlike" outcome. [7]

Still other systems choose a middle ground, reducing the marginal value of additional points as the margin of victory increases. Sagarin chose to clamp the margin of victory to a predetermined amount. [8] Other approaches include the use of a decay function, such as a logarithm or placement on a cumulative distribution function.

In-game information

Beyond points or wins, some system designers choose to include more granular information about the game. Examples include time of possession of the ball, individual statistics, and lead changes. Data about weather, injuries, or "throw-away" games near season's end may affect game outcomes but are difficult to model. "Throw-away games" are games where teams have already earned playoff slots and have secured their playoff seeding before the end of the regular season, and want to rest/protect their starting players by benching them for remaining regular season games. This usually results in unpredictable outcomes and may skew the outcome of rating systems.

Team composition

Teams often shift their composition between and within games, and players routinely get injured. Rating a team is often about rating a specific collection of players. Some systems assume parity among all members of the league, such as each team being built from an equitable pool of players via a draft or free agency system as is done in many major league sports such as the NFL, MLB, NBA, and NHL. This is certainly not the case in collegiate leagues such as Division I-A football or men's and women's basketball.

Cold start

At the beginning of a season, there have been no games from which to judge teams' relative quality. Solutions to the cold start problem often include some measure of the previous season, perhaps weighted by what percent of the team is returning for the new season. ARGH Power Ratings is an example of a system that uses multiple previous years plus a percentage weight of returning players.

Rating methods

Permutation of standings

Several methods offer some permutation of traditional standings. This search for the "real" win–loss record often involves using other data, such as point differential or identity of opponents, to alter a team's record in a way that is easily understandable. Sportswriter Gregg Easterbrook created a measure of Authentic Games, which only considers games played against opponents deemed to be of sufficiently high quality. [9] The consensus is that all wins are not created equal.

I went through the first few weeks of games and redid everyone’s records, tagging each game as either a legitimate win or loss, an ass-kicking win or loss, or an either/or game. And if anything else happened in that game with gambling repercussions – a comeback win, a blown lead, major dysfunction, whatever — I tagged that, too.

—  Bill Simmons, sportswriter, Grantland [10]

Pythagorean

Pythagorean expectation, or Pythagorean projection, calculates a percentage based on the number of points a team has scored and allowed. Typically the formula involves the number of points scored, raised to some exponent, placed in the numerator. Then the number of points the team allowed, raised to the same exponent, is placed in the denominator and added to the value in the numerator. Football Outsiders has used [11]

The resulting percentage is often compared to a team's true winning percentage, and a team is said to have "overachieved" or "underachieved" compared to the Pythagorean expectation. For example, Bill Barnwell calculated that before week 9 of the 2014 NFL season, the Arizona Cardinals had a Pythagorean record two wins lower than their real record. [12] Bill Simmons cites Barnwell's work before week 10 of that season and adds that "any numbers nerd is waving a “REGRESSION!!!!!” flag right now." [13] In this example, the Arizona Cardinals' regular season record was 8-1 going into the 10th week of the 2014 season. The Pythagorean win formula implied a winning percentage of 57.5%, based on 208 points scored and 183 points allowed. Multiplied by 9 games played, the Cardinals' Pythagorean expectation was 5.2 wins and 3.8 losses. The team had "overachieved" at that time by 2.8 wins, derived from their actual 8 wins less the expected 5.2 wins, an increase of 0.8 overachieved wins from just a week prior.

Trading "skill points"

Originally designed by Arpad Elo as a method for ranking chess players, several people have adapted the Elo rating system for team sports such as basketball, soccer and American football. For instance, Jeff Sagarin and FiveThirtyEight publish NFL football rankings using Elo methods. [14] Elo ratings initially assign strength values to each team, and teams trade points based on the outcome of each game.

Solving equations

Researchers like Matt Mills use Markov chains to model college football games, with team strength scores as outcomes. [15] Algorithms like Google's PageRank have also been adapted to rank football teams. [16] [17]

List of sports rating systems

Bowl Championship Series computer rating systems

In collegiate American football, the following people's systems were used to choose teams to play in the national championship game.

Further reading

Bibliographies

Academic work

Related Research Articles

<span class="mw-page-title-main">Elo rating system</span> Method for calculating relative skill levels of players

The Elo rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess or esports. It is named after its creator Arpad Elo, a Hungarian-American physics professor.

A tournament is a competition involving at least three competitors, all participating in a sport or game. More specifically, the term may be used in either of two overlapping senses:

  1. One or more competitions held at a single venue and concentrated into a relatively short time interval.
  2. A competition involving a number of matches, each involving a subset of the competitors, with the overall tournament winner determined based on the combined results of these individual matches. These are common in those sports and games where each match must involve a small number of competitors: often precisely two, as in most team sports, racket sports and combat sports, many card games and board games, and many forms of competitive debating. Such tournaments allow large numbers to compete against each other in spite of the restriction on numbers in a single match.

Pythagorean expectation is a sports analytics formula devised by Bill James to estimate the percentage of games a baseball team "should" have won based on the number of runs they scored and allowed. Comparing a team's actual and Pythagorean winning percentage can be used to make predictions and evaluate which teams are over-performing and under-performing. The name comes from the formula's resemblance to the Pythagorean theorem.

<span class="mw-page-title-main">Peach Bowl</span> Annual American college football postseason game

The Peach Bowl is an annual college football bowl game played in Atlanta, Georgia, since December 1968. Since 1997, it has been sponsored by Chick-fil-A and is officially known as the Chick-fil-A Peach Bowl. From 2006 to 2013, it was officially referred to as simply the Chick-fil-A Bowl. The winner of the bowl game is awarded the George P. Crumbley Trophy, named after the game's founder George Crumbley.

<span class="mw-page-title-main">Bowl Championship Series</span> American college football playoff series

The Bowl Championship Series (BCS) was a selection system that created four or five bowl game match-ups involving eight or ten of the top ranked teams in the NCAA Division I Football Bowl Subdivision (FBS) of American college football, including an opportunity for the top two teams to compete in the BCS National Championship Game. The system was in place for the 1998 through 2013 seasons and in 2014 was replaced by the College Football Playoff.

A sports league is a group of individual athletes, sports teams or clubs who form a league to compete against each other and gain points in a specific sport. At its simplest, it may be a local group of amateur athletes who form teams among themselves and compete periodically, at its most complex, it can be an international professional league making large amounts of money and involving dozens of teams and thousands of players.

Jeff Sagarin is an American sports statistician known for his development of a method for ranking and rating sports teams in a variety of sports. His Sagarin Ratings have been a regular feature in the USA Today sports section from 1985 to 2023, have been used by the NCAA Tournament Selection Committee to help determine the participants in the NCAA Men's Division I Basketball Championship tournament since 1984, and were part of the college football Bowl Championship Series throughout its history from 1998 to 2014.

There are a number of formats used in various levels of competition in sports and games to determine an overall champion. Some of the most common are the single elimination, the best-of- series, the total points series more commonly known as on aggregate, and the round-robin tournament.

Mid-major conferences in American college sports at the NCAA Division I level are athletic conferences that are not among the Power conferences. The grouping is used particularly in men's college basketball to describe conferences outside of the Big Ten, Big 12, Big East, SEC, Pac-12, and ACC, collectively referred to as the Power Six or "high majors".

Football Outsiders (FO) was a website started in July 2003 which focused on advanced statistical analysis of the National Football League (NFL). The site was run by a staff of regular writers, who produced a series of weekly columns using both the site's in-house statistics and their personal analyses of NFL games.

<span class="mw-page-title-main">Kenneth Massey</span> American sports statistician (born 1975)

Kenneth Massey is an American sports statistician known for his development of a methodology for ranking and rating sports teams in a variety of sports. His ratings have been a part of the Bowl Championship Series since the 1999 season. He is an assistant professor of mathematics at Carson–Newman University in Tennessee.

In a group tournament, unlike a knockout tournament, there is no scheduled decisive final match. Instead, all the competitors are ranked by examining the results of all the matches played in the tournament. Typically, points are awarded for each match, with competitors ranked based either on total number of points or average points per match. A special type of group tournament is the Round-robin tournament, in which each player plays against every other player.

In American college football, the 2006 BCS computer rankings are a part of the Bowl Championship Series (BCS) formula that determines who plays in the BCS National Championship Game as well as several other bowl games. Each computer system was developed using different methods which attempts to rank the teams' performance. For 2006, the highest and lowest rankings for a team are dropped and the remaining four rankings are summed. A team ranked #1 by a computer system is given 25 points, #2 is given 24 points and so forth. The summed values are then divided by 100. The values are then ranked by percentage. This percentage ranking is then averaged with the Coaches Poll and Harris Poll average rankings, each receiving equal weight, and the results become the BCS Rankings.

In American college football, the 2007 BCS computer rankings are a part of the Bowl Championship Series (BCS) formula that determines who plays in the BCS National Championship Game as well as several other bowl games. Each computer system was developed using different methods which attempts to rank the teams' performance. For 2007, the highest and lowest rankings for a team are dropped and the remaining four rankings are summed. A team ranked #1 by a computer system is given 25 points, #2 is given 24 points and so forth. The summed values are then divided by 100. The values are then ranked by percentage. This percentage ranking is then averaged with the Coaches Poll and Harris Poll average rankings, each receiving equal weight, and the results become the BCS Rankings.

In sports, strength of schedule (SOS) refers to the difficulty or ease of a team's/person's opponent as compared to other teams/persons. This is especially important if teams in a league do not play each other the same number of times.

<span class="mw-page-title-main">Fordham Rams football</span> Intercollegiate American football team for Fordham University

The Fordham Rams football program is the intercollegiate American football team for Fordham University, located in the borough of The Bronx in New York City. The team competes in the NCAA Division I Football Championship Subdivision (FCS) and are members of the Patriot League. Fordham's first football team was fielded 142 years ago in 1882; the team plays its home games on campus at 7,000-seat Coffey Field.

A chess rating system is a system used in chess to estimate the strength of a player, based on their performance versus other players. They are used by organizations such as FIDE, the US Chess Federation, International Correspondence Chess Federation, and the English Chess Federation. Most of the systems are used to recalculate ratings after a tournament or match but some are used to recalculate ratings after individual games. Popular online chess sites such as Chess.com, Lichess, and Internet Chess Club also implement rating systems. In almost all systems, a higher number indicates a stronger player. In general, players' ratings go up if they perform better than expected and down if they perform worse than expected. The magnitude of the change depends on the rating of their opponents. The Elo rating system is currently the most widely used. The Elo-like ratings systems have been adopted in many other contexts, such as other games like Go, in online competitive gaming, and in dating apps.

David Rothman was an American statistician, public policy advisor, and the creator of a computerized college football ranking system.

<span class="mw-page-title-main">College Football Playoff</span> Postseason tournament in American college football

The College Football Playoff (CFP) is an annual postseason knockout invitational tournament to determine a national champion for the National Collegiate Athletic Association (NCAA) Division I Football Bowl Subdivision (FBS), the highest level of college football competition in the United States. It culminates in the College Football Playoff National Championship game. The inaugural tournament was held at the end of the 2014 NCAA Division I FBS football season under a four-team format. The CFP Board of Managers voted in 2023 to expand the playoff to twelve teams beginning in 2024, an arrangement that will last at least through the end of the 2025 season. After 2025, the current contract between all major players expires and a new contract will be drawn up, with indications that additional expansion may take place as soon as 2026.

<span class="mw-page-title-main">Mountain West Conference Football Championship Game</span> College football championship game

The Mountain West Conference Football Championship Game is an annual postseason college football game played to determine the champion of the Mountain West Conference (MW).

References

  1. ↑ Fagan, Ryan (2011-03-09). "Sorting through teams on one big bubble". Sporting News . Retrieved 2011-03-24. This is a look at 20 of the teams (in alphabetical order) residing on this year's big ol' bubble. We've included three statistical rankings. The RPI (ratings percentage index, taken from collegeRPI.com) is considered the standard and is provided to committee members during the selection process. The two other ranking indexes include margin of victory in their formulas—the Pomeroy ratings (at kenpom.com) and Sagarin ratings (via USA Today)—aren't new but have played an increased role in discussions about potential seeds during this college basketball season.
  2. ↑ Ken Massey [@masseyratings] (3 Nov 2014). "@kenpomeroy human polls have limited value. Computer systems can objectively track all the teams. http://www.masseyratings.com/cb/compare.htm #all351" (Tweet). Retrieved 9 Nov 2014– via Twitter.
  3. ↑ Jamieson, Jeremy P. (2010). "The Home Field Advantage in Athletics: A Meta-Analysis" (PDF). Journal of Applied Social Psychology. 40 (7): 1819–1848. doi:10.1111/j.1559-1816.2010.00641.x . Retrieved 11 November 2014.
  4. ↑ Barnwell, Bill (December 20, 2013). "Safe at Home". Grantland . Retrieved November 11, 2014.
  5. ↑ Russo, Ralph D. (11 November 2014). "Oregon up to 2 in playoff rankings; TCU to 4th". Associated Press. Retrieved 12 November 2014.
  6. ↑ Stewart Mandel [@slmandel] (12 Nov 2014). "Committee doesn't use an SOS ranking. It looks at opponents' record and opponents' opponents record" (Tweet). Retrieved 12 Nov 2014– via Twitter.
  7. ↑ Richards, Darryl (2001). "BCS removes margin-of-victory element". Fox Sports . Retrieved 12 November 2014.
  8. ↑ Sagarin, Jeff (Fall 2014). "NCAAF Jeff Sagarin Ratings". USA Today . Retrieved 12 November 2014.
  9. ↑ Easterbrook, Gregg (18 November 2014). "More flags on D spins scoreboards". ESPN . Retrieved 19 November 2014.
  10. ↑ Simmons, Bill (24 October 2014). "Week 8 Picks: A Gambling Epiphany". Grantland . Retrieved 19 November 2014.
  11. ↑ Schatz, Aaron; Alamar, Ben; Barnwell, Bill; Bill Connelly; Doug Farrar (2011). Football Outsiders Almanac 2011: The Essential Guide to the 2011 NFL and College Football Seasons. CreateSpace. p. xviii. ISBN   978-1-4662-4613-3.
  12. ↑ Barnwell, Bill (November 5, 2014). "NFL at the Half: Breaking Down the Numbers". Grantland . Retrieved January 7, 2015.
  13. ↑ Simmons, Bill (7 November 2014). "Revisiting the Y2K-Compliant Quarterbacks" . Retrieved 10 November 2014.
  14. ↑ Silver, Nate (4 September 2014). "Introducing NFL Elo Ratings". FiveThirtyEight . Retrieved 10 November 2014.
  15. ↑ Mills, Matt (21 December 2014). "Using Continuous-Time Markov Chains to Rank College Football Teams". The Spread. Retrieved 21 December 2014.
  16. ↑ "Ranking NFL teams using Network Science". LinkedIN . 17 March 2016. Retrieved 17 March 2016.
  17. ↑ "Modifying Google's Page Ranking Algorithm to rank teams". Reddit . 21 December 2014. Retrieved 22 December 2014.
  18. ↑ Weng, Ruby C.; Lin, Chih-Jen (2011). "A Bayesian Approximation Method for Online Ranking" (PDF). Journal of Machine Learning Research. 12: 267–300.
  19. ↑ "Wayne Winston: Analytics in the World of Sports". Indiana University Bloomington - Kelley School of Business - Operations & Decisions Technologies. Nov 25, 2013. Retrieved 8 Nov 2014.
  20. ↑ "Numbers game". Washington Times . April 13, 2004. Retrieved 8 Nov 2014.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.