Can Crusher Ratings - vol.1

Discussion in 'UFC Discussion' started by dogooder, Jan 3, 2015.

  1. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    I'm so pumped for tonight. Is Jones/Cormier the biggest fight ever? According to Can Crusher Ratings, it just might be. Here's the top 10 fights according to combined fighter rating at the time of the fight:

    [​IMG]

    Jones and Cormier currently total 102 + 93 = 194 points, which puts them first all-time. And if that weren't enough, Weidman-Belfort at the end of February has the potential to be even bigger. It's a great time to be an MMA fan.

    My goal here was to develop a simple and predictive system that gives reasonable and interesting results, and I think I've done that. The best "system" is probably Vegas, which predicts the winners 68% of the time according to BestFightOdds. On the same set of fights, CCR picks the correct winners in 64% of UFC fights and 65% of non-UFC fights (over 2000 fights in each group). And it's well calibrated: fighters that it claims have a 30% chance of winning end up winning about 30% of the time. For all fights listed on Sherdog since 2001 where both fighters have at least three previous fights, it correctly predicts the winners 70% of the time.

    There's other objective systems out there, including Fight Matrix and ScoreCardMMA. I think both of those are really cool, but they're also complicated and arbitrary (and neither aim to be predictive). I'll explain how my system works in the next post and end here with tonight's card:

    [​IMG]

    The moneylines mostly agree with the bookmakers. The big exception is Dunham-Damm, where Dunham is about -500 but CCR has it even.
     
  2. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Other sports have advantages over MMA as far as rating systems go. They may have hundreds of games a season with only a dozen or two teams, and they may have a balanced schedule so that a team's record is a fairly good indication of their skill. A team may have many players contributing to a their success, so that their skill level stays relatively constant over time even when they lose or gain players by injuries or trades. Many games are determined by the accumulation of points, so "worse" teams may win a few battles but ultimately lose the war.

    Contrast that with MMA, where everyone has a puncher's chance, where "styles make fights", and the extreme case where "worse" fighters can lay-and-pray or wall-and-stall to a decision win. Skills can vary greatly as fighters learn new techniques or lose their chins. And there's thousands upon thousands of fighters, with only a fraction having more than a dozen or two fights their whole career.

    As motivation, let me talk about the simple case where two fighters fight each another a bunch of times. Suppose one of them wins s times out of n fights. What's the best guess at the probability that he wins the next fight? You might want to say it's just his win percentage s/n, but this would mean for example that a fighter who was 1-0 would have a 100% chance of winning the next fight. That's not very reasonable.

    Laplace's rule of succession says that the best guess is (s+1)/(n+2). In the 1-0 case this is (1+1)/(1+2) = 2/3 or 67%. In general, this means that we add two fights to a fighter's record, one win and one loss. So if a fighter wins 100 fights in a row, his chance of winning the 101st is 101/102 = 99.02%.

    CCR uses the Bradley-Terry framework, which says that the likelihood that fighter i beats fighter j is Prob(i beats j) = p_i/(p_i + p_j), where p_i and p_j are the fighters' strengths. Often in this setup the strengths are converted to ratings by setting p_i = c^r_i where c is a constant (for example, c = 10^(1/400) in the ELO rating system). This way the probability depends on the difference of the two ratings rather than the ratio of the two strengths, and differences are often easier to work with and understand intuitively.

    CCR tries to find the most likely set of ratings given the fight outcomes. Bayes' theorem says that Prob(ratings | results) = Prob(results | ratings) * Prob(ratings) / Prob(results). So maximizing the left-hand side (the probability of the ratings given the fight results) is the same as maximizing the right-hand side. Prob(results) is constant since the results are fixed. Prob(results | ratings) = probability of the fight results given the ratings, is just the product of Prob(i beats j) over all fights where fighter i beat fighter j. And Prob(ratings), called the "prior", is where the assumption about the extra win and loss comes in. That is, Prob(ratings) is equal to the product of Prob(i beats 0)*Prob(0 beats i) over all fighters i, where fighter 0 is the "average fighter" (or "dummy") with fixed rating r_0 (usually = 0).

    Solving for the maximum produces a decent set of ratings. (** I'm leaving out one additional detail which I'll discuss in a few paragraphs.) But there were some anomalies. There were some...


    ...can crushers. Not that there's anything wrong with that... Some undefeated guys seemed to be rated a little too high, or maybe some veterans seemed to be rated a little too low. Not a huge deal, but enough to make me take another look at the prior.

    If MMA has an advantage over other sports from a rating perspective, it's in the match-making. Generally fights are between fighters of comparable skill, meaning that just knowing the match-ups (and not knowing the results) tells us something about how the ratings should cluster. The prior used above corresponds to a win and loss to an average fighter, but the rule of succession was meant for an average opponent. Thus to account for match-making, CCR divides each fighter's extra win and loss equally among their opponents, including the two dummies.

    (Dividing it only among their real opponents can lead to unreasonable ratings. For example, suppose a person had only one fight that was a loss to Jon Jones. Jones currently has a rating of 102 and the "average fighter" has a rating of 32. Let's take those to be fixed values for the moment. The original prior corresponded to a win and loss to the dummy, meaning a total record of 0-1 against Jones and 1-1 against the dummy. This translates to a rating of 31.98 -- he doesn't drop much from the Jones loss because he was expected to lose badly. The prior corresponding to a win and loss divided equally among his real opponents means a record of 1-2 against Jones. This translates to a rating of 95, which is good enough to be #6 on the pound-for-pound list! Finally, the prior with the extra win and loss divided equally among his three opponents (Jones and the two dummies) means a record of 0.333-1.333 against Jones and 0.667-0.667 against the dummy. This translates to a rating of 43, which seems to be a reasonable compromise between the other two numbers. (Anthony Pina's actual rating is 43.))

    ** The additional detail: CCR also accounts for a fighter's skill changing through time. Instead of calculating one rating, it calculates a fighter's rating at the time of each fight. Those ratings are connected by a Wiener process (no joke), which depends on a parameter w. If w is very small, then the ratings are nearly constant, meaning that it treats a fighter's skill as constant over their whole career. If w is very large, then the ratings are nearly disconnected from each other, meaning that a fighter's rating is determined by only their last fight. So it's about finding the right balance between the two. Ultimately there's a range of reasonable values, but I settled on w = 6 points/sqrt(year). (This system is called Whole-History Rating, and you can read the paper at that link for more details including the algorithm that's used to determine the ratings.)

    A few other notes: CCR ignores no contests, disqualifications and fights against unknown opponents, but it includes amateur and TUF fights (since those give some indication of a fighter's skill). It counts a draw as 1/2 win and 1/2 loss and a split decision as 2/3 win and 1/3 loss. It scales the natural ratings by a factor of ten and rounds them, because rating differences smaller than that are negligible. (A one point rating difference translates to a win probability of 52.5%-47.5% or moneyline of
     
  3. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Here's a graph of various pound-for-pound ranks, with points calculated monthly. By coincidence the ratings roughly translate to letter grades. You can think of #100 = 75 points being a "C" average (although being #100 in the world is pretty good!). Then #10 is around 90 points or an "A", #1000 is around 60 points or a "D", and so on. It's a bit tricky looking earlier than 1995 because there aren't really 100 active fighters. There's been just six fighters with ratings above 100: Bas Rutten, Fedor Emelianenko, Georges St. Pierre, Anderson Silva, Jon Jones and Chris Weidman.

    [​IMG]

    Here's a graph of nine of the ten fighters that have held the #1 Pound-for-Pound ranking since 1995. I'm omitting Igor Vovchanchyn who became #1 by default for two months at the start of 2001 after Bas dropped from inactivity and Hendo lost to Wanderlei and before Big Nog won the Rings King of Kings Tournament. This is the big dip at that point in the pound-for-pound graph.

    [​IMG]

    You can click the graphs to get a larger view.
     
  4. Sumbog

    Sumbog Green Belt

    Joined:
    Jun 8, 2011
    Messages:
    1,127
    Likes Received:
    1
    Location:
    Canada
    102+93= 195

    In my province they don't allow you to take in crushed cans, they only unaltered cans.

    Some pretty cool numbers to look at I suppose, graphs are always pretty. The issue I have with these number crunching systems is they remove the emotional aspect of sport which I believe is an essential component. You can always compare the rough numerical values, but it is tough to add situational aspects. For example how do you account for the majesty of Mark Coleman winning the 2000 Grand Prix, or in that same era, Sakuraba having beat Royce in a 90 minute match? The numbers are interesting non the less.

    Also sorry for screwing up your nice chain of posts.
     
    Last edited: Jan 3, 2015
  5. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Here are the top fighters in each division. One oddity I'll mention is Robbie Lawler, who's only #7 WW. He just beat #3 Hendricks but it was a split decision, and he lost a unanimous decision to Hendricks not even a year ago. Generally, the winner of a fight comes out with the better rating, but roughly 5% of the time they don't, including about 25% of the time when the result is a split decision. This was one of those times. CCR gives Hendricks a 59% chance (-140) of winning the rubber match.

    [​IMG]

    I plan to do event previews and recaps each week, along with other random stuff I think people might find interesting. Let me know what you think, good or bad. Suggestions are welcome.
     
  6. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Ha, can't put anything past you. :) 101.8 + 92.5 = 194.3, it's just rounding.
     
  7. Taric

    Taric Gold Belt

    Joined:
    Jan 24, 2013
    Messages:
    18,780
    Likes Received:
    17,386
    How the fuck isn't the #1 WW vs the #1 LW, and former #1 WW, even on the list? Terrible.
     
  8. SlicerDM

    SlicerDM Peace is a lie, there is only passion Double Yellow Card

    Joined:
    Mar 18, 2014
    Messages:
    9,942
    Likes Received:
    5,545
    Location:
    The Great White North
    So since I didnt want to read your paragraphs. What are DC's chances of victory tonight my white belt friend who isn't like the other typical fucking retard white belts?
     
  9. aross

    aross Purple Belt

    Joined:
    Nov 1, 2011
    Messages:
    1,856
    Likes Received:
    70
    cool stuff do you have a website
     
  10. WilfordBrimley

    WilfordBrimley Purple Belt

    Joined:
    Sep 8, 2011
    Messages:
    1,964
    Likes Received:
    5
    Location:
    Nunya Bidness
    Can we just go back to talking shit mindlessly?
     
  11. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    GSP vs Penn was 96 + 85 = 185, #14 on the list.
     
  12. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Ah, it was in the second graphic. DC +250 or a 28% chance of winning.
     
  13. dogooder

    dogooder White Belt

    Joined:
    Apr 1, 2013
    Messages:
    85
    Likes Received:
    0
    Not at the moment, but maybe at some point.
     
  14. kurtwpg

    kurtwpg Silver Belt

    Joined:
    Apr 18, 2008
    Messages:
    12,987
    Likes Received:
    696
    That's odd. Conor is 1 point behind the guy who Frankie gets no credit for beating.
     
  15. JCS_FM

    JCS_FM Orange Belt

    Joined:
    Jan 1, 2008
    Messages:
    313
    Likes Received:
    6
    I like your style...

    Fedor/Nog seems tough to overcome.
     
  16. MacDougall

    MacDougall Brown Belt

    Joined:
    May 25, 2013
    Messages:
    3,321
    Likes Received:
    1,577
    Location:
    Brisbane, Australia
    It seems pretty cool but surely the calculation on Lawler is off? He loses one fight to Hendricks in his entire UFC run, beats him, beats MacDonald, Ellenberger, Brown and Koscheck and he's somehow below Askren, Woodley and Lombard who have nowhere near as good wins? Actually neither does MacDonald.

    Certainly has potential though. I like the way you're calculating pound for pound rankings especially.
     
  17. KingNothing148

    KingNothing148 Purple Belt

    Joined:
    Jun 19, 2006
    Messages:
    2,129
    Likes Received:
    22
    That was the first thing I wondered. I would love to dig through some of this stuff throughout the history of the sport. I plan to read the paper you posted soon and I'll hit you up with any questions I have.
     
  18. Clippys Mom

    Clippys Mom Brown Belt Banned

    Joined:
    Jun 6, 2003
    Messages:
    2,759
    Likes Received:
    2,999
    Location:
    The Clinic
    Nice work dogooder,
    Very interesting, its nice to see some heavy content around here.
    Many posters are going to flame you for making them read and think.:icon_lol:

    Also, most sherdoggers are experts in all things, so prepared to be shat on for not having the "theory of everything" nailed down.
     
  19. Zap

    Zap It's clobberin' time.

    Joined:
    Aug 5, 2008
    Messages:
    7,587
    Likes Received:
    4,379
    Location:
    Kuzcotopia
    Too many graphs. You're like Marshal on "How I Met Your Mother".
     
  20. RockyLockridge

    RockyLockridge Red Belt

    Joined:
    Oct 25, 2013
    Messages:
    9,006
    Likes Received:
    4,196
    MMath to the extreme
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.