| Formulas |
| I would describe a Chessmetrics rating as "a weighted and padded simultaneous performance rating". That may sound complicated, but I think it's pretty easy to understand if you break it down step-by-step. Let's go through it slowly. |
| First of all, what is a "performance rating"? Performance ratings are most commonly used to measure a player's performance in a particular chess tournament. The idea is to estimate how strong a player is, based only on the evidence of their results during that one event. For instance, let's say you score 3 out of 5 in a six-player tournament where your five opponents are rated 2400 (on average). Even for such a simple example, there are several ways to calculate performance rating. Sometimes you're allowed to average the opponents' ratings together, and sometimes you have to treat them one-at-a-time. For me, I like to keep things as straightforward as possible, so I use the simple linear expectancy model that I myself developed, where each 10% increase in percentage score corresponds to an 85 point advantage in rating. The actual formula is: |
| Performance Rating = Average Opponents' Rating + [(PctScore - 0.50) * 850] |
| So in this case, your 60% score (3/5) means you were 85 rating points better than your opponents, and so your performance rating would be 2400+85 = 2485. However, that is a performance rating for a single event. And at the moment we're talking about how to calculate someone's overall rating. |
| So, an overall rating system based upon performance ratings would take a larger set of your games, not just from one tournament, but all of your available games over a particular span. The Professional Ratings system, based upon a formula invented a decade ago by Ken Thompson, uses your last 100 games. For my Chessmetrics ratings, I use a particular span of time, rather than a particular number of games. |
| Before I go any further, I need to stop and define the next word I used in my definition: "weighted". It wouldn't work too well to just take an average of your past 100 games, because there would be too much emphasis placed on the games from way back when, and not as much on your more recent results. So Ken Thompson's formula weighs the games linearly; your last game gets a 100% weighting, the one from 50 games ago gets a 50% weighting, the one from 90 games ago gets a 10% weighting, and so on. Thus the Professional Ratings are easy to classify as a "weighted performance rating". |
| Enough about the Professional Ratings; let's talk about Chessmetrics. The Chessmetrics ratings also use a "weighted performance rating", but it is based upon how many months ago the games were played, going back four years. So a game you played last month would get a 100% weighting, a game you played 24 months ago would get a 50% weighting, a game you played 47 months ago would get a 2% weighting, and so on. I did a lot of analysis on this, using various weighting curves, and various durations. In each case, I actually used the resultant ratings to try and predict the future, to see how well those ratings would have worked at predicting players' upcoming results. Since we have all that data from the past, why not use it to "train" the best rating formula possible? Out of all the options I tried, the best-performing scheme was a very simple one: weighting the months linearly, and using a 48-month span. |
 |
| However, the choice to use performance ratings as your real rating scheme, brings up an interesting question: what if a player didn't play very many games over that time period? Or what if they played a lot of games? One very interesting limitation to the traditional performance rating calculation is that it doesn't matter! You get just as much credit for scoring +1 in two games, as you would for scoring +12 in twenty-four games. You can see how ridiculous this is by looking at tournament crosstables (the ones that show you players' current performance ratings) for players early in a tournament; you will routinely see performance ratings in the 2800's or even 2900's, by players who aren't nearly that strong but who have managed a very high percentage score in their first few games. It's a basic, fundamental flaw in performance ratings, and I haven't ever seen anyone try to fix it! |
| This was not a problem for the Professional Ratings, which always use the same large number of games, but for me it was problematic, especially because I was trying to rate players from long ago, where there wasn't always a lot of game data available. I also kept it in the back of my mind that it would be great if we could improve the performance ratings formula for individual events, too. Traditional performance rating calculations encounter a divide-by-zero error if you have a 100% score. That's a real shame, because one of the big questions about chess history is what the greatest performances of all time were, and obviously three of the top candidates are Bobby Fischer's 6/6 score in his 1971 match against Mark Taimanov, and Bobby Fischer's 6/6 score in his 1971 match against Bent Larsen, and Bobby Fischer's 11/11 score at the 1963-4 U.S. Championships. |
| BONUS: Since you are being so patient and reading through all of this, I'll save you the need to peek ahead. I did solve this problem, with a better performance rating calculation that you can do in your head (if you're good at division). It really works well. And the answer is that Fischer's U.S. Championship performance (against opponents who on average were rated below 2600) was the 48th best of all time, and his performance against Taimanov (rated around 2730 by today's standards) was the 10th best of all time, and his performance against Larsen (rated around 2750 by today's standards) was the 2nd best of all time, a good 12 points below Anatoly Karpov's performance at Linares 1994. |
| Anyway, I wanted to reward the players who played more games, and I also wanted to reward the players who faced tougher opposition. I took the same approach again, and experimented with various modifiers to the basic performance rating formula, to see what worked best at predicting future results. I came up with something that not only works well, but I think it also makes a lot of sense too. Here's what it does: |
| We have already seen what a "weighted performance rating" is. What I actually use is a "weighted and padded performance rating". I take your raw performance rating, achieved across a certain number of games against a certain average opponents' strength. Then I "pad" the results by adding four fake draws against that same level of average opponents' strength. So if you had played against 2400-strength opposition, I would pad your results with an extra 2/4 against that same 2400-strength opposition. Then I give you three fake draws against 2300-level opposition, and because the padding tends to pull down your rating, I add 43 points back in at the end. That's it. |
| One way to understand this approach is to think about it in terms of different guesses about how strong a player someone is. We can look at their raw performance rating, and use that as one possible guess. Or we could look at "the company you keep", and guess that you are about as strong as the opponents you face. Or we could give a nod to the bell curve and say that it's more likely that you're normal, and we can just guess a relatively low number like 2300. And for each of those guesses, there is a weighting given. For the performance rating, it's the number of games you actually played. For the strength of your opponents, the weighting is fixed at 4 games. And for the 2300-guess, the weighting is fixed at 3 games. |
| The really neat thing here is that this exact formula works well both for overall rating calculations and predicting the future, but also for single event results. It kind of makes sense if you think about it. I have this formula that estimates your strength, given a certain quantity of games results. Why not restrict that quantity to just the games from one event? That would tell us our best guess at their strength, when using only the evidence of that one event, which after all is what a tournament performance rating is supposed to tell you, remember? |
| There are several different ways to express this same formula, depending on how you are approaching it, but let's keep it as simple as possible. We already calculated the performance rating before, so now we'll use it in the rating formula: |
| Rating = (PerformanceRating * NumGames) + (AvgOppRating * 4) + (2300 * 3) | + 43 |
| (NumGames + 7) |
|
| If you use this with my 4-year-linear-weighted approach, you get the actual Chessmetrics rating. If you just use this for a single event, you get a Chessmetrics Performance Rating. You can use it with the FIDE ratings too and it works just fine. For example, if you look at Garry Kasparov's last tournament in Linares, he scored 8/12 against average opposition of 2733. Using my raw performance rating calculation, that's a raw performance of 2875 (because a 67% score means you get 142 points above your opponents). We combine the 12 games at a 2875-level, with 4 games at a 2733-level, and 3 games at a 2300-level. Averaging those together, and then adding 43 back in, that comes out to 2797, suggesting that if we knew nothing else about Garry Kasparov's history other than his performance at Linares 2005, we would estimate that his true strength is 2797. Not as incredible as all those 2850+ performances we see in crosstables, but probably a more reasonable estimate, considering the small number of games (statistically speaking). |
| Before I go any further I just want to emphasize a very important point here. This system rewards players who play a lot of games. That's because there are always 7 "fake" games that provide the padding. If you play a lot of games, the 7 games don't have much effect on your rating. If you only play a few games, they can play a dominant role in your rating, as they should! Now think about that four-year time window. Your games are weighted linearly, so maybe a particular game gets a 36% weighting one month, but the next month it is slightly farther back in the past, so it only gets a 34% weighting. This means that as you go further forward in time without playing any additional games, your weighted number of games will get smaller and smaller. And so the padded games, which are typically a much lower performance rating (because of those three 2300-level games), play an increasingly larger role, and the net effect is that your overall rating will slowly go down, at a steady rate. |
| The reason I call attention to this is that it is an extremely important difference between my Chessmetrics ratings and both the FIDE and Professional schemes. In both of those, if you don't play any games, your rating doesn't change. It could be ar |
