The points to exchange P for this match are calculated as K * (W - We)
where
K (or I in FIFA's terminology) is the weight of the match = 5
W is the actual result = 1 (a win for home team)
We is the expected result = 1/(1 + 10^(- (1200 - 1500) / 600)) = 0,2402531
P = 5 * (1 - 0,2402531) = 3,79873
So rating points for team A after this match will be 1200 + 3,79873 = 1203,79873 and rating points for team B after this match will be 1500 - 3,79873 = 1496,20127. The rating points for team A remain unchanged until they play their next match when their 'value before a match' will still be 1203,79873.
Now FIFA can for instance decide to keep the 'value before a match' constant for a team during all matches played by the team between one published ranking and the next published ranking. The provided information is unclear about such details. For now I think they will follow the classic elo-approach as much as possible, so that means an update of the 'value before a match' after each match.
The crux of this system is the starting value to use when a team hasn't played a match calculated with this elo-system yet. In classic elo when a new team is introduced in the ranking it gets an arbitrary starting value assigned, a sort of scientific guess about it's strength compared to the other teams in the ranking. That guessed rating will, according to the formulas above, be automatically adapted with each match the new team plays and after some 30 matches the rating of the new team will be converged to a sort of steady state: 'the real strength' of the new team at that moment. During those 30 matches 'the real strength' of the new team is therefore indeterminate.
Now with FIFA introducing this new elo-like calculation method, they will have to choose starting values for all teams. About the starting values to use FIFA says in it's introductionary document only this:
"One of the main advantages of SUM is that it allows for a smooth transition from the current ranking formula to the new one without displacement of teams in the existing ranking table. The current FIFA / Coca-Cola World Ranking will be replaced seamlessly by the new SUM formula without gain or loss of member association rank positions".
I assume, based on this information, that the starting values will be the June ranking points, but it could be anything really.
These starting values will be applied to the first match each teams plays from June 4th onwards, because June 3rd was the deadline for the published June ranking. All matches played on June 3rd or earlier were incorporated in the June ranking. Matches from June 4th onwards will be calculated with the new method. So that explains partly why I use the June ranking points as starting values.
Other, some smaller, details are also unsure yet:
- FIFA published in June two sets of match type weights K (or I) within two weeks. The second set was a factor 10 smaller than the first set. What are the correct weights for each match type ?
- what is the actual result (W) of a knock-out match that goes into extra time for each team in that match ?
- what is the actual result (W) of a knock-out match that's decided by a penalty shoot-out for each team in that match ?
- what to do with the points of teams that lose a knock-out match in extra time or after penalties? Subtract or not ?
So the resulting August ranking could be anybody's guess really ! We will just have to wait and see what FIFA publishes in August as their first ranking calculated with the new method and just try to deduct what assumptions they decided to apply about all these issues.
Or maybe FIFA publishes a document in the meantime that's 'a bit less vague' about the used method. I don't hold my breath for that to happen, because that would mean real transparancy from FIFA :)
About me:
Software engineer, happily unmarried and non-religious. You won't find me on Twitter or other so called social media. Dutchman, joined the blog in March 2018.
The rankings converge after about 30 games only assuming the following:
ReplyDelete1. The K value is high enough (consider for example a K value of 0.01 - obviously the rankings won't converge)
2. The strength of the teams doesn't change too much during those 30 games - 30 matches for national teams means at least 4 years, so I don't think this assumptions hold for international football.
Therefore any rankings for international football would always be at least one of the following :
1. The rankings reflect past strength, and not the current strength of the teams.
2. The rankings are very volatile and based on luck.
3. The rankings take more than just the actual results of the national teams into consideration.
I think most who have followed along with the new formula during the World Cup agree that the K values are too small. Of course that's based on the assumptions that you (and most of us) have made about the starting point values. I don't know how seriously FIFA or the company who created the formula tested it with past results, but they clearly failed to try the "extremes" like the #1 team going out in the group stage or the #20 team making the final.
ReplyDeleteThere's still the negative points problem too with the teams on and near 0 points in the June 2018 rankings.
I think the ranking system start after every world cup, I mean after 4 years. Example- France win world cup, so France Ranking No. is 1 and Croatia Ranking No. is 2 and Beljium Ranking No. is 3.
ReplyDeleteThe K-values are too low, so I agree with Cody here. If you double the K-factor you would get a much more realistic ranking, which sees Brazil on top, Belgium no.2 and France no.3, calculated with the June ranking points. The top 3 would be separated by just a few points, so for example if France would have won the match against Denmark instead of the obligatory draw, they would have gone on top.
ReplyDeleteSo doubling the K-factor would solve a lot of the problems, but not all and doubling is arbitrary.
And the matter of the points after a knockout-game, that's something FIFA has to be transparent about. If they choose an approach: fine. If they choose another approach: fine also. But FIFA needs to be consistent and transparent about this. Now it's a bit of a black box.
In my calculations for the KO-games, I kept all the losing teams on the same points they had before that game. And a win after PSO, I considered to be a full win (so W=1). Why? Because if you consider a win after PSO to be a draw (W=0.5), a stronger team that wins a PSO could still lose points. Take Russia vs Croatia for example. The win expectancy was much, much higher for Croatia. They won on PSO and when you consider that a draw, they would have lost around 22 pts. which would go to Russia. So in this case, the losing team would gain points, while the winning team would lose points or (at maximum) would stay on the points they had.
Indeed, it's anyone's guess really
If FIFA starts with unchanged rankings they would still have the freedom to start with any set of ranking points that keeps teams in the same order.
ReplyDeleteIf I were running the changeover I would backtest the proposed ELO system and work out the 5 year average ELO points for #1, #2, #3 and so on and then assign that number of ranking points to the #1, #2 and #3 teams per the existing world ranking per the previous system.
Given the imperfection in the previous system I expect the best outcome is for FIFA to artificially bunch initial ranking points for most teams fairly tightly around a median point and then it is much more likely that there will be some rapid early fluctuations in any new system before this decreases over time.
That is much better than the worst case of rankings hardly changing for several years due to the scenarios others have assumed in their previous analysis on this site and elsewhere on the internet.
FIFA has issued a new bulletin about the calculation method for the new ranking. See: https://resources.fifa.com/image/upload/fifa-world-ranking-technical-explanation-revision.pdf?cloudid=p9nvs47hyrmxtlthw9cj
ReplyDeleteThe formula itself and the k- (or i-)factor remain the same, the only thing that is different from the old bulletin is the explanation of the PSO-rules and a clear statement about negative points in knockout-games.
If I read the bulletin correctly and apply it on the results, it seems Germany will be the leader of the August rankings.
Seems odd to me that FIFA needed a whole month to figure out they don't want to change anything to their new calculation method.
After reading the publication a bit better, I've noticed the K-factor HAS changed. It has been scaled down a factor 10. I firmly believe thes means FIFA will have to adjust the ranking points beacause if they use the current points, nothing will change anymore.
DeleteI'm getting more curious by the minute to see what the new ranking will look like.
It seems I guess the 0,75 points for a win in penalty shoot-out, but not after extra-time that will be 1 as a win in regular times
ReplyDelete"Note: In some instances, games end with a winner after regular or extended time, but still include
ReplyDeletea PSO to determine the team that eventually will get to the next round. These games are treated
as normal wins and defeats". No more events like Jordan-Kyrgyzstan.
And another important statement: "Teams that earn negative points in the knock-out round
of a final competition (e.g. as a result of losing, or even winning after PSO against a weaker team)
do not lose any points".
Again, still no details about starting points but I am conviced too that they will not the actual points
Well they're out!
ReplyDeleteThey seem to have changed the starting points to produce a more radically altered rankings post World Cup:
With respect to the introduction of SUM, it was important to transition smoothly from the previous FWR
to the new one. For the initial seeding of teams in the new FWR, teams were evenly distributed over a
range of approximately 800 to 1600 points1
. The point difference between two adjacent teams was set
at 4 points.
https://www.fifa.com/fifa-world-ranking/ranking-table/men/index.html
ReplyDeleteOn first glance it look like they have done a good job. But the devil is in the detail...
ReplyDeleteThey have not done a good job, the importance factors are very small, and in the long run the rankings would be very slow to react.
DeleteIn the short term the starting values are too close and therefor the ranking doesn't reflect the real strength
If FIFA rankings are going to change slowly (and considering how little the Nations League will be valued by FIFA), the next qualification for the World Cup in Europe could be a bloodbath. We could find Germany, Netherlands, Poland, Italy, and Iceland in pot 2, and Serbia, Turkey, Scotland, Greece, and Russia in pot 3.
DeleteI think people are now over-exaggerating when they talk about how slow the rankings will change. It was a problem when 1st and 15th were separated by almost 600 points. FIFA fixed it by adjusting the points.
DeleteFor example, if Germany defeats France in the Nations League, they'll gain 10 points which is enough potentially climb from 15th to 11th in one match.
This comment has been removed by the author.
DeleteIt is enough right now, because the starting values were too close to reflect the reality.
DeleteIn the long run the differences would converge to the real differences and get bigger, so 10 points would be close to nothing.
@cody
DeleteI think most people/websites that criticised the slowness of the new FIFA system had speculated that points would be spread out based on the old rankings points rather than the 4 points used in the actual implementation.
@amir
If you are correct that eventually teams will be spread out and not rise/fall as much from low weighting matches (and I think you are) then the new ranking system will no doubt be criticised for not reflecting current form. Criticism aside, whether or not that is actually a bad thing is interesting given that the old FIFA rankings much more highly than the World Football ELO ratings many sites have long argued is a superior system.
In any case FIFA may end up varying match weights, the 600 point differential formula and/or introduce a goal margin adjustment in the future if the new system is not performing to expectations. I expect that won't happen until 2022 at the earliest so will be interested to see how the next 4 or 5 years play out.
The impact of not losing points in the knockout stage is noticeable. When I applied the past four years of results starting in June 2016 (with the #1 team having 1600 points), we would now have three teams over 1800, four more over 1700, and ten more over 1600.
ReplyDeleteSorry, that should state starting in June 2014. So the final numbers include two World Cups.
DeleteI guess upper ranking figures will grow pretty quickly which makes team comparisons over time very difficult, there should probably be an annual (or monthly) scaling down of all teams to offset this
DeleteWhen applied backwards from 2014, the 1st place team stayed around 1800 points during the last two years of the cycle. That seems to be the balance point of the #1 team much like Elo tops out around 2100 most of the time.
DeleteAs for point growth, over the entire 2014-18 cycle, the total points grew by less than 1%. I'm trying not to fan boy FIFA too much here, but I think they've done a really good job with the formula. If I changed anything, I'd probably make the World Cup have a little more weight (maybe I=70 and I=80).
Cody, you'd need to simulate the model for several World Cup cycles to be able to conclude what happens to the # 1 points in the long run. I can believe that with the simulation you did, the 2018 World Cup wouldn't have affected the # 1 score much because the winning team would have been quite a bit behind pre-tournament. This won't happen in *every* World Cup.
DeleteIn fact, it's not mathematically possible for the # 1 score to top out anywhere because points are regularly fed into the system and never taken out of it. With some luck, enough points might trickle down during every World Cup cycle for the gaps between different positions to stabilize somewhere eventually. I'm not sure if that will happen either but at least it could.
Inflation percentage is quite a meaningless number because zero is an arbitrary thing in Elo. FIFA could have just as well launched the ranking with initial points ranging from 800 to 68. Or from 1,001,600 to 1,000,868. Those would be mathematically equivalent to what we have now but the inflation rate would look very different.
Cody if you have a simulation over the last four years, it could be interesting to know how progressed the ranking of these teams, at the beginning and at the end of the simulation: Wales, Kyrgyzstan, Syria, India
Deletejos,
DeleteHere's a spreadsheet with the simulated rankings at the start of each August between 2014 and 2018.
https://app.box.com/s/jefq8yi8swjbm8izh62jm1quvhws6z55
Many thanks Cody, very interesting. My fears seem confirmed, only the teams taht qualify for main tournaments can significantly modify their situation. The larger part of the World will see only small movements for years. If the new system was applied four years ago, Curacao (currently 81th in this month ranking, would have gone up only at position 164 from the 182th(after winning the caribbean cup against Jamaica and a qualification for the major continental tournament). India (currently at place 96, would be only in position 145. Syria is probably the best example: they started from position 140 and finished only in 103 after finishing in the top 5 asian nations in the WC2018 qualifiers. 4 years and a generation of footballers thrown into the toilet.
DeleteAnd friendlies? Their impact is so insignificant that they could erase them from the calculation, at this point.
Isn't India one of the nations that "hacked" the old system resulting in an over-inflated ranking?
DeleteI think your point about Curacao is fair. One of their problems is that so many of their higher valued games occur against non-FIFA CFU members. I think the better Caribbean nations will be the big winners from the CONCACAF Nations League.
Ok then forget about India and Curacao. Syria 103th goes only 3 position up to Cuba. If India had a "boost" thank to the old system, Cuba was well protected by the new. After early exits from both the last caribbean and WC qualifiers, bad results, they only drop from position 95 to 106. If in the next month Liberia (ranked 158th) ipotetically wins at home against Congo RDC, ranked 36th in the world at this moment, they gain 21,101 points (and only five positions in the current ranking). They need more than ten other similar "shocking" results to finally enter in the top 100. They have to beat the 37th team in the world in a competitive match ten times to have a place in the top 100. And if they will qualify for the CAF cup of nations, if they lose all the three matches in the group stage they probably will see rapidly frustrated the progress in points previous achieved.
DeleteAmir is right, they have done artificially a small interval of 4 points between teams at the beginning, and overestimated the i-constant for World Cup matches (in proportion, in the old system it was 1,6 respect a qualification match, now is 2,0 in the new). So now seems the system works decently, but after some time values will go towards real strengh and movements will become more difficult, expecially in the qualification cycles. In a few days you will see in the african qualifiers in September how the points exchange will be low.
ReplyDeleteAdjust points would be eventually absurd, intricate, unfair and will expose Fifa to criticism for ranking manipulation.
But I think i-constants can be periodically reviewed. After all, the regional strenght was revised every 4 years with the old system, I hope it will be the same in the future with constants because this ranking will become very boring in few time.
However, in my opinion current Elo system never works well in football. It can works well in chess or tennis where you play a lot and you meets the same opponent more times. But in the national team football teams play too rarely, and 30 matches to converge into real strenght is too mutch. Movements are too slow, perhaps a formula that converges after 10 match (about the duration of a qualification cycle) can works better.
A few more things about my simulation of 2014-2018
ReplyDeleteThe number of ranking points in the system started at 254,768
After the 2018 World Cup, the total points in the system was 256,842 (increase of about 0.8%).
If you remove the rule where a team can't lose points in the knockout round, the total points ends up at 255,041 (because a PSO adds points because 0.75 + 0.50 > 1).
I suppose after 3 or 4 World Cup cycles, the increase of points in the system would start to become an issue. A simple fix would be to remove the excess points added to system in the last ranking before the World Cup.
But I'm sure FIFA will overhaul the system the next 12 years anyway.
September probable ranking, I wish to view it now. Please Ed, list the ranking probablity...
ReplyDeleteEd, what approach did FIFA choose in the end - do the rating points used to calculate win expectancy change after each game within the same window, or did they decide to use the fixed points published in the last FIFA rankings throughout the entire international window?
ReplyDelete