Friday, January 14, 2011

January 2011: FIFA Ranking vs. Elo Ratings

Top 10 differences for countries with a better Elo Rating:

Team - FIFA Ranking - Elo Rating - Difference

Solomon Islands 178 119 59
Kuwait 99 53 46
Fiji 151 112 39
Papua New Guinea 203 165 38
Wales 113 76 37
Korea DPR 109 73 36
Tahiti 183 148 35
Iraq 98 64 34
New Caledonia 155 121 34
Saudi Arabia 78 45 33

Top 10 differences for countries with a better FIFA Ranking:

Team - FIFA Ranking - Elo Rating - Difference

Algeria 36 94 -58
Botswana 54 102 -48
Armenia 60 105 -45
Gabon 40 84 -44
Puerto Rico 131 175 -44
Dominica 129 168 -39
St. Kitts and Nevis 121 158 -37
Luxembourg 132 167 -35
Grenada 94 128 -34
Antigua and Barbuda 103 137 -34

Top 50 nations only (in the top 50 in at least one of the rankings)

Top 10 differences for countries with a better Elo Rating:

Team - FIFA Ranking - Elo Rating - Difference

Saudi Arabia 78 45 33
China PR 79 48 31
Iran 65 35 30
Poland 72 45 27
Costa Rica 69 43 26
Colombia 48 24 24
Venezuela 60 36 24
Sweden 33 14 19
Ukraine 34 15 19
Ecuador 52 33 19

Top 10 differences for countries with a better FIFA Ranking:

Team - FIFA Ranking - Elo Rating - Difference

Algeria 36 94 -58
Gabon 40 84 -44
Slovakia 20 51 -31
Burkina Faso 42 73 -31
Guinea 46 77 -31
Northern Ireland 43 71 -28
Slovenia 17 40 -23
Tunisia 44 65 -21
Greece 11 27 -16
Nigeria 32 47 -15

Matches I would have liked to see before the next ranking:

Saudi Arabia - Algeria
China PR - Armenia
Iran - Slovakia
Poland - Northern Ireland
Costa Rica - Gabon
Colombia - Slovenia
Venezuela - Botswana
Sweden - Montenegro
Ukraine - Greece
Ecuador - Burkina Faso

About me:

Christian, husband, father x 3, programmer, Romanian. Started the blog in March 2007. Quit in April 2018. You can find me on LinkedIn.

20 comments:

  1. Any idea why ELO rates all those Oceania teams so much higher? ELO is generally considered the better ranking, but I have to agree with FIFA, those teams have no business being that high. Solomon Islands better than half the World???

    ReplyDelete
  2. @ Lorric - A lot has to do with the start rating chosen for the teams. ELO starts new teams off at a round number (a multiple of 100) - which is basically just a guess at how good they are - in the hope that after 30 games, their true strength will be found.

    Now - imagine - the first Oceania teams to play were Australia and New Zealand - if you choose too high a start rating for these teams, then this will make all other Oceania teams seem better than they really are, and their start ratings will also be too high. Because Oceania teams (with the exception of Australia (now Asian) and New Zealand) do not play against teams from other confederations (with a few very rare exceptions), these artificially high start ratings are not 'fixed' by subsequent results.

    There are ways around this, but the ELO system has no means of actually calculating a start rating (based on the team's first results) nor does it have a way of bringing the start ratings down if it is subsequently discovered that they were too high. The only way it would happen is if Oceania teams played against teams from other confederations more often.

    ELO is a good system, and certainly an improvement on FIFA's (I believe FIFA use an ELO system for their women's ranking?), but it needs a few more adjustments to reflect reality a little better.

    ReplyDelete
  3. @mcruic

    Thanks. I believe both FIFA and ELO have characteristics that should be merged. I think FIFA is better with regard to not factoring in margin of victory. Nobody cares if you win 1-0 or 10-0, 3pts is 3pts. For instance I don't like that China got more points for beating Kuwait than Syria got for beating Saudi Arabia because China won by 2 goals. But ELO is better by not punishing teams for playing friendlies that mess up an average score. Personally I think friendlies should not be used for a ranking that determines seedings. FIFA's ranking would be the best if it excluded friendlies. That would take out the issues with those, and the lack of a bias for home advantage would be no problem, as teams always play home and away anyway in qualifying, or neutral in tournaments. Even the tournament host would be alright as it would do as a compensation for their lack of qualifiers.

    ReplyDelete
  4. @ Lorric

    I see quite a few people don't like the margin of victory component of the ELO ratings. I believe it is necessary, however.

    I would say that it DOES matter if you win 1-0 or 10-0. It gives quite a good indicator of the relative strengths of the 2 teams. It just depends HOW you factor in this margin of victory. In the example you mention, you have to remember that Saudi Arabia and Kuwait had similar ELO ratings before their respective matches (KSA 1642, KUW 1625), so in terms of quality, China beat a team of similar standing to Saudi Arabia, and beat them by a better score than Syria did. If China had beaten Kuwait 10-0 and 2-1, wouldn't you say that the 10-0 win was better and more deserving of points?

    Another thing - San Marino are surely not the worst team in the world (according to FIFA - which does not use margin of victory). They lose almost every match, but the reason for this is more to do with the fact that all of their games are against UEFA teams. They would not, for example, lose 31-0 to Australia, or for that matter, 7-0 to Tonga, as American Samoa have in the past.

    The fact that FIFA does not show a difference between American Samoa and San Marino, however, is almost entirely attributable to the fact that no points are given or taken away for defeats. Teams that lose regularly all have zero points, so technically, it would be possible to play and lose 10 matches by one goal against Brazil, Spain, etc. and find yourself on 0 points, in the same position as teams who lost 25-0 to the Tongas and Samoas of the world. The system loses a lot in accuracy due to it NOT factoring in margin of victory.

    In my rankings at roonba.com, I've made an attempt at rectifying perceived problems with the FIFA ranking (and the ELO ranking), and have come up with a hybrid ELO system which I think works quite well.

    The only real test of the strength of various ranking systems would be to do a trial of 6 months or a year, and see which one came out on top when predicting results of matches. As far as I know, this has not been done across the board, and is quite difficult anyway due to the fact that FIFA's is updated monthly, and other rankings are updated after each match.

    The one mistake that FIFA's ranking makes, I believe, is that it is more like a league table with regard to offering 3 points for a win, 1 for a draw, 0 for a defeat. People look to a ranking to see how teams compare strengthwise, and this is why so many people criticise FIFA's ranking - it doesn't conform to what they perceive as the correct order (Egypt in the top 10 for example).

    ReplyDelete
  5. Ah yes, I also think it should be 2pts for a win in a ranking system, not 3.

    I used to agree with the margin of victory thing, but not anymore. I'm seeing a reason why unfold right now, as Japan are currently beating Saudi Arabia 5-0. That will be a monster jump for Japan in the ELO, but the only reason they're winning like that is because Saudi Arabia couldn't care less because they were already out.

    2pts, 1pt, 0pts are all that matter imo.

    I've thought of San Marino myself before, that I bet they could get out of UEFA's bottom pot if they went on a Caribbean or Oceanic tour. But the bottom line is they still have to PROVE it on the field, and they haven't done, so they should be ranked at the bottom. We can't move them up just because of what we think, no matter how sound the thought process, it's a results business.

    I don't want margin of victory used, but I do like ELO's way of redistributing points between teams. I don't think the penalty of losing a match to Argentina should be as harsh as the penalty for losing a match to Albania.

    Japan just completed their 5-0 victory. I wonder where that will put them.

    I didn't have a problem with Egypt's high ranking until they started getting beatby teams they should be wiping the pitch with. 2 African cups and a reasonable performance at the Confederations Cup deserves a strong ranking, Egypt's recent performances, excluding that friendly tournament, do not.

    I don't think a ranking should be good at predicting results, it's primary purpose is deciding seedings, thus it should be as fair as possible. I don't care if the World knows Team X is weaker than a whole bunch of teams if Team X has put together a string of surprise results and earned that spot and the seeding that goes with it.

    ReplyDelete
  6. @ Lorric

    Yes, I can see where you are coming from.

    I see your point about the Japan v Saudi Arabia game - but in that case, why not just ignore the match altogether (including awarding 2 points for a win against a team that does not care)?

    Regarding San Marino - it's fair to argue that they HAVE proved it with their results. It just depends on your point of view with regard to what a defeat constitutes.

    A 4-0 defeat to Spain, for example, is inarguably a more impressive result than a 31-0 defeat to Australia, and certainly constitutes proof of sorts that the former team is likely to be better than the latter. It is certainly no less proof than ranking Team A above Team B because Team A have beaten a poor team, while Team B has 'only' drawn with a good team.

    Also, a team who puts together a string of surprise results is only being perceived to be putting together a string of surprise results because the FIFA rankings are not very good at positioning the teams.

    If their primary purpose is to decide seedings, then they should indeed be based on the current strengths of teams and should do their best to reflect this with their methodology, not to reflect who has had a string of surprise results (which were not a surprise to anyone who actually knows about the actual strengths of football teams), or indeed a simple string of victories against unimpressive teams while other teams were inactive.

    In this way, FIFA's ranking, in its perceived simplicity, actually does nothing more than multiply its own inaccuracies over time.

    I see no reason why ELO or any similar ranking would not provide a much fairer and realistic method to seed teams. FIFA's ranking is not really fair - it is only fair in that it is good at following its own rules - which actually, any system could say. The rules are the problem. The arbitrary figures that are chosen - choosing to take into account the POSITION of the team instead of the number of points they have is rather silly, I believe. The number of points is eminently more important than the position for determining the relationship/difference in quality between 2 teams.

    ReplyDelete
  7. Does it really matter whether San Marino have a couple of more points than American Samoa? Both are amateur and terribly bad at what they're doing. I believe FIFA excluded goals to simplify the rankings and rightly so!
    ELO on the other hand takes into account all matches played since 1872 which is more of an all-time ranking rather than a current ranking which FIFA has...

    ReplyDelete
  8. @ mcruic

    You can't ignore the match. Again, it doesn't matter what we think. All that matters is results. And the cleanest way is simply wins draws and losses.

    Again, I already said taking a loss to a weak team should damage your ranking more than a loss to a strong one. But I don't agree with the margin of victory thing. It's all too easy for a team to just jack it in. Usually a heavy victory involves cheap goals that if that team was playing properly they wouldn't concede. Just keep it simple. A victory is 2pts, a draw 1pt and a loss 0pts. And in case there was any confusion about that in my last message, I mean in the rankings, the ranking points earned are twice as much for a win over a draw, as opposed to the FIFA system of 3X more pts for a win than a draw. And that would surely prevent Team A being ranked above team B for a victory over a weak team while team B tied a strong one.

    I wasn't talking about any specific ranking for surprise results. Let's look to the Asian Cup. Group B is really helping me. Before the competition, I'm sure most of the World, myself included, expected Saudi Arabia and Japan to just walk past Syria and Jordan. Instead, Saudi Arabia have nothing and Jordan have 7pts. That is a string of surprise results, and Jordan should be rewarded. And they will be, since the Asian cup seeds qualifiers by performance in the last one. Jordan will be I believe in pot A for 2015 qualifiers and Saudi Arabia in Pot C. I don't like that particular system, but it is what it is. I just wouldn't want someone saying "Oh Saudi Arabia are still better" just because of the name they've made for themselves in that region. They should pay the price for their dismal performance.

    Same for Greece and their top seeding in qualifiers for a while after Euro 2004. Were there lower seeded teams percieved to be better? Yes. Did Greece deserve to be top seeded? YES. They earned it.

    I already told you my preferred method of ranking teams, although for a WORLD ranking, friendlies might need using after all. You bring up another point I forgot though. I too believe points, not positions, should determine ranking points. My England should not be worth almost as much as Spain for a win. Spain should be worth a LOT more than my England because there's an enormous gap in points between them.

    ReplyDelete
  9. @ Soccer-db.net

    This is a common misconception - ELO includes all results since 1872, but the older results have hardly any effect. The latest results only have a perceivable effect on the rankings - same with FIFA.

    The question of whether it matters about San Marino and American Samoa is not really worth answering - but I will anyway - it matters if you care to include these teams in the ranking. If it doesn't matter, don't include them. Does it matter if Spain has a few more points than Brazil? They are both really good at what they do - can you see now how elitist that argument sounds?

    Simplifying a ranking system is not an argument that holds any weight - it may make it easy to understand, but people only want to understand it because it doesn't seem right. If it was better at doing its job, there would be less need to understand it.

    ReplyDelete
  10. @ mcruic

    Okay, mcruic, I hope you appreciate this, because yesterday I wrote out a big long message, and then it got deleted, ha ha. I hate it when that happens.

    What you just said is like on a computer game online when someone quits out and denies you the victory points. Japan can't be denied their points just because Saudi Arabia decided not to show up. It's not Japan's fault, and they shouldn't be punished for it.

    I've already said I don't think the punishment for losing to an Argentina should be the same as the punishment for losing to an Albania. But also I believe a defeat is a defeat, a draw is a draw, and a win is a win. Score is irrelavant. Too many ways score is not indicative of team strength. Keep it nice and simple. The competitions of the World are seeing this too, ditching goal difference in favour of head to head, which I agree with wholeheartedly.

    Also, I said I believe in 2pts for a win in a ranking system, not 3. That would prevent a win over a weak team being more valuable than a draw with a strong one.

    Surprise results. I'm not talking about any particular ranking system for deciding what was a surprise or not. Greece winning Euro 2004 was a surprise. And they got top seeded for a while in qualifying competitions. Were there teams seeded below Greece that would have been widely regarded as favourites in a match between the two? Yes. Did Greece deserve to be seeded above these teams? YES. They earned it.

    Same for the ever helpful Group B at the Asian Cup. I'm sure most of the World, myself included, expected Japan and Saudi Arabia to walk past Syria and Jordan. Instead, Jordan grabbed 7pts. That to me is a string of surprise results. I wouldn't want someone saying "Oh, Saudi Arabia will still be a top seed, because they're Saudi Arabia." They should be punished for their poor performance. And Jordan rewarded. And they will be. Probably disproportionately too, as the Asian Cup seeds 2015 qualifiers on performance in 2011. I believe Jordan will be in Pot A for 2015 qualifiers and Saudi Arabia in Pot C! There's potential for a nasty group of death in Asian qualifying there...

    Well, I already told you what I think is the best way to rank teams, but you have brought up another point which I forgot about, I too firmly believe points, not positions, should determine points for beating teams. It is not right that my England are worth almost as much points if a team takes them down as Spain. The gap between Spain and England is massive, both in points and in reality. In fact, the top 4 are in a totally different class to everyone else.

    ReplyDelete
  11. @ Lorric

    Again, I can see your points. I think you are right about score not showing strength in certain cases, but in general it does.

    That is why Germany beats San Marino 8-0 or 10-0 or whatever, and not 1-0. If it was not important, then it would not happen so often that teams of a certain strength beat teams of another certain strength by a certain number of goals. Teams (in general) don't just give up when they score 1 goal. The difference in strength between the teams is therefore best shown by the difference in goals scored.

    Team A beats Team B - this shows that Team A is better than Team B, but it doesn't show the degree to which this is true. The whole point of a ranking is to show the degree to which teams are better or worse than others, so they can be put in an ordered list.

    Of course, we are coming at this from different angles, so it's not going to end in agreement. I am advocating a ranking that shows CURRENT PLAYING STRENGTH, while you are advocating something different. I think if FIFA spent as much time as we are debating this, then their ranking would be just fine.

    ReplyDelete
  12. To clarify; I don't think bottom ranking matters. Spain or Brazil will end up in World Cup hence seedings are important. San Marino and American Samoa aren't even in the same confederation. I've always been of opinion any rankings below 50 aren't showing inter-confederation strenght adequatly because the teams rarely or never play teams outside their own confederation..

    ReplyDelete
  13. @ mcruic

    I don't think the score shows difference in strength enough. If you're losing, with little chance of getting anything out of a game, it often maks sense to stop trying, if you have a game in another three days, don't go breaking your neck for foolish pride. I did used to think big victories should be rewarded in a ranking system, but not anymore. You don't even need them, because teams play all sorts all teams, so the rankings will sort themselves out, probably quite well in a good system.

    You are wanting a ranking that displays teams in their true World order. I don't think such a thing is actually possible. The very nature of the game means "wrong" results will occur, teams will get screwed because the greatest sport in the World is still stuck in the stone age. And the sport itself even if officiated by a perfect robot referee still would produce many upsets, it's the nature of the game.

    A ranking based on who deserves to be seeded may cause some eyebrow raisers, and occasionally produce an unfair grouping, but at least you'd be able to track the results and understand why in a fair and logical system.

    soccerdb.net also brings up a good point, ranking systems are pretty good at measuring teams within their own confederation, because they're all playing by the same rules. I don't think a true World Ranking is possible. Maybe not even if we had True WORLD Cup qualifiers.

    I'd like to see a ranking system that decided seedings that all national teams understood, so they understood the importance of fighting for results, even if already eliminated from a competion. I'd like to see it on TV in the build up to the game the pundits talking about how a match is crucial to a team's seeding for the next qualifiers, eg. "Denmark will wrap up the playoff spot here with a draw, and could win the group with a win, depending on the result of the other match today. But Moldova will be assured of a spot in Pot D for World Cup qualifying with a win, and will have a decent chance of getting it with a draw, so expect an exciting game here in Moldova with everything to play for."

    ReplyDelete
  14. @Lorric

    Difference in goals does tend to show difference in strength, as I said. The proof is in the results. Of course you will get shock results - otherwise the ranking would remain the same forever. That's not what I'm arguing for.

    What I'm saying is that a system that includes margin of victory is better at ranking teams. It's not an opinion, if you look at the number of times there have been 'shock' results in ELO and 'shock' results in FIFA's ranking (where lower-ranked teams beat higher-ranked teams) - FIFA's comes off worse (by having more shock results, therefore decreased accuracy at showing true strength)

    What I think should exist is a system that shows changes quickly, and a system that doesn't need 2 years to 'sort itself out'. These systems already exist, and they work much better than FIFA's, that's just a fact. They are just as easy to explain as FIFA's too, they just use different variables as inputs (i.e. - number of goals scored in a match - not such a hard concept for armchair fans to understand).

    A true world ranking IS possible - it's not possible with FIFA's system, but it's possible with other systems that can do it just fine.

    As far as I am concerned, teams should be less concerned about fighting for a simple 1-0 win, and going for as many goals as they can - the basic purpose of football is to score goals - that is what fans go to see. By saying that margin of victory doesn't matter, good teams can just give up at 2-0 and not try to score any more, even though normally they might win 10-0. What fun is that for anybody, and what good is a ranking system that encourages this sort of attitude?

    I think if people just take time to understand other ranking systems, they might warm to them and see that they are just as easy to understand as FIFA's, and make considerably more sense - and see that they promote goalscoring rather than just the notion of win or lose. If you're going to be rewarded for losing 5-1 rather than 5-0, it gives an incentive to keep going. Surely not a bad thing?

    ReplyDelete
  15. Perhaps I'd have less of a problem with goals influencing rankings if they had less of an impact. Personally I think they matter way too much in Elo. I even have a problem with it in the UEFA Coefficient, and the score isn't even that big for banging in goals. But you know whoever gets San Marino will get a leg up on everyone else. I was annoyed that England ended up in a 5 team group and on top of that got Montenegro as 5th seed (not in my wildest dreams could I imagine the consequences of that though...) bottom line, it's 80,000 pts + goals we miss out on in that system. Shouldn't stop England staying a top seed for Euro qualifying, but might impact on seeding in the final tournament if we qualify.

    I prefer ELOs ranking system to FIFAs, though I don't know how you know which one produces more upsets within the system. They both have elements if you ask me that would benefit each other.

    As for change quickly, 2 years is certainly too much time to sort things out, but at the same time, teams deserve some protection for what they have accomplished in the past if they have done well.

    I would never want a team to give up at 2-0 no matter who they were playing, but if up 3-0 against a vastly inferior team, I'd rather they slack off than someone gets their knee busted by some gym worker, trying to go through the defence, or overzealously picks up a yellow card tackling some bricklayer trying to get the ball back to score more goals.

    ReplyDelete
  16. @mcruic

    Yes, FIFA use a slightly adjusted Elo system for the women's ranking.

    I'm also for a ranking that shows the current playing strength and promotes goal scoring.

    A true world ranking IS possible - it's not possible with FIFA's system, but it's possible with other systems that can do it just fine.

    What rankings systems are you referring to? Besides the Roon Ba ranking. Voros McCracken's and the ESPN Soccer Power Index?

    @Lorric

    Yes, Saudi Arabia will be in Pot C for the 2015 AFC Asian Cup preliminaries draw, unless Korea DPR choose not to compete in the qualifiers.

    ... But Moldova will be assured of a spot in Pot D

    I don't think we'll see such things from pundits. Very rarely are rankings (club or national team) covered on TV or in the written press.

    bottom line, it's 80,000 pts + goals we miss out on in that system.

    Those 80,000 points would be averaged with the other points gained. So yes, it would have increased England's total a wee bit, but I'm sure it won't impact the seeding for the final tournament. I'll revisit this when the qualifiers are over.

    ReplyDelete
  17. Hi there Edgar!

    I have a question on DPR, I've seen the Asian Challenge Cup 2012 which they have been drawn into. Do they have the option of going for that, and then if they fail, going for the normal qualifying route?

    Also, I really think the AFC Asian Challenge Cup should be treated as contninetal qualifiers now under the new system, with home and away qualifiers instead of group hosts, and nobody getting fast tracked to the last 8. It now looks like a proper qualifying campaign. I know it's only wikipedia that I have seen, but it also seems to suggest that when it gets down to the last 8, it's still home and away qualifiers. Teams are going to be playing a lot of matches that reach the last 8.

    I agree with your prediction we'll never see pundits talking like that, but I'd still like to see it.

    It would be at the very least 81,002pts with 2 wins. That's an interesting idea of yours, take all the teams that advance out of the 5 team groups, and see how they'd do with an extra 81,002pts. It could be enough to move someone up a pot at Euro 2012.

    ReplyDelete
  18. For a comparison of rankings (in confederations cup 2009) see this:
    http://vorosmccracken.com/?p=134

    ReplyDelete
  19. @Lorric

    Okay, mcruic, I hope you appreciate this, because yesterday I wrote out a big long message, and then it got deleted, ha ha. I hate it when that happens.

    Sorry about that, Lorric. It's Blogger anti-spam system for comments.

    ReplyDelete
  20. @ Edgar

    I don't really understand how that works, but I know it took repeated copy and pasting to get my second message through. It's alright though. I always copy my messages now just in case.

    ReplyDelete