How do you quantify skill in a game? There are mathematical methods to evaluate the decision space, that work for abstract games like Chess or Go, but there is nothing for modern board games.
Board Game Arena's ELO ranking system rewards consistency. If you play a lot of games and win a lot, you will have a high ELO. If you assume that the high numbers of players at BGA evens out ability, you can correlate the high ELOs of top-ranked players with high skill ceilings of the games. In other words, high skill games will give big rewards to the highest skill players. A classic example might be chess - Magnus Carlsen is never going to lose to an average club player. On the other hand, no one above the age of 8 is going to lose a game of tic-tac-toe.
To test my hypothesis, I looked at the ELO of the top ranked player for the top 50 most played games at BGA. In general the most obviously skillful games mapped to the highest ranks. For example, Hive (ELO 1015) and Azul (ELO 1005) are abstracts with lots of tough choices and consequences. Also the most obvious luck-fests mapped to the lowest ranks, for example LLAMA (ELO 402) or King of Tokyo (ELO 476)
There were a few surprising outliers. Earth (ELO 539) and Viticulture (ELO 576) are both proper euros with play times over an hour, but the top players have relatively low ELOs. Patchwork is a 15 -minute pentomino game, but the top player has an ELO of 975.
It seems that high ELOs correlate to shorter games. eg. Patchwork or Azul, punishing slower games like Ark Nova (ELO 728) or Terra Mystica (ELO 740), but Tzolkin (ELO 961) is long and does well.
Thousands of people are playing these games to win and rank up every day so there must be some truth in this. Some games have gone up in my estimation and some have gone down.
What do you think? Are you surprised at some of these results?
[Note for the pedants. BGA calls it ELO, but the rating method was created by Arpad Elo, a Hungarian-American physics professor.]