The Computer: The phantom judge in figure skating?

If you have watched any of the Olympic figure skating, you have probably heard about the new judging system. There are now 12 judges. Of those 12 judges, 9 of the judges are chosen at random by a computer. Of those 9, the highest and lowest scores are removed. The remaining 7 scored go into judging the competition.

Those familiar with statistics and probability (obviously this would not include the idiots who came up with the scoring system) will notice immediately that this has a critical, unintented flaw. The computer can potentially have a big influence over the final results.

John Emerson, an assistant professor of statistics at Yale, has a very detailed page describing the nature of the flaws. He breaks down a recent figure skating competition that was scored under the new rules and discovered:

Only 50 of the 220 possible panels would have resulted in the same ranking of the skaters following the Short Program. Scores calculated using all of the twelve judges would have resulted in the same ranking, but with slightly different numerical scores.

Random elimination of a different set of judges could have radically changed these standings. Only [the first place finisher's] standing was secure; each of the other skaters could have placed as high as 2nd or as low as 5th in the Short Program. If the scores had been similarly close following the Free Skate (they were not, fortunately), the medal standings would have been determined by the random selection of the panels of judges.

The mathematics behind it all is very interesting – but the jist of it all is that they need to bag the whole random/throw-out-scores thing and just go with the 12 judges.

Scores for the Olympics

He’s updated his page and now provides a breakdown of the Olympic pairs results. There was a 1/8 chance that the 4th place finishers would have had the Bronze medal – based on the selection of the judges. He has provided his Excel files to anyone who wants to run through the calculations for themselves.

Click here to go to his page