Quick Navigation

Northern Illinois University

Northern News


News Release

Contact: Mark McGowan, Office of Public Affairs
(815) 753-9472

February 22, 2002

Tools available to make judging
of Olympic figure skating fairer

NIU professor says Rasch model
would have awarded Kerrigan gold

DeKalb, Ill. — Figure skaters could benefit from a more fair scoring system if the International Skating Union investigates use of existing measurement methods, says a professor of kinesiology and physical education from Northern Illinois University.

 

Meanwhile, professor Marilyn Looney says, one of the proposed “improvements” to judging of skating actually would prove more detrimental to the fairness of the scoring process.

 

Looney, who teaches measurement and evaluation, believes the Rasch model’s detection and subsequent removal of a judge’s idiosyncrasies and its equalized estimates of a skater’s ability present a fairer picture of a skater’s performance. Looney says Nancy Kerrigan, silver medallist in the 1994 Olympics, would have won gold if the Rasch model were in use.

 

Developed in the 1960s by Danish statistician Georg Rasch, the Rasch model (as expanded upon in 1989 by Mike Linacre, a U.S. statistician) would keep a skater’s ability independent of a judge’s idiosyncrasies and the actual programs skated. Software developed to perform a Rasch analysis detects when a judge gives an unexpected score, such as a good score to a poor skater and vice versa.

 

The skaters’ ability measures are adjusted for unique characteristics of the program skated and the idiosyncrasies (bias) of the judges, says Looney, who uses the word “bias” cautiously. “The Eastern European countries may have a different idea of what is aesthetically pleasing,” she says. “It’s a cultural influence versus a political bias.”

 

But those small differences can cost the deserving skater a gold medal, she says, especially if bloc voting is taking place. Use of the Rasch model may have quelled the controversy that erupted following the pairs results which awarded gold to the Russians and silver to the Canadians, she says.

 

“With a real-time Rasch analysis, the bloc bias which included the French judge might have been identified immediately,” she says.

 

However, the International Skating Union (ISU) also would have to develop decision-making rules to know what to do if the Rasch model is used and idiosyncrasies are detected. “They’re going to have to have some policies in place,” Looney says. “They need to be able to say, ‘We found bloc voting. Now this is the action we will take.’ ”

 

Looney says the ISU’s suggested scoring system only may cause more trouble. It proposes 14 judges — there are nine now — with a random discarding of seven scores.

 

“I did a little exercise in my research methods class last fall. I had 14 students, and we used a random number table to randomly select a sample of six. The sample did not include any male students, although there were five in the class,” she says. “We did the method properly, but we didn’t get a representative sample. My population was very small to begin with, and then we randomly drew a small subset. Using a random selection process does not guarantee that a representative sample of judges from different cultural backgrounds will be selected.”

 

The ISU proposal of seven randomly selected scores also frees judges to smile on their fellow countrymen.

 

“If I know they’re going to randomly select a group of judges, I’ll give my country’s skater a high score because I’m going to play the odds,” she says. “If they don’t select me, I haven’t lost anything. But if they do select me, I’ve really helped my skater.”

 

Looney, whose interest began after Oksana Baiul won the gold medal in 1994, has other recommendations to improve the judging beyond implementation of the Rasch model:

  • Pay the judges and make them more independent from their country’s skating federations. “I’ve heard several people make this suggestion, which is a good one,” she says. “Because they’re volunteers, they’re under a lot of pressure to not upset their federations in order to continue as judges.”
  •  Assign equal weight to the short and long programs. Because the long program counts for more, a judge’s bias in the short program probably does not carry as much weight in the overall outcome. “If you finish in the top three in the short program and you win the long program, you’ll win the gold medal,” she says.
  •  Make tie-breakers consistent so the tie-breaker is the winner of the technical program in both the long and short programs, something Kerrigan has suggested as well.

Looney says real-time bias analysis should be conducted on a trial basis. “We can definitely flag the errant judges, and that’s better than what they’re doing now,” she says. “If they do reprimand someone, it is after the competition is over.”

# # #