PhysioNet/Computing in Cardiology Challenge 2012: Top scores

The new PhysioNet website is available at: https://physionet.org. We welcome your feedback.

Following the final deadline for submission of entries on 25 August 2012, each participant selected (or allowed the organizers to select) a previously submitted entry to be evaluated using Challenge test set C, which was used for validation only and not made available to participants. Scores obtained in the set C evaluations determined the final ranks shown below.

Event 1 (binary prediction of survival or in-hospital death)

ParticipantScore
Alistair Johnson, Nic Dunkley, Louis Mayaud, Athanasios Tsanas, Andrew Kramer, Gari Clifford0.5353
Luca Citi, Riccardo Barbieri0.5345
Srinivasan Vairavan, Larry Eshelman, Syed Haider, Abigail Flower, Adam Seiver0.5009
Martin Macas, Michal Huptych, Jakub Kuzilek0.4928
Henian Xia, Brian Daley, Adam Petrie, Xiaopeng Zhao0.4923
Steven L Hamilton, James R Hamilton 0.4872
Natalia M Arzeno, Joyce C Ho, Cheng H Lee0.4821
Chih-Chun Chia, Gyemin Lee, Zahi Karam, Alexander Van Esbroeck, Sean McMillan, Ilan Rubinfeld, Zeeshan Syed0.4564
Alexandros Pantelopoulos0.4544
Deep Bera, Mithun Manjnath Nayak0.4513
Sample and random predictors
SAPS-I (in m-code)0.3125
SAPS-I (in C)0.3097
86% randomly predicted to survive0.1386

The top 10 (of 28) participants in event 1 are listed above. The score is the smaller of sensitivity (the fraction of in-hospital deaths that are predicted) and positive predictivity (the fraction of predictions of in-hospital deaths that are correct). The range is 0 (worst) to 1 (ideal).

Event 2 (estimation of in-hospital mortality risk)

ParticipantScore
Luca Citi, Riccardo Barbieri17.88
Tongbi Kang, Yilun Su, Lianying Ji20.58
Martin Macas, Michal Huptych, Jakub Kuzilek24.70
Chidube Ezeozue24.93
Alistair Johnson, Nic Dunkley, Louis Mayaud, Athanasios Tsanas, Andrew Kramer, Gari Clifford29.86
Mehmet Kayaalp36.38
Tom Pollard, Demetrio Martinez38.23
Deep Bera, Mithun Manjnath Nayak45.01
Antonio Bosnjak, Guillermo Montilla48.61
Natalia M Arzeno, Joyce C Ho, Cheng H Lee51.69
Sample and random predictors
SAPS-I (in m-code)68.58
SAPS-I (in C)35.21
86% randomly predicted to survive10137.7

The top 10 (of 23) participants in event 2 are listed above; 5 participants in event 1 did not enter event 2. The score is a range-normalized Hosmer-Lemeshow H statistic, which approaches an ideal value of 0 as risk is estimated more accurately and over a broader range; see Challenge scoring for details.