Computers in Cardiology Challenge 2001 Top Scores (Final)
The top scorers in the 2001 challenge were announced during the 25 September plenary session of Computers in Cardiology in Rotterdam. The top score and the award in event 1 was obtained by Günther Schreier and colleagues of the Austrian Research Centers Seibersdorf (Graz, Austria). In event 2, the top score was obtained by Wei Zong and colleagues at the Harvard-MIT Division of Health Sciences and Technology (Cambridge, Massachusetts, USA); because this entry came from one of the PhysioNet core research groups, however, it was unofficial, and the award in event 2 was given to the team with the highest official score, who were once again Günther Schreier and colleagues. We congratulate and thank all of the participants in this challenge.
Wei Zong and Günther Schreier receive congratulations from challenge organizer George Moody at CinC 2001.
(Photo courtesy of Bill Sanders.)
The immediate goal of the Computers in Cardiology Challenge 2001 was to develop a fully automated method to predict the onset of paroxysmal atrial fibrillation/flutter (PAF), based on the ECG prior to the event. Entrants developed and tested methods using a database created for this challenge.
The test set of the PAF Prediction Challenge Database consists of 50 pairs of half-hour ECG recordings. Each pair of recordings is obtained from a single 24-hour ECG. Subjects in group A experienced PAF; for these subjects, one recording ends just before the onset of PAF, and the other recording is distant in time from any PAF (there is no PAF within 45 minutes before or after the excerpt). Subjects in group N do not have PAF; in these, the times of the recordings have been chosen at random. Entrants were told that between 20 and 30 of the 50 subjects belong to each group; the exact size of each group (28 in group A, 22 in group N) was disclosed only after the conclusion of the challenge in September.
Brief descriptions of the methods used can be viewed by following the links in the tables below to abstracts submitted by many of the entrants for presentation at Computers in Cardiology 2001; please note that these abstracts were written no later than May 2001, and do not mention results achieved since then. For details on the CinC Challenge 2001, follow the links at the bottom of this page.
Event 1 (PAF Screening)
Event 1 was intended to determine if subjects in group A can be distinguished from those in group N. (In other words, can individuals at risk of PAF be identified within a larger population, based on their ECGs?) The number of correctly classified subjects (0 to 50) is the event 1 score. The best methods were able to achieve roughly 80% classification accuracy on the test set.
The top scores in event 1 are:
| Score | Entrant | Date | Entries |
|---|---|---|---|
| 41/50 82% |
G Schreier, P Kastner, and W Marko Austrian Research Centers Seibersdorf, Graz, Austria |
17 September | 8 |
| 40/50 80% |
W Zong and RG Mark Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA, USA (unofficial entry) |
12 September | 7 |
| 37/50 74% |
R Sweeney and colleagues Guidant Corp., St. Paul, MN, USA |
8 May | 3 |
| 36/50 72% |
C Maier, M Bauch, and H Dickhaus University of Applied Sciences, Heilbronn, Germany |
19 September | 2 |
| 35/50 70% |
C Marchesi and M Paoletti Università di Firenze, Firenze, Italy |
27 April | 1 |
| 34/50 68% |
KS Lynn and HD Chiang Cornell University, Ithaca, NY, USA |
28 April | 6 |
| 33/50 66% |
CC Yang National Yang-Ming University, Taipei, Taiwan |
21 April | 4 |
| 33/50 66% |
JA Kors Erasmus University, Rotterdam, The Netherlands |
10 July | 2 |
| 32/50 64% |
P de Chazal and C Heneghan University of New South Wales, Sydney, Australia |
13 September | 1 |
| 32/50 64% |
R Loesch | 14 September | 6 |
Each entrant's best score is shown, along with the date when they achieved that score. Many entrants submitted multiple entries, and the 'Entries' shown indicate how many entries were submitted by each entrant up to and including the one that scored highest (later entries, and entries that did not receive scores because of formatting errors were not counted); this gives some sense of how much 'tuning' may have taken place. The entry noted as 'unofficial' came from one of the PhysioNet core research groups, and was therefore not eligible for awards, although the entrant followed all of the rules of the competition.
Event 2 (PAF Prediction)
Event 2 was intended to determine if subjects in group A have distinctive and detectable changes in their ECGs immediately before PAF. (In other words, is the imminent onset of PAF predictable in an individual known to be at risk of PAF?) A successful method for doing so should be able to determine which record of each pair of group A records immediately precedes PAF. If the identities of the group A records were known, it would be sufficient to classify these records only; since the goal of event 1 was to identify group A, however, we did not provide this information! Entrants in event 2 of the challenge therefore were required to classify exactly one of each pair of records in the test set as `A' (defined as ``immediately preceding PAF, if the patient belongs to group A''), and the other as `N' (defined as ``not immediately preceding PAF''). One point was awarded for each correctly classified record pair, so that the raw event 2 scores range from 22 to 50 (the lower bound is 22, the number of subjects in group N, because the group N subjects are always considered correctly classified). In the table below, the scores have been adjusted by subtraction of 22 from the raw scores, so that the adjusted scores can range between 0 and 28.
The top scores in event 2 are:
| Score | Entrant | Date | Entries |
|---|---|---|---|
| 22/28 79% |
W Zong and RG Mark Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA, USA (unofficial entry) |
1 May | 1 |
| 20/28 71% |
G Schreier, P Kastner, and W Marko Austrian Research Centers Seibersdorf, Graz, Austria |
19 August | 2 |
| 19/28 68% |
P de Chazal and C Heneghan University of New South Wales, Sydney, Australia |
28 April | 1 |
| 19/28 68% |
C Maier, M Bauch, and H Dickhaus University of Applied Sciences, Heilbronn, Germany |
11 September | 3 |
| 18/28 64% |
KS Lynn and HD Chiang Cornell University, Ithaca, NY, USA |
29 April | 2 |
| 17/28 61% |
P Langley, D di Bernardo, J Allen, E Bowers, F Smith, S Vecchietti,
and A Murray Freeman Hospital, Newcastle upon Tyne, UK |
30 April | 1 |
| 17/28 61% |
D Gamberger and T Smuc Rudjer Boskovic Institute, Zagreb, Croatia |
23 August | 2 |
| 16/28 57% |
CC Yang National Yang-Ming University, Taipei, Taiwan |
23 April | 1 |
| 16/28 57% |
R Sweeney and colleagues Guidant Corp., St. Paul, MN, USA |
8 May | 1 |
| 15/28 54% |
L Almarro UPV, Valencia, Spain |
30 April | 1 |
As in event 1, each entrant's best score is shown above, along with the date it was achieved and the number of entries submitted (excluding any entries submitted after the one that received the best score, and any that were not scored because of formatting errors).
Links:
- Read about the Computers in Cardiology Challenge 2001.
- View or download the training and test data sets.
- Although the competition has ended, you may still submit your results for (unofficial) scoring.
- The correct classifications for event 1 are available here, and those for event 2 are here. (But wait! Are you sure you don't want to try the challenge first?)
- Read about Computers in Cardiology 2001, which was held in Rotterdam, 23-26 September 2001.
- Read about last year's highly successful Computers in Cardiology Challenge 2000.
- What should be the topic of next year's challenge? Do you have a data set that can help in creating a challenge? Please send us your ideas!