A challenge from PhysioNet and Computers in Cardiology 2004
| 11 April 2004: The autoscorer is now available. Submit your entry using the autoscorer and receive your score by return email within a few minutes. Top scores are also available and are continously updated by the autoscorer. |
| 30 April 2004: A set of unaudited beat annotations has been posted for record a04 of test set A. These were inadvertently omitted from the original posting of the challenge database. |
| 1 May 2004: Twenty particpants submitted initial results for scoring before the first deadline passed. If your initial results were received before the first deadline, any results you submit before the final deadline of 14 September can be used to improve your standing. If you missed the first deadline, you may still submit results for unofficial scores. |
|
Papers presented by participants in this challenge are now available. Read about the winners of this challenge here! |
Introduction
The fifth annual PhysioNet/Computers in Cardiology Challenge focusses on this question:
Is it possible to predict if (or when) an episode of atrial fibrillation will end spontaneously?
Atrial fibrillation (AF) is the most common serious cardiac arrhythmia, affecting more than two million people in the US alone. Unlike venticular fibrillation, which is invariably fatal if it is not interrupted, it is possible for atrial fibrillation to be sustained indefinitely, since the ventricles continue to perform the essential function of driving the circulation, albeit inefficiently. The risks of sustained atrial fibrillation are nevertheless serious, and include strokes and myocardial infarctions caused by the formation of blood clots within stagnant volumes in the atria. Evidence suggests that spontaneously terminating (paroxysmal) atrial fibrillation, or PAF, is a precursor to the development of sustained AF.
Although spontaneously terminating episodes of AF are often very short (perhaps a few seconds in duration), it is interesting to note that longer episodes lasting several minutes also occur. These appear to be very similar to sustained (non-terminating) AF. Subtle changes in rhythm during the final minutes or seconds of such episodes may lead to (or predict) termination of AF. Improved understanding of the mechanisms of spontaneous termination of atrial fibrillation may lead to improvements in treatment of sustained AF. If it were possible to recognize the conditions under which PAF is likely to self-terminate, it might also be possible to intervene in affected individuals to increase the likelihood of self-termination of what would otherwise be sustained AF.
Organization of the Challenge
The fifth in our annual series of challenges was announced on 23 September 2003 at Computers in Cardiology in Chalkidiki, Greece. At that time, we posted a collection of 80 digitized ECG recordings, the AF Termination Challenge Database, containing labelled training data and unlabelled test data, to support this challenge. To enter the challenge, you will need to:
- develop methods to classify the unlabelled test data
- send your classifications to PhysioNet for scoring no later than Saturday, 1 May 2004 at noon GMT
- send an abstract describing your methods and discussing your findings to Computers in Cardiology no later than 7 May 2004
- (optionally) send up to 4 revised entries per event (see below), no later than Tuesday, 14 September 2004 at noon GMT
- (optionally) submit the source code for your classifier, no later than Wednesday, 15 September 2004 at noon GMT.
If your abstract is accepted, you will be expected to prepare a four-page manuscript (due on Tuesday, 21 September 2004) for publication in the conference proceedings, and you will have the opportunity to discuss your work at the conference. To be eligible for an award, you must submit an abstract and attend the conference.
We invite you to submit the source code for your classifier for possible posting on PhysioNet. One of PhysioNet's major goals is to foster the creation and free dissemination of high-quality software for research on clinically and scientifically interesting subjects. Software contributed in the course of previous challenges has stimulated new collaborations among its authors, and offers rare opportunities to compare the strengths of varied approaches objectively. We will select well-constructed submissions and will post them with full credit to their authors on PhysioNet. We encourage you to participate in this activity as part of the challenge, and we offer additional awards to the authors of the most successful algorithms submitted. A selection of these algorithms will be posted on PhysioNet following the conference.
As in most of our previous challenges, there are two events, and you are welcome to participate in either or both of them. Up to four awards of US$250 will be presented during a plenary session of Computers in Cardiology in September. The top-scoring particpant in each event will receive an award of US$250, and the top-scoring participant among those who have submitted the source code for their classifiers in each event will receive an award of US$250. Qualified participants may receive more than one award.
Although your initial classifications are due by 1 May 2004, you may attempt to improve your results by submitting a limited number of revised entries, until the final deadline of Wednesday, 15 September 2004. The participants who have achieved the best scores on or before the deadline are the winners of each event.
The challenge database
The 80 recordings in the AF Termination Challenge Database are each one minute in length (excerpted from longer recordings),and each contains two simultaneously recorded ECG signals. The cardiac rhythm is atrial fibrillation in each case. QRS annotations produced by an automated detector are included for the convenience of those who may wish to study the interbeat interval time series rather than (or in addition to) the ECG signals themselves; note, however, that these annotation sets are unaudited and contain small numbers of errors. Each of the 80 records belongs to one of three groups:
- Group N: non-terminating AF (defined as AF that was not observed to have terminated for the duration of the long-term recording, at least an hour following the segment).
- Group S: AF that terminates one minute after the end of the record.
- Group T: AF that terminates immediately (within one second) after the end of the record.
| Group N | Group S | Group T | |
|---|---|---|---|
| Learning set | n01, n02, ... n10 | s01, s02, ... s10 | t01, t02, ... t10 |
| Test set A | about 15 records | - | about 15 records |
| Test set B | - | 10 records | 10 records |
How to enter
- Develop an algorithm for classifying the recordings in test set A or B (or both). The algorithm must perform this task unaided (manual and semiautomated methods are not eligible).
- Your entry needs to be prepared in a special text format. For event 1, download this template, and replace the "?" characters with your classifications ('N' or 'T') for each of the 30 records in test set A. For event 2, download this template, and replace the "?" characters with your classifications ('S' or 'T') for each of the 20 records in test set B.
- Follow the instructions on the entry form to submit your entry. You will receive your score by return email. If your score is among the top scores, it will be among those listed on the top scores page.
- Write and submit an abstract describing your work following the instructions on the Computers in Cardiology web site. Please select "Computers in Cardiology Challenge" as the topic of your abstract, so that it can be identified easily by the abstract review committee.
- (Optional) Send the sources for your classifier by email to PhysioNet. Use the subject line "Challenge 2004 entry
source", and be sure to include:
- Your name and email address
- All sources needed to produce a working version of your classifier (except for readily available standard libraries and header files)
- A note describing how to produce a working version of your classifier (a commented Makefile is ideal), and how to run your classifier
Important dates
Late submissions will not be accepted.
- Saturday, 1 May 2004, noon GMT
- Deadline for submission of initial entries to PhysioNet. Please don't wait until the last minute! If you miss this deadline, we encourage you to participate unofficially (your classifications will be scored, but will not be eligible for an award).
- Friday, 7 May 2004
- Deadline for submission of abstracts for Computers in Cardiology 2004.
- Tuesday, 14 September 2004, noon GMT
- Deadline for submission of final entries to PhysioNet.
- Sunday-Wednesday, 19-22 September 2004
- Computers in Cardiology, Chicago, Illinois.
Frequently Asked Questions
If your question is not answered below, please consult the PhysioNet FAQ.How do I get a password for submitting my entry?
If you have not registered your email address, if you do not have a password, or if you have forgotten your password, please go to the sign-in page to register your address and to obtain a new password. Be sure that the email address you use for your entry matches the one that you supplied when you signed in.
Why did the autoscorer reject my entry?
Valid entries must be in plain text format, as in the templates (see the links above). Don't submit HTML documents, MS Word .doc files, or anything else except plain text; the autoscorer won't like it!
Valid entries must also include a classification for each record in the event that you are entering. There are 30 records in test set A (event 1) and 20 in test set B (event 2). Incomplete entries are rejected.
For each event, you may submit up to five valid entries; any further entries in that event are invalid and will be rejected. Only your top-scoring entry in each event determines your standing.
But I can get five more entries using my friend's email address!
The autoscorer won't recognize that ... but the challenge organizers will. Please respect the spirit of the challenge. As we have advised in previous challenges, if you are tempted to submit many entries in order to discover the correct classifications, try playing Mastermind instead!
How are the scores determined?
The score is the number of correct classifications (so a higher score is always better). The maximum possible scores are 30 for event 1 and 20 for event 2. If there is a tie in any event, the award will go to the first participant to submit a top-scoring entry in that event.
Several records appear to include segments that do not appear to be AF. Is there really AF throughout?
The segments were chosen very carefully and with reference to the entire 24-hour recordings from which they were extracted. In a few cases, there are segments with the appearance of low atrial ectopic rhythm that are in fact AF; this appears to be the case in s09 and t09 (from the learning set). Records a24 (from test set A) and b06 (from test set B) begin with sinus rhythm. Record b09 (in test set B) does not contain sinus rhythm.
Can I enter the challenge using a semi-automated method?
You are welcome to participate unofficially (by submitting results for scoring and by submitting an abstract to Computers in Cardiology), but semi-automated methods are not eligible for awards. Please send a brief note to let us know what you are doing.
Why don't you have a challenge about ...?
Each year, we receive many suggestions for challenge topics. We encourage you to write to us with further suggestions.