PhysioNet/CinC Challenge 2011 FAQ

• What are the deadlines?
• Are the base and gain values the same for all signals?
• Are improperly formatted entries counted?
• What are the best scores?
• How is the computation time incorporated into the scoring function?

To be eligible for one of the major awards, you must:

• Submit a preliminary entry in event 1 no later than noon GMT on Saturday, 30 April 2011.
• Submit an acceptable abstract (about 300 words) about your work on the Challenge to CinC no later than Sunday, 1 May 2011. The abstract deadline will not be extended. was extended by 24 hours to Monday, 2 May 2011. Abstracts that do not include a preliminary event 1 score are unlikely to be accepted. An abstract that briefly introduces the Challenge is here.
• Submit a final entry in at least one event no later than noon GMT on Friday, 5 August 2011. (This has been extended from the originally published deadline of Monday, 1 August, but we will not be able to extend it further.)
• Submit a paper (4 pages) describing your work on the Challenge no later than noon GMT on Thursday, 1 September 2011.
• Attend CinC 2011 (18-21 September 2011 in Hangzhou, China) and discuss your work.

For all records, the base value should be 0 and the gain 200/mV. If you plan to rely on this feature, however, it is best if your software simply checks the input, and normalizes it (i.e., scales and shifts the sample values) only if needed.

Only entries that yield a valid score will be counted against the 5 entry limit for event 1.

Top 10 scores in event 1:

0.932 xzh
0.926 gar
0.920 cth
0.916 die
0.912 gga
0.912 nir
0.908 ire
0.904 seb
0.900 nop
0.896 ben

These scores measure classification accuracy (the fraction of ECGs for which a participant's entry includes a classification that matches the reference classification for that ECG). Only matches contribute to the scores (there is no deduction for an error or a missing classification).

The scores visible on this page prior to 20 July were preliminary scores, because they were based on preliminary reference classifications. Now that the final reference classifications are available, the scores shown above are final scores. The list above shows the final ranking in event 1, since the deadline for submitting additional entries has passed.

The final scores for all three events are here.

Scoring for event 2 is based on accuracy only, although there will be a time limit, which we will enforce only if the submitted code appears to be "stuck". Currently, the time limit has not been set, but (based on the intended application, which is to provide feedback on the quality of an ECG while the patient is still present) the limit is likely to be 30 seconds or a minute per ECG on our reference phone.

Scoring for event 3 will be directly related to computation time (as measured by the Android app framework). You can safely assume that (1) the faster of two entries with the same accuracy will receive a higher event 3 score, and (2) speed will not be able to make up for a significant difference in accuracy (for example, if entry A is half as accurate and fifty times faster than entry B, entry B will receive the higher score).