bxb -r record -a reference-annotator test-annotator [ options ... ] rxr -r record -a reference-annotator test-annotator [ options ... ] mxm -r record -a reference-annotator test-annotator [ options ... ] epic -r record -a reference-annotator test-annotator [ options ... ] sumstats file plotstm file ecgeval nst [ options ... ]
The motivation for developing the MIT and AHA databases was to provide material for evaluating the accuracy of arrhythmia detectors, particularly with respect to ventricular arrhythmias. Between 1984 and 1987, the Association for the Advancement of Medical Instrumentation (AAMI) sponsored the development of a recommended practice (designated ECAR) for using the databases for this purpose. The aim of ECAR is to specify the evaluation methodology in sufficient detail to permit reproducible testing, and to encourage informed comparisons of the performance of ventricular arrhythmia detectors in the analysis of these standard test recordings. More recently, the AAMI has developed, and ANSI has adopted, a standard (designated EC38) for ambulatory electrocardiographs. EC38 specifies standard protocols for evaluating the automated analysis algorithms that are included in many such devices. These protocols include those developed for the earlier recommended practice, and extend them to evaluation of supraventricular arrhythmia and ischemia detection. EC38 specifies the use of `bxb', `rxr', `mxm', and `epic' to perform evaluations, and further specifies the use of the MIT DB (as well as two other databases included on the MIT-BIH Arrhythmia Database CD-ROM), the AHA DB, and (for devices that perform analysis of the ST segment) the ESC DB. If you are interested in this subject, obtain copies of the American National Standard for Ambulatory Electrocardiographs (ANSI/AAMI EC38--1994) and Testing and Reporting Performance Results of Ventricular Arrhythmia Detection Algorithms (AAMI ECAR--1987; see section Sources).
To evaluate an arrhythmia detector using this software, obtain for each
DB record to be used in the test an annotation file containing the
detector's analysis of each beat. These are referred to as the `test'
annotation files (or the `algorithm' annotation files, in EC38 and ECAR).
The placement of the beat annotations must match those in the reference
annotations within 150 msec; thus it is not necessary to place
annotations precisely at the PQ junction (as in the AHA DB reference
annotations) or on the major local extremum (as in the MIT DB reference
annotations). If the detector is capable of shut-down (i.e., if it
inhibits its QRS detection function during periods that it judges are
unreadable), the test annotation files should include a NOISE
annotation with subtyp = -1 at the beginning of each period of
shut-down, and a NOISE annotation with any other subtyp
at the end of each such period. (If the record ends while the detector
is shut down, the annotation file should include a final `end of shut-down'
annotation as above to permit correct shut-down accounting.) If the
detector is capable of ventricular fibrillation detection, the test
annotation files should also include VFON and VFOFF
annotations; it is not necessary to mark flutter waves (use FLWAV
annotations to do so if desired). See the man page for `epic',
in the ECG Database Applications Guide, for information on marking
atrial fibrillation, ischemic ST episodes, and ST deviation measurements
in test annotation files. Any annotations that appear in the
first five minutes of an annotation file are treated as belonging to the
detector's learning period, and are not used in the evaluation. The
evaluation software examines such annotations only to determine the
detector's state (normal, shut down, or in VF) at the beginning of the
test period.
Program `bxb' implements the beat-by-beat comparison algorithm described in ECAR (section 4.3) and in EC38 (section 4.2.14.2.2). By default, the output is in a self-explanatory matrix format. The `-L' option, which must be followed by two file names, specifies that the output of `bxb' should be written in line format, for further processing by `sumstats'. The line-format output includes column headings only if the output file must be created from scratch. In this way, `bxb' can be used repeatedly to build up a line-format tables for multiple records. Among the other options is `-o', which causes `bxb' to generate an output annotation file (with annotator name `bxb') indicating agreements and discrepancies between the input annotators.
`rxr' can be used to performed the run-by-run comparison described in ECAR (section 5.3) and in EC38 (section 4.2.14.2.4). `mxm' compares heart rate, HRV, or other measurements, as described in EC38 (section 4.2.14.2.3). `epic' evaluates VF and AF detection, and ST analysis, as described in EC38 (sections 4.2.14.2.5 and 4.2.14.2.6). These programs also accept a `-L' option to produce line-format output as for `bxb'.
`sumstats' derives the record-by-record, episode-by-episode, and aggregate performance statistics described in ECAR (sections 4.6.1 and 5.6) and EC38 (section 3.2.14.3) from line-format output files produced by `bxb', `rxr', `mxm', and `epic'. The input file must include the column headings so that `sumstats' can recognize the file type. The output includes a copy of the input, with aggregate statistics appended at the end. `plotstm' generates a PostScript scatter plot of ST measurement comparisons gathered by `epic', as described in EC38 (section A.3.2.14.2.6.1).
The easiest way to use these programs is to run `ecgeval', which generates a script (batch) file to run `bxb', `rxr', etc., for each record in a database. See Evaluating ECG Analyzers (in the ECG Database Applications Guide) for details.
By adding noise to annotated ECG records, the noise tolerance of an arrhythmia detector can be measured. This idea was described by the author, along with W.K. Muldrow and R.G. Mark, in "A noise stress test for arrhythmia detectors", Computers in Cardiology 11:381-384 (1984). Program `nst' adds calibrated amounts of noise to ECGs (or other signals), generating an output record in DB format. `nst' was used to generate the graded series of noisy ECG records in the `nstdb' directory of the MIT-BIH Arrhythmia Database CD-ROM. These records are among those specified as standard test material by EC38 (section 3.2.14.2).
Go to the first, previous, next, last section, table of contents.