Between 1984 and 1987, the Association for the Advancement of Medical
Instrumentation (AAMI) sponsored the development of a protocol for the
use of the first two of these databases, which was published as an
AAMI Recommended Practice.
Between
1990 and 1994, the ambulatory ECG subcommittee of the AAMI ECG committee
developed a standard for ambulatory ECG monitors, significant
portions of which address the issue of the accuracy of automated analysis
performed by some of these devices. The ambulatory ECG standard (EC-38)
builds on the evaluation protocol adopted for the earlier Recommended Practice
(ECAR), incorporating provisions for the use of all five of the databases
listed above, with extensions for assessing detection of
supraventricular arrhythmias and transient ischemic ST changes. The
standard breaks new ground in establishing specific reporting requirements
for the performance of automated ECG analyzers on standard tests using
the databases listed above.
A significant constraint imposed on evaluators by the EC-38 standard is that they must obtain annotation files containing the analysis results of the device under test. Although the device itself need not produce these files, EC-38 specifically requires that they be produced by an automated procedure, which must be fully disclosed. The intent of this requirement is to permit reproducible independent evaluations in which neither the proprietary data of the developers (the analysis algorithms) nor that of the evaluators (the test signals and reference annotations) need necessarily to be disclosed. By defining the interface between the developer and the evaluator to be the annotation file, the responsibilities of each party are clearly defined: the developer must make certain that the device's outputs are recorded in the annotation file in the manner intended by the developer, but in the language of the standard; the evaluator must make certain that the algorithms used to compare the device's annotation files with the reference annotation files conform to the specification of the standard. The format and content of these annotation files is specified in detail below. For many existing devices, it may be difficult or impossible to obtain such annotation files without the cooperation of the developers. Newly-designed devices should incorporate the necessary ``hooks'' for producing annotation files.