file: NEWS.TXT G. Moody 16 February 1996 Last revised: 23 April 1997 Recent changes in the DB Software Package Copyright (C) Massachusetts Institute of Technology 1997. All rights reserved. This file summarizes changes made to the DB Software Package between the first release of the current major version and that of the current minor version. For less recent changes, refer to `OLDNEWS.TXT'. For more recent changes, refer to `NEWNEWS.TXT'. The first release of the current major version was 9.0. The first release of the current minor version is 9.7.0. ======== Contents ======== A. Changes in the DB library 9.7: Win32 DLL support, record name in DB path, writing annotations out-of-order 9.6.2: Changes in Macintosh handling of ISO 9660 CD-ROMs 9.6.1: LINK annotations 9.6: better handling of DB_HIGHRES for multi-frequency records new function getspf() 9.5: DB_MAXSIG increased to 32 9.4.2: support for use of the DB library with Fortran programs added 9.4.1: bug fix in setheader 9.4: new functions dbgetskew, dbsetskew, dbgetstart, dbsetstart newheader and setheader can now write skew and byte offset fields added version reporting to dberror 9.3.2: bug fixes in db_addtopath, several functions in signal.c, more memory for `aux' strings in getann 9.3.1: bug fix in isgsettime 9.3: new MS Windows DLL, portability improvements 9.2: support for skew correction 9.1: new functions setmsheader(), iannclose(), oannclose() support for multi-segment records support for null records B. Changes in DB applications app: new applications dbcollate, ihr, mrgann, skewedit, sortann, sqrs125, wrsamp bug fixes in dbdesc, epic, mxm, pschart, psfd, tach, xform new features in dbdesc, pschart, psfd, rdsamp, tach psd: a new set of applications for power spectral density estimation C. Notes on reading multi-segment records D. Notes on annotation of multi-frequency records E. Notes on out-of-order annotations F. Documentation ============================ A. Changes in the DB library ============================ Changes in version 9.7 ---------------------- lib/annot.c, lib/db.h, lib/dbio.c, lib/dblib.h, lib/signal.c Support for compiling the DB library as a 32-bit MS Windows DLL has been added (thanks to Bob Farrell). Define the symbols `_WIN32' and `_WINDLL' when building the DLL, and define the symbol `_WIN32' when compiling an application that uses the DLL. The string `%r' is replaced by the record name wherever it appears in the DB path. For example, if the DB path is `:/cdrom/mimicdb/%r:/cdrom/mitdb', a request to read the header file for record 055 will cause the DB library to search in the current directory (since the DB path begins with an empty component), then in `/cdrom/mimicdb/055', and then in `/cdrom/mitdb'. It is now possible to write annotations out-of-order using `putann'. If this is done, `dbquit' (or `oannsettime') rewrites the completed annotation file in order using the new `sortann' application (see below), unless the environment variable DBNOSORT is set. A pair of bugs in the internal function `readheader' (in signal.c) caused problems when multi-segment records were closed (using dbquit) and reopened (for example, in WAVE when using the `Reload' button with such records). These bugs have been fixed. Changes in version 9.6.2 ------------------------ lib/dbio.c, lib/dblib.h If compiling the DB library on a Macintosh, define the symbol FIXISOCD if "ISO 9660 File Access" is pre-version 5.0. Changes in version 9.6.1 ------------------------ lib/annot.c, lib/ecgcodes.h, lib/ecgmap.h A new annotation type, LINK, has been defined, with the mnemonic `@'. The aux field of a LINK annotation should contain a URL in the form used by Web browsers. LINK annotations allow extended text, tables, images, and other data to be associated with an annotation file. If the aux field contains any whitespace, the URL is assumed to end at the first whitespace character, and anything following the whitespace is assumed to be descriptive text to be displayed by a DB browser such as WAVE. Changes in version 9.6 ---------------------- lib/signal.c A new function, getspf, reports the number of samples per signal per frame returned by getvec. In normal operation, this is 1, but while reading a multi-frequency record in getvec's high-resolution mode (i.e., after calling setgvmode(DB_HIGHRES)), getvec returns 2 or more samples per signal per frame. A number of changes in signal.c and annot.c now make use of high-resolution mode almost completely transparent to DB applications. The recommended practice is to call setgvmode(DB_HIGHRES) *before* calling annopen, dbinit, getvec, sampfreq, strtim, or timstr. By doing so, all DB_Time data visible to the application are in units of the high-resolution sampling intervals. Note that the resolution of annotation times in annotation files created while in high-resolution mode remains that of the low-resolution sampling intervals. This may change in a future version of the DB library. Changes in release 9.5.0 ------------------------ lib/db.h DB_MAXSIG, the maximum number of input or output signals that can be open at once, has been increased from 16 to 32. Changes in release 9.4.2 ------------------------ fortran/ A set of wrapper functions that permit use of the DB library with Fortran programs is now provided in the `fortran' subdirectory of the DB Software Package. The directory includes the wrappers themselves (dbf.c, written in C for use with Fortran programs), a sample Fortran program that uses the wrappers (example.f), a `make' description file for compiling the sample program and the wrappers, and a `README' file with brief notes on using the wrappers. Changes in release 9.4.1 ------------------------ lib/signal.c In release 9.4.0 only, DB library function setheader() did not properly interpret the skew setting made by dbsetskew() in the case of an oversampled signal in a multi-frequency record. This has been corrected. Changes in version 9.4 ---------------------- Four new functions were added to the DB library to simplify management of records with signal files containing prologs or intersignal skews. These functions are not documented in the ninth edition (July 1995) of the ECG Database Programmer's Guide. The functions are: int dbgetskew(DB_Signal s) returns the skew (in frames) for input signal s void dbsetskew(DB_Signal s, int skew) sets the skew (in frames) for signal s (see next paragraph) long dbgetstart(DB_Signal s) returns the byte offset of sample 0 of input signal s within its signal file void dbsetstart(DB_Signal s, long bytes) sets the byte offset for signal s (see next paragraph) newheader() and setheader() now write header files containing skew and byte offset fields if you have previously set non-zero values for these parameters using dbsetskew() or dbsetstart(). Note that dbsetskew() and dbsetstart() affect only the values to be written by newheader and setheader(); they cannot be used to modify how getframe() deskews the input signals or how isigsettime() calculates byte offsets. If dberror is invoked before any errors have occurred, the character string it returns now contains the DB library version number. Versions earlier than 9.4 returned an empty string in this case. Changes in version 9.3.2 ------------------------ A bug in the internal function db_addtopath (in dbio.c) was fixed. The bug caused unnecessary computation but did not produce incorrect results. Bugs were fixed in several functions in signal.c, all relating to the use of free() to release storage that, in rare circumstances, may not have been allocated in the first place. None of the standard applications are known to have been affected by these bugs. Memory for `aux' strings in getann is now allocated separately for each annotation file, so that reading many consecutive annotations from one annotation file will not affect the most recent annotation read from another annotation file. Thanks to Rashmi Sarlashkar for this suggestion. In signal.c, a few Linux-specific lines that had been inserted in versions 9.3 and 9.3.1 were found to be unnecessary and were removed. Changes in version 9.3.1 ------------------------ A bug in isgsettime caused incorrect results when searching for an odd-numbered sample in the second or later segment of a multi-segment record written in format 212, if the number of samples per frame was also odd (unfortunately, not as far-fetched a combination of circumstances as it sounds). This bug has been fixed. (If your applications were linked dynamically with any version of the DB library from 9.0 through 9.3, simply installing the latest version fixes this bug; it is not necessary to recompile your applications in this case.) Changes in version 9.3 ---------------------- The DB library can now be compiled as a Microsoft Windows-compatible dynamic link library (DLL), using either Microsoft or Borland compilers. Several minor changes have been made to simplify compilation under HP/UX, Solaris 2.x, Linux, and the Macintosh OS. Changes in version 9.2 ---------------------- As noted in the ECG Database Programmer's Guide, samples having the same sample number in different signals of the same record are treated as simultaneous. Often, this assumption is incorrect; for example, multitrack analog tape recordings may be played back during the digitization process using a reading head with a different azimuth than the recording head, thus introducing time shifts between signals (intersignal skew). In ambulatory ECG recordings, this skew can be 40 milliseconds or more. If signals with known time relationships, such as calibration signals, have been recorded, the amount of skew can be measured. DB library version 9.2 corrects for previously measured skew when reading signals from a record for which skew measurements have been written in the record's header file. Skew correction does not require modification of signal files (only the header files need be edited). Refer to `header.5' in the `doc' directory for details. Changes in version 9.1 ---------------------- Three new functions have been added in version 9.1: . setmsheader() writes a header file for a multi-segment record (see below). . iannclose() closes a single input annotation file . oannclose() closes a single output annotation file Previous versions of the DB library did not support a means for closing individual annotation files. Version 9.1 is the first to support reading and writing multi-segment records. A multi-segment record consists of two or more concatenated segments. Each segment is an ordinary DB record, with its own header file and signal file(s). In any given multi-segment record, all signals must appear in the same order within each segment (signals may not be omitted), and the sampling frequency of any given signal must be the same in each segment. Segments of multi-segment records must be ordinary records (it is not permitted to nest one multi-segment record within another, for example), and the length of each segment must be specified (the DB library does not impose this requirement on ordinary records that are not part of a multi-segment record). There are no other restrictions on segments; specifically, it is permitted to mix segments with different storage formats, and for any segment to appear more than once. A special header file (created either manually or by using setmsheader()) specifies the record name for each segment in a multi-segment record. Once this special header exists, the multi-segment record can be read by any DB application. Note that only the signal files of the segments are "linked" by the multi-segment record's header; annotation files associated with the individual segments are *not* readable as part of the multi-segment record (although an annotation file associated directly with the multi-segment record can be created and read just as for an ordinary record). From the point of view of a DB application, reading a multi-segment record is exactly like reading an ordinary record; specifically, isigsettime() works as expected, permitting jumps forward and backward between as well as within segments. Thus existing DB applications can be made capable of reading multi-segment records simply by relinking them with DB library version 9.1. Version 9.1 is also the first to support signal format 0 (as a means of supplying a "null" signal). Format 0 "signals" require no storage space, and appear to DB application programs to be like any other signals except that all samples of a format 0 signal have the value specified as the ADC zero level in the header file. DB application programs can also write signals in format 0; samples written in this format disappear forever, without error messages. A "null record", consisting of 2 format 0 signals "sampled" at 360 Hz with a length of 5 seconds, is included in the `microdb' directory (`null.hea'; there is no associated signal file). Such "null records" can be useful for efficient representation of gaps between segments of multi-segment records. A sample multi-segment record is also included in the `microdb' directory (`multi.hea'); its segments include two copies of record `100s' (also in the `microdb' directory), with record `null' between them. Support for multi-segment records permits storage of very long records in files of manageable size. During real-time data acquisition, it may be desirable to record signals in segments to reduce the amount of data loss in the event of system failure, or to permit the use of multi-volume storage media. It is now possible, for example, to handle records containing up to 2**31 (~ 2 billion) sample frames, even if the storage media or operating system cannot accommodate files of that size. ============================= B. Changes in DB applications ============================= app/ calibrate Several bugs in and limitations of `calibrate' have been corrected as of version 9.4.1. `calibrate' now works with multi-segment records (it rewrites the segment headers only; the header for the multi-segment record is not affected, since it does not contain gain fields). `calibrate' now preserves the skew and byte offset fields in any header files it rewrites. `calibrate' does not insert baseline fields for signals that are not DC-coupled. A new option, `-v', can be used to force `calibrate' to ask for calibration pulse limits, rather than relying on those found in the calibration file. dbcollate This program is new with DB software version 9.3. It creates a multi-segment record from single-segment records. dbdesc This program has been modified to recognize multi-segment, multi-frequency, and null records. `dbdesc' can now determine the duration of most records, even if the header file does not include this information. The version distributed beginning with release 9.3.1 contains minor bug fixes that correct problems with recognition of multi-segment records. epic A bug caused epic to match each reference ST measurement with the following test measurement (rather than with the closest test measurement before or after the reference measurement). This bug, and another that caused incorrect output of the reference ST measurement when epic was compiled by some 16-bit compilers, have been fixed as of version 9.3.2 (thanks to Rashmi Sarlashkar for the bug reports and fixes). If you have used earlier versions of epic, you may not notice any difference in the output statistics if you repeat your tests with a current version; the differences, if any, are likely to be insignificant, provided that your test measurements are frequent, as recommended, and do not fluctuate widely from beat to beat. Also note that the change made to getann in version 9.3.2 (see above) fixes other problems in epic that were related to limited memory for `aux' strings. ihr This program is new with DB software version 9.3.2. It generates an instantaneous heart rate time series from an annotation file, as does tach, but it does not resample its output to obtain uniform sample intervals. Its output is thus unsuitable for conventional power spectral density estimation, but is ideal for PSD estimation using the Lomb periodogram. mrgann This program is new with DB software version 9.3.2. It can combine two annotation files for the same record into one. Typical applications include merging independent sets of annotations for two or more signals in a multi- signal record, or replacing a segment of an annotation file with annotations from another annotation file. The -i option (for specifying the input annotator names) was incorrectly shown as -a in the ninth edition (July 1995) of the ECG Database Applications Guide. This error has been corrected in the tenth edition (March 1997) and in the man page (doc/mrgann.1); mrgann itself has not been changed. A bug in `mrgann' has been corrected as of version 9.6.2. The bug prevented proper operation of `mrgann' if the time associated with the first `-m' option was 0 (the beginning of the record). mxm The calculations of both normalized and unnormalized RMS error performed by versions of this program dated earlier than 15 May 1996 were incorrect, and depended on the number of measurements made (in general, the erroneous values became smaller as the number of measurements increased). For this reason, values obtained using earlier versions of this program are not directly comparable with those obtained using the current version, although comparisons among values obtained using earlier versions are valid for purposes of ranking. Thanks to Denny Dow for pointing out the problem! pschart A bug that caused incorrect output of the recording date has been fixed. The version distributed beginning with DB software version 9.3.1 includes a workaround for the MS-DOS COMMAND.COM bug that makes it impossible to pass an empty string in the argument list of a command. This workaround is needed to permit full use of `-a', `-c', and `-T' options (see doc/pschart.1). If no page title is specified for the output of `pschart' and `psfd', these programs insert the date of the record (if this is specified in the record header) in the page title. Previous versions of `pschart' and `psfd' have always printed the date of the beginning of the record in this case, but this can be misleading when records span more than one date. Beginning with release 9.4.1, `pschart' and `psfd' now use the date of the last data plotted on each page in the page title. New options for printing marker bars have been added to `pschart' and `psfd'; these changes required revisions to both the ps*.c sources and the ps*.pro PostScript prolog files. See the revised man pages (doc/pschart.1 and doc/psfd.1) for details on the `-M' options. There is now an explicit definition (PAPERDEF) in Makefile for the default paper size to be assumed by pschart and psfd. (It has been possible for some time to change the default by compiling these applications with an appropriate default paper size definition, but this feature was not well documented.) As before, if no explicit choice is made, these applications assume US letter size (8.5 x 11 inches, or 216 x 279 mm) paper. The algorithm used to set a vertical offset for each plotted trace is now much less sensitive to transient spikes in the signals. Beginning with release 9.6.0, pschart's new `-H' option can be used to print multi-frequency records in `high-resolution' mode. psfd Bugs that caused incorrect output of counter values, the recording date, and (in certain cases) the final marginal timestamp have been fixed. A new `-H' option has been added to `psfd' (to adjust the spacing between traces). The version distributed beginning with DB software version 9.3.1 includes a workaround for the MS-DOS COMMAND.COM bug as noted above for `pschart' (see doc/psfd.1). Also see the comments under pschart above, about page titles, marker bars, the default paper size, and the calculation of vertical offsets for each trace. rdsamp Versions of `rdsamp' earlier than 9.4.1 produced incorrect time values when reading a multi-segment record with the `-p' option. This has been corrected. Beginning with release 9.6.0, rdsamp's new `-H' option can be used to read multi-frequency records in `high-resolution' mode. skewedit This program is new with version 9.3. It rewrites a header file, modifying the skew fields in accordance with user specifications. (It has no means of determining the skew itself.) sortann This application is new in version 9.7. Use it to process an annotation file containing out-of-order annotations (see below), rearranging them in canonical order (time order, with simultaneous annotations in `chan' order). `sortann' is designed to work most efficiently on nearly-ordered annotation files, but it can sort arbitrarily arranged annotations. tach A new `-s' option (to set a smoothing parameter) has been added. New options (`-Vs', `-Vm', `-Vh') have been added (to prefix each output sample by its time in seconds, minutes, or hours respectively; `-Vs' is equivalent to the existing `-V' option, which remains usable). xform Several minor bugs have been fixed, and xform now handles defaults better when output record specifications are supplied interactively. A bug in the interpolation code has been fixed as of version 9.3.2, thanks to Rashmi Sarlashkar. Beginning with release 9.6.0, xform's new `-H' option can be used to read multi-frequency records in `high-resolution' mode. psd/ Beginning with version 9.3.2, the DB Software Package now includes a small set of applications for power spectral density estimation. These general- purpose applications work on text files, independently of the DB library. Sample scripts are also provided, to illustrate how these applications can be used together with `ihr' and `tach' for frequency-domain analysis of heart rate variability. See psd/README for details. Beginning with version 9.4.2, `fft' options for inverse transforms (-i and -I) now work properly. ========================================= C. Notes on reading multi-segment records ========================================= Using DB library version 9.1 or later, applications can read multi-segment records, in most cases without requiring any changes to existing code for reading single-segment records. One of the few cases in which you may need to modify code that works for reading single-segment records is illustrated by the following excerpt: static DB_Siginfo s[DB_MAXSIG]; char *record_name; ... if (isigopen(record_name, s, 2) < 2 && isigopen(record_name, s, DB_MAXSIG) < 2) /* give up */ else /* do something with signals 0 and 1 */ This code attempts to be efficient by opening only the signals it needs (in the first call to isigopen), but this attempt may fail if the record contains more than two signals in signal group 0. The second call to isigopen is intended to open all available signals if the first attempt fails; this idea works for a single-segment record, but fails for a multi-segment record. It should be rewritten as: int flag = 0; static DB_Siginfo s[DB_MAXSIG]; char *record_name; ... if (isigopen(record_name, s, 2) == 2) flag = 1; else dbquit(); if (flag == 0 && isigopen(record_name, s, DB_MAXSIG) >= 2) flag = 1; if (flag == 0) /* give up */ else /* do something with signals 0 and 1 */ The rule of thumb is that whenever isigopen() or dbinit() fails, your application should invoke dbquit() before trying again. This is not required for single-segment records, but it's a good practice in any case. In the unlikely event that your application needs to detect when it has opened a multi-segment record, the following method (used in `dbdesc') may be useful: int nsig; static DB_Siginfo s[DB_MAXSIG]; DB_Time t; ... nsig = isigopen(record_name, s, DB_MAXSIG); t = strtim("e"); /* t is the length of the entire record in samples */ if (s[0].nsamp != t) /* s[0].nsamp is the length of the first segment */ /* it's a multi-segment record */ else /* it's not */ ================================================= D. Notes on annotation of multi-frequency records ================================================= The `time' field of an annotation specifies the sample number of the sample to which the annotation refers. In a multi-frequency record, however, the sample number of any given sample depends on how the record is being read. In the default mode, used by the applications included in the DB Software Package, getvec reads one sample per signal per frame, and the sample number is identical to the frame number. In "high resolution" mode, getvec returns each sample of each signal at least once (replicating samples of any signals sampled at less than the maximum frequency). You can select "high resolution" mode by including the line setgvmode(DB_HIGHRES); as the first DB library call in your application. An example may help to clarify this point. In the MIMIC database, each "frame" in the signal file contains 4 samples of each ECG signal, and 1 sample of each non-ECG signal, and there are 125 frames per second. Thus, when reading a MIMIC record in the default mode, the samples that occur 1 second from the beginning of the record have sample number 125 (i.e., you must call getvec 125 times to get to these samples). When reading the same record in "high resolution" mode, however, you must call getvec 500 times to get to the same samples, and their sample number is therefore 500. The getann, putann, and iannsettime functions contained in versions 9.0 through 9.5 of the DB library expect to deal with times expressed as frame numbers. In default mode, where sample numbers and frame numbers are equal, this is not a problem, and this behavior permits all of the existing DB applications to read multifrequency records in default mode without modification. In "high resolution" mode, however, this behavior is less desirable, since it requires applications that read or write annotations to be aware of the difference between sample numbers and frame numbers. Although applications that created annotation files while in "high resolution" mode were free to use sample numbers as times, doing so created other problems when these annotation files were read by other applications, which had no way of knowing if times had been expressed in frames or in samples. Beginning with release 9.6.0, getann, putann, and iannsettime now expect to deal with times expressed as sample numbers, whether the application is in default mode or in high-resolution mode. This change greatly simplifies the process of enabling high-resolution operation of existing DB applications. When in high-resolution mode, putann currently converts sample numbers to frame numbers for output to annotation files, and getann performs the inverse conversion when reading annotation files. Thus any annotation file can be read correctly in either default or high-resolution mode. An obvious limitation of this approach is that it is not possible to preserve information about the exact location of an annotation within a frame at present. A more subtle consequence of this scheme is that applications that write annotations must still be careful to maintain frame-number/chan order in their output streams. Since putann converts sample numbers to frame numbers before writing annotations, all annotations within any frame must be presented to putann in chan order. Note further that, for any given chan value, only one annotation may be written in each frame. It is likely that these restrictions will be removed in future versions of the DB library. ==================================== E. Notes on out-of-order annotations ==================================== [The following material also appears in the ECG Database Programmer's Guide (doc/dbu.tex).] DB applications may generally assume (and most of them do assume) that all annotations in any given annotation file are in canonical order. Successful use of `iannsettime' requires that this assumption be correct. Early versions of the DB library (before version 6.2) defined canonical order as time order. More recent versions of the DB library define canonical order as `time' and `chan' order (thus annotations are arranged first in `time' order, and any simultaneous annotations are arranged according to the value of their `chan' fields, from smallest to largest). The combination of the `time' and `chan' fields of an annotation defines a unique `location' in a virtual array of annotations which an annotation file represents. No two annotations may occupy the same location in this virtual array. This restriction was enforced by versions of the DB library earlier than version 9.7. In these versions of the DB library, `putann' required that annotations be written in canonical order, and refused to write any out-of-order annotations supplied to it. Current versions of the DB library do not impose this requirement. In version 9.7 and later versions, `putann' accepts and records out-of-order annotations and multiple annotations that occupy the same location. If any such annotations have been written, the completed annotation file is rewritten in canonical order by `dbquit' or `oannclose'. This is accomplished by running `sortann' as a separate process using the ANSI C `system' function. If this function is not available, or if `sortann' cannot be run, `dbquit' (or `oannclose') emits a warning message describing how to post-process the annotations to put them into canonical order. Although it is possible using current versions of the DB library to write two or more annotations to the same location, only the last annotation written to any given location is retained in the canonically-ordered annotation file. Thus that an application that generates an annotation file can change the `anntyp', `subtyp', `num', or `aux' fields of a previously-written annotation simply by writing another annotation to the same location (i.e, with the same `time' and `chan' fields). As a special case, an application may delete a previously-written annotation by writing a NOTQRS annotation to the same location. To move an annotation to a different location (i.e., to change its `time' or `chan' fields), it is necessary to delete it from the original location, and then to insert it at the desired location, using two separate invocations of `putann/. In unusual circumstances, an unsorted annotation file may be useful (for example, as an aid for debugging the application that produced it; `rdann' can be used to list all of the annotations in such a file, in the order in which they were written). In some environments, the use of the ANSI C `system' function may be a security problem, and you may wish to avoid automatic sorting of annotations for this reason. Set the environment variable DBNOSORT (to any value) at run time, or define the symbol DBNOSORT when compiling the DB library, if you wish to suppress automatic annotation sorting by `dbquit' and `oannclose'. ================ F. Documentation ================ The ninth edition of the ECG Database Programmer's Guide was reprinted in October, 1996, with minor revisions and additions to bring it up-to-date as of version 9.6.1 of the DB Software Package. The tenth edition of the ECG Database Applications Guide, incorporating changes through version 9.6.2, was published in March, 1997. These guides are provided in HTML, PostScript, and source (LaTeX, TeXinfo, and troff) form as part of the DB Software Package. These include further revisions and additions, and are up-to-date as of version 9.7.0. The most recent versions of these guides may be viewed on the World Wide Web (point your web browser to http://ecg.mit.edu/).