Databases and Systems

figure 1, and all flows into the ID database, which is described in a subsequent
section.
Conversion to ASN.l
Transforming data into the same format allows them to be used by software
independently of the original source, a major requirement for integration. Within this
uniform format the accuracy of the identifiers used to link, or point, to other records
determines the accuracy and actual meaning of the links themselves. For literature
articles, NCBI has added an integral identifier, the PubMed identifier. For
convenience, many articles contain both a MEDLINE unique identifier and a
PubMed identifier. Genetic sequence identifiers are more complex and can occur in
different forms and places within each data record. Before this integration and use of
sequence identifiers can be explained, the way in which NCBI represents the data
must be clear.
So that the databases and other later applications can use a common format and be
insulated from parsing the input data, the diverse streams of data are converted to
Abstract Syntax Notation (ASN. 1), an international standard [7]. When using
ASN.l, one file describes the format to which other files (messages) of data must
correspond. This file can be considered the definition of the message format. The
data conforming to the message format can be understood by later application
software that is thereby insulated from details of the original input formats. The
messages use a “tag-value” format, in that an identifier describes the value that
follows; however, analogous to programming language record definitions, the ability
to use both recursion and user defined types in the definition of the message format
allows for almost infinite complexity in the messages themselves. Since ASN. 1
messages can thus become rather complex and are not intended to be human
readable, other report formats, such as GenBank flatfile format, are used to display
this data.
“asn.all” ASN. 1 message definition
The particular format, or message definition, plays a central role. It is what describes
the syntax into which all the sequence record information must be parsed. The
original definition proposed by Jim Ostell [8] has been used with minor
modifications for over five years. It is available as the file, “asn.all” in the NCBI
toolkit distribution (ftp:ncbi.nlm.nih.gov; directory toolbox/ncbi_tools) and is
discussed in detail in the NCBI Programmers Toolkit documentation [8]. “asn.all”
describes the format of both the literature and genetic sequence messages. NCBI also
makes use of other ASN. 1 message definitions.
So that the data in the ASN.l messages can be used by software, C language
structures that map fairly closely to the “asn.all” definitions were designed, as well as
software (object loaders) that could read and write these messages from C language
structures to files and vice versa. These original structures and object loaders were
hand crafted. More recently, the program “asncode” was written by the author and
13