Third IMO Greenhouse Gas Study 2014

160 Third IMO GHG Study 2014
Pre-processing AIS data
The first stage is to parse all the terrestrial AIS (T-AIS) and S-AIS data to create consistent individual data files
for each MMSI, as the MMSI is the key unique identification in AIS. Each source of AIS data needs to be parsed
separately into a universal format to allow combined processing in later stages. Since AIS data was provided
in various formats, a pre-processor subprogram was used for the processing of all AIS data (see Figure 1 for a
display of the user interface of the pre-processor). Together with this, there were requirements from the data
providers that all ship locations be anonymized before the data was shared. This restricted the parsed data to
the following fields:
• MMSI
• IMO unique code
• Time of message
• Speed over ground
• Draught
The pre-processor facilitated the consortium partners to define their AIS data structure (e.g. time stamp
pattern, field indices). While most (typically more than 99%) of the AIS data lines are successfully converted
into the common selected format, the remaining non-relevant, false messages are removed from the set. Such
messages may contain the following:
• Incorrectly formatted dates;
• Dynamic messages with no longitude, latitude or speed information;
• Messages without nine-digit numerical MMSI codes; valid MMSI codes are in the format MIDXXXXXX
where the first three digits represent the Maritime Identification Digits (MID) and X is any figure from
0 to 9 (ITU, 2012).
Besides the task of parsing the information from one format to other, the pre-processor adds a region
identifier as an additional field into the output while the precise coordinates are omitted. To achieve this, the
pre-processor used locations defined by polygons (in the format of GIS shapefiles) which were obtained from
Marine Regions (2014) to define the different sea regions shown in Figure 2.
Additionally, the pre-processor adds a speed-over-ground estimate (knots) for processed LRIT data. LRIT data
was not used to generate activity estimates, but as a validation data set. It is explored in greater detail in
Annex 3. The speed estimate generated for LRIT is based on the ship coordinates and the time difference
between two consecutive messages, since ship speed is not included in an LRIT message.