File Import Tool Improvements

In Version 2.3, AirVision significantly enhances the File Import Tool to better handle new data types, primarily meta data for PM lab sample data and AirToxics data.
In the original model, the File Import Tool assumed each row in the file contained data for different parameters, and could only import the numerical values for that data. Example:



In the Enhanced Tool, the ‘several parameters on one row’ templates have been enhanced to be able to denote that columns represent additional data properties for the parameter data point, including:
· Flags (see “Flag Mapping Table” below)
· AQS Method Code
· AQS Null Codes
· Qualifier Code (one)

Example:


The Enhanced File Import Tool also includes the capability to do ‘row-wise’import, where a single row represents a single data point, with several additional properties associated with that single data point. This is a common case for air toxics data.
In this mode, it is even common to have the same file represent multiple sites and parameters, so it is necessary to create a ‘key’ string for each site/parameter to match the identifier that is used within the imported file.
First, we would go into the Site/Parameter editor and use the ‘Metatags’editor at the bottom of the Site form and the Parameter form to identify the string we will be looking for in the imported filefile, using keys "SiteIdentifier" and "ParmIdentifier"

<<examples>>

Once this is done, we can set up a template to look for that particular string in the file:



Available Fields include:
· Site / Parameter Identifier (by meta tag)
· Flag
· Final Value
· Tare / Final Weight
· Sample Volume, Temp, Barometric Pressure
· Null Code, Qualifier Code
· Sample ID, Cannister ID
· Sample Retrieved Date, Time, Date/Time
· Sample Analyzed Date, Time, Date/Time
· Minimum Detectable Value
· Uncertainty
· Frequency Code
· Duration Code
· Sample End Date, Time, Date/Time


Flag Mapping Table
Some devices provide data flags in the form of a string, which can be mapped into AirVision’s standard flags. Supported formats include:
· Binary string (e.g., “1101010”, where each digit represents presense/absence of a flag)
· Hex (e.g., “1F0A”, were each hex char represents a 4-bit binary map, each bit = 1 flag)
· Decimal Binary (e.g., “1024”, where if the decimal number is converted to binary, each bit represents a possible flag
· String (e.g, “FD<”, where each character represents a flag, but does not map equally to AirVision’s standard flag table.
By allowing the user to create a “Rosetta Stone” table to map the instrument’s particular flags to AirVision’s flags, each Flag Mapping Table can be referenced in the File Import Tool to map that column and its content into AirVision flags.
For example, the Met One BAM provides a series of “1”and “0”fields to represent flags in the BAM.
Station, 1
Time,Conc(mg/m3),Qtot(m3),no(V),WS(MPS),WS(MPS),RH(%),Delta(C),WS(MPS),E,U,M,I,L,R,N,F,P,D,C,T,
02/06/09 12:00, 0.380, 0.000, 0.189, 2.5, 2.2, 31, 52.7, 35.9,0,0,1,0,0,0,0,1,0,0,0,1,

Since each column maps to a different flag, we actually have to create multiple Flag Map table entries and map each column to its own flag map. Example


However, Met One is considering an ‘all in one’ field, where the commas are removed and the binary pattern is given in one column, e.g.:
Station, 1
Time,Conc(mg/m3),Qtot(m3),no(V),WS(MPS),WS(MPS),RH(%),Delta(C),WS(MPS),FLAGS
02/06/09 12:00, 0.380, 0.000, 0.189, 2.5, 2.2, 31, 52.7, 35.9,001000010001,

This makes the flag map table easier for us- only one entry! Note that we define bit 0 as the rightmost bit, and bit N as the leftmost bit, which is actually the opposite of the way the old string is listed.