Sign in or 
| Version | User | Scope of changes |
|---|---|---|
| Nov 5 2010, 4:29 PM EDT | NSCAPCD | 4 words added, 2 words deleted |
| Nov 5 2010, 4:27 PM EDT | NSCAPCD | 149 words added, 43 words deleted |
Changes
Key: Additions Deletions
This page is designed to offer a list of ideas of ways to use the optional Automatic Data Validation Processor (ADVP) for real-time data quality assurance.
Please feel free to add your own ideas and experiences!
1. Basic Ozone Rule: If OZONE > (fixed value) and AMBTEMP < (fixed value), then flag as suspect, set data grade
2. Stuck WDR Sensor: If Variance of WDR/VWDR < 2%, then flag as suspect, send email alarm
3. Cross-Site Comparison: If SITEA-(any parameter) Differs > X% from SITEB-(parameter), flag as suspect, set data grade
4. BAM Sample Failure: When one of the critical sampling parameters (flow, power, nozzle performance, etc) exceeds its alarm set points the BAM writes a "1" in the appropriate parameter error locations within that hours data string. The BAM also writes the full scale PM value into that hours PM value. Depending on how your BAM range is set up this could be either a "985" or "1000". This abnormally high value is intended to "flag" the operator that a sampling error has occurred. The operator can then check the BAM System Menu (item #7 "Last 100 Errors") to detemrine exactly which error occurred. Taking advantage of this constant value one can apply an ADVP Rule that will inform the operator in a matter of minutes rather than discovery in data review. Basic logic is: If PMxx = 985 (where PMxx = your BAM's PM parameter name), then flag as invalid, (set null code "?"),code), send email alarm. For more detailed info on sample errors see BAM Manual Sections 6.5 & 9.4
5. Simple Historical Comparison: If (any parameter) Differs > X% from previous N-year average, flag as suspect, set data grade
6. Fireworks: Set a JULDAY channel in the logger (math pack, equation is "JULDAY()="), and set rule If PM25LC > 100 and JULDAY >185 and JULDAY < 188 (or JULDAY = 186 and adjust for leap years) then set null code for fireworks. Other dates may be inserted for local festivals.
7. Annotation-based-null-coding. Choose a series of unique keywords that are mapped to null codes. Have operators add annotations to the data (e.g., "wildlife") and set an ADVP rule to look for that keyword. If Annotation Contains (string), then add null code.
8. Excessive negative instrument drift. If (parameter template) Value < -0.05, then set value = 0 , send e-mail alarm to operator.
9. General instrument problems, where diagnostic / metadata is collected. Note that new Parameter Templates will need to be set up to represent the diagnostic data, since ADVP rules only reference parameter template tags. Rules might consist of: If O3_SAMPFLOW < 0.5 then flag suspect, send email alarm to operator. Thresholds can be set for PMT voltages, sample pressures, etc.
10. Add HRDAY() math calculation channel in logger, compare solar radiation to day and night conditions (e.g., if HRDAY > 9 and HRDAY < 16 and RAINFALL < 0.01 and SOLARRAD < 500, then flag suspect, send email alarm).
11. Comparison of CO at certain urban sites to rush hour conditions.
Please feel free to add your own ideas and experiences!
1. Basic Ozone Rule: If OZONE > (fixed value) and AMBTEMP < (fixed value), then flag as suspect, set data grade
- Alternate: solar radiation comparison more accurate than AMBTEMP, use SOLARRAD if you have it available.
2. Stuck WDR Sensor: If Variance of WDR/VWDR < 2%, then flag as suspect, send email alarm
3. Cross-Site Comparison: If SITEA-(any parameter) Differs > X% from SITEB-(parameter), flag as suspect, set data grade
4. BAM Sample Failure: When one of the critical sampling parameters (flow, power, nozzle performance, etc) exceeds its alarm set points the BAM writes a "1" in the appropriate parameter error locations within that hours data string. The BAM also writes the full scale PM value into that hours PM value. Depending on how your BAM range is set up this could be either a "985" or "1000". This abnormally high value is intended to "flag" the operator that a sampling error has occurred. The operator can then check the BAM System Menu (item #7 "Last 100 Errors") to detemrine exactly which error occurred. Taking advantage of this constant value one can apply an ADVP Rule that will inform the operator in a matter of minutes rather than discovery in data review. Basic logic is: If PMxx = 985 (where PMxx = your BAM's PM parameter name), then flag as invalid, (set null code "?"),code), send email alarm. For more detailed info on sample errors see BAM Manual Sections 6.5 & 9.4
5. Simple Historical Comparison: If (any parameter) Differs > X% from previous N-year average, flag as suspect, set data grade
- % tolerance may be vastly different for different parameters, e.g., ozone and particulate may have low tolerances.
6. Fireworks: Set a JULDAY channel in the logger (math pack, equation is "JULDAY()="), and set rule If PM25LC > 100 and JULDAY >185 and JULDAY < 188 (or JULDAY = 186 and adjust for leap years) then set null code for fireworks. Other dates may be inserted for local festivals.
7. Annotation-based-null-coding. Choose a series of unique keywords that are mapped to null codes. Have operators add annotations to the data (e.g., "wildlife") and set an ADVP rule to look for that keyword. If Annotation Contains (string), then add null code.
8. Excessive negative instrument drift. If (parameter template) Value < -0.05, then set value = 0 , send e-mail alarm to operator.
9. General instrument problems, where diagnostic / metadata is collected. Note that new Parameter Templates will need to be set up to represent the diagnostic data, since ADVP rules only reference parameter template tags. Rules might consist of: If O3_SAMPFLOW < 0.5 then flag suspect, send email alarm to operator. Thresholds can be set for PMT voltages, sample pressures, etc.
10. Add HRDAY() math calculation channel in logger, compare solar radiation to day and night conditions (e.g., if HRDAY > 9 and HRDAY < 16 and RAINFALL < 0.01 and SOLARRAD < 500, then flag suspect, send email alarm).
11. Comparison of CO at certain urban sites to rush hour conditions.
