Sign in or 
| Version | User | Scope of changes |
|---|---|---|
| Nov 5 2010, 4:27 PM EDT | NSCAPCD | 149 words added, 43 words deleted |
| Nov 4 2010, 11:16 AM EDT | NSCAPCD | 31 words added, 21 words deleted |
Changes
Key: Additions Deletions
This page is designed to offer a list of ideas of ways to use the optional Automatic Data Validation Processor (ADVP) for real-time data quality assurance.
Please feel free to add your own ideas and experiences!
1. Basic Ozone Rule: If OZONE > (fixed value) and AMBTEMP < (fixed value), then flag as suspect, set data grade
2. Stuck WDR Sensor: If Variance of WDR/VWDR < 2%, then flag as suspect, send email alarm
3. Cross-Site Comparison: If SITEA-(any parameter) Differs > X% from SITEB-(parameter), flag as suspect, set data grade
4. BAM Sample Failure: When Ifone PMxxof =the 985critical (wheresampling PMxxparameters =(flow, yourpower, nozzle performance, etc) exceeds its alarm set points the BAM PMwrites a "1" in the appropriate parameter name),error thenlocations flagwithin asthat invalid,hours (setdata nullstring. codeThe "?"),BAM sendalso emailwrites alarm.the full scale PM value into that hours PM value. Depending on how your BAM range is set up this could be either a "985" or "1000". This abnormally high value is intended to "flag" the operator that a sampling error has occurred. YouThe operator can accessthen check the BAM System Menu,Menu (item #7 (Last"Last 100 Errors) Errors") to ascertaindetemrine exacltyexactly whatwhich sampleerror failureoccurred. flagTaking wasadvantage setof forthis constant value one can apply an ADVP Rule that hour.will inform the operator in a matter of minutes rather than discovery in data review. Basic logic is: If PMxx = 985 (where PMxx = your BAM's PM (e.g.parameter name), then flag as invalid, (set Flow,null code "?"), send email Tapealarm. For Failure,more Powerdetailed etc.)info Seesee BAM Manual Sections 6.5 & 9.4
5. Simple Historical Comparison: If (any parameter) Differs > X% from previous N-year average, flag as suspect, set data grade
6. Fireworks: Set a JULDAY channel in the logger (math pack, equation is "JULDAY()="), and set rule If PM25LC > 100 and JULDAY >185 and JULDAY < 188 (or JULDAY = 186 and adjust for leap years) then set null code for fireworks. Other dates may be inserted for local festivals.
7. Annotation-based-null-coding. Choose a series of unique keywords that are mapped to null codes. Have operators add annotations to the data (e.g., "wildlife") and set an ADVP rule to look for that keyword. If Annotation Contains (string), then add null code.
8. Excessive negative instrument drift. If (parameter template) Value < -0.05, then set value = 0 , send e-mail alarm to operator.
9. General instrument problems, where diagnostic / metadata is collected. Note that new Parameter Templates will need to be set up to represent the diagnostic data, since ADVP rules only reference parameter template tags. Rules might consist of: If O3_SAMPFLOW < 0.5 then flag suspect, send email alarm to operator. Thresholds can be set for PMT voltages, sample pressures, etc.
10. Add HRDAY() math calculation channel in logger, compare solar radiation to day and night conditions (e.g., if HRDAY > 9 and HRDAY < 16 and RAINFALL < 0.01 and SOLARRAD < 500, then flag suspect, send email alarm).
11. Comparison of CO at certain urban sites to rush hour conditions.
Please feel free to add your own ideas and experiences!
1. Basic Ozone Rule: If OZONE > (fixed value) and AMBTEMP < (fixed value), then flag as suspect, set data grade
- Alternate: solar radiation comparison more accurate than AMBTEMP, use SOLARRAD if you have it available.
2. Stuck WDR Sensor: If Variance of WDR/VWDR < 2%, then flag as suspect, send email alarm
3. Cross-Site Comparison: If SITEA-(any parameter) Differs > X% from SITEB-(parameter), flag as suspect, set data grade
4. BAM Sample Failure: When Ifone PMxxof =the 985critical (wheresampling PMxxparameters =(flow, yourpower, nozzle performance, etc) exceeds its alarm set points the BAM PMwrites a "1" in the appropriate parameter name),error thenlocations flagwithin asthat invalid,hours (setdata nullstring. codeThe "?"),BAM sendalso emailwrites alarm.the full scale PM value into that hours PM value. Depending on how your BAM range is set up this could be either a "985" or "1000". This abnormally high value is intended to "flag" the operator that a sampling error has occurred. YouThe operator can accessthen check the BAM System Menu,Menu (item #7 (Last"Last 100 Errors) Errors") to ascertaindetemrine exacltyexactly whatwhich sampleerror failureoccurred. flagTaking wasadvantage setof forthis constant value one can apply an ADVP Rule that hour.will inform the operator in a matter of minutes rather than discovery in data review. Basic logic is: If PMxx = 985 (where PMxx = your BAM's PM (e.g.parameter name), then flag as invalid, (set Flow,null code "?"), send email Tapealarm. For Failure,more Powerdetailed etc.)info Seesee BAM Manual Sections 6.5 & 9.4
5. Simple Historical Comparison: If (any parameter) Differs > X% from previous N-year average, flag as suspect, set data grade
- % tolerance may be vastly different for different parameters, e.g., ozone and particulate may have low tolerances.
6. Fireworks: Set a JULDAY channel in the logger (math pack, equation is "JULDAY()="), and set rule If PM25LC > 100 and JULDAY >185 and JULDAY < 188 (or JULDAY = 186 and adjust for leap years) then set null code for fireworks. Other dates may be inserted for local festivals.
7. Annotation-based-null-coding. Choose a series of unique keywords that are mapped to null codes. Have operators add annotations to the data (e.g., "wildlife") and set an ADVP rule to look for that keyword. If Annotation Contains (string), then add null code.
8. Excessive negative instrument drift. If (parameter template) Value < -0.05, then set value = 0 , send e-mail alarm to operator.
9. General instrument problems, where diagnostic / metadata is collected. Note that new Parameter Templates will need to be set up to represent the diagnostic data, since ADVP rules only reference parameter template tags. Rules might consist of: If O3_SAMPFLOW < 0.5 then flag suspect, send email alarm to operator. Thresholds can be set for PMT voltages, sample pressures, etc.
10. Add HRDAY() math calculation channel in logger, compare solar radiation to day and night conditions (e.g., if HRDAY > 9 and HRDAY < 16 and RAINFALL < 0.01 and SOLARRAD < 500, then flag suspect, send email alarm).
11. Comparison of CO at certain urban sites to rush hour conditions.
