| TECH-TIP - Ensuring Water Quality Data Integrity
AWWA Articles

TECH-TIP - Ensuring Water Quality Data Integrity

The integrity of turbidity data and water quality data for other parameters is critical to the water treatment plant and distribution system optimization process.  Making appropriate data-driven decisions for process control depends on having accurate and representative data.  Additionally, the data generated by online and lab instrumentation may, in some cases be used, to determine regulatory compliance.  Compliance is not only determined by the instrument reading and numerical value obtained relative to the maximum contaminant level (MCL), but also by the frequency of measurement, which helps ensure compliance with monitoring and reporting requirements.  For example combined filter effluent (CFE) turbidity is typically required to be monitored every 4 hours, while individual filter effluent (IFE) turbidity data is typically required to be monitored continuously and reported on a 15-minute interval.  Not having this data available, such as in the situation where a turbidimeter light source may be burnt out and the instrument not collecting data, can result in a monitoring and reporting violation.  By taking some basic steps to ensure the validity of water quality data, operators can help to maintain water quality data that is “beyond reproach.”


What is Data Integrity?

As defined by businessdictionary.com, data integrity refers to “the accuracy and consistency of stored data.”  The website goes on to state that “Data integrity is imposed…through the use of standard rules and procedures, and is maintained through the use of error checking and validation routines.”  Although this definition stems from a business-related website referring to databases, it is equally accurate in defining data validity in reference to water quality analysis.  The following sections of this Tech-Tip refer to specific steps operators can take to help ensure the validity of water quality data they collect and use for process control and compliance, with an emphasis on turbidity data.   



Obtaining accurate analytical results depends on measuring a sample that is representative of the process.  If drawing the sample from a pipe, be sure to draw the sample from a location that avoids the collection of air or sediment.  It is often best to draw the sample from the top or side of the pipe.  When delivering samples to online instrumentation, take care to avoid excessively long sample lag times to ensure the delivery of a representative sample to the instrument.  Instrument manufacturers typically specify the ideal instrument flow rate.  For example, the ideal flow rate for a Hach 1720E is 250-750mL/min.  Since bubbles can positively interfere with turbidity measurements, the use of an internal (built-in) or external bubble trap is recommended to minimize the impact of entrained air on the turbidity measurement.  Many instruments also have a “bubble reject” mode in which the instrument disregards momentary turbidity excursions most likely to be due to the presence of bubbles in the sample.  Sample lines feeding the instrument should remain clean, so as to avoid contamination.  This can be achieved by periodically inspecting and replacing the instrument sample and drain lines.  Sample flow may be verified through the use of a flow meter or a loss of flow alarm, which can notify operators of any flow changes that need to be further investigated.  Plumbing the drain line to include an air gap can provide a helpful visual of the presence of flow as well as an indicator of any potential fouling in the turbidimeter body.


Turbidimeter Instrument Physical Condition

Turbidity is measured based on the amount of light scattered at a 90-degree angle to the instrument’s light source.  Anything that scatters light or affects the amount of light produced or detected by the instrument has the potential to affect the accuracy of the measurement.  Therefore, the physical condition of an online turbidimeter can also impact the accuracy of measurements.  For instruments using a white light source, it is important to know that the intensity of this light source can change and diminish over time.  Instrument calibration adjusts for this, to some extent, but the light source must also be replaced on an annual basis.  Refer to the manufacturer’s instructions regarding replacement of the light source, as well as for specific servicing instructions for instruments that may use an LED or laser light source.  The photodetector or photocell used to measure light scatter should also be inspected on a regular basis to ensure its integrity (not cracked, dirty, or leaking fluid).  The photocell and lens (if present) are part of the optical measurement system and may be cleaned periodically, preferably directly prior to calibration, if they are found to be dirty.  The integrity of the turbidimeter body itself can impact measurement accuracy.  Be sure that the turbidimeter body is clean and free from scale or any particle buildup, which can scatter light.  Many turbidimeter bodies may be cleaned using a simple brush.  Also, take the time to periodically inspect the turbidimeter body to ensure it is intact and that the head of the turbidimeter is properly seated in the instrument body, for applicable models.


Instrument Software Setup

Instrument controllers are typically provided with a default software setup configured.  The default setup may not be appropriate for all monitoring applications, so it is important to review and update the controller configurations for new instruments that are installed.  Controller setup options of particular importance include the following:


  • Error Mode – What does the instrument do with the output signal when an error occurs?  Many controllers are set up to hold the signal for the last reading.  For plants already monitoring very low turbidity levels, this change may not be immediately obvious on a trend chart, which could result in errors being inadvertently overlooked.  To alleviate this problem, considering changing what the controller does with the output signal for errors (this is typically adjustable) or set a rate of change alarm that will notify staff if the thousandth place of the turbidity measurement does not change over a defined period of time.  Most instruments will typically also hold all outputs during a calibration. 
  • Output Scaling – If used, an instrument’s 4-20 mA output signal should be scaled over an appropriate range for the sample being measured.  For example, the 4 mA output may represent 0 NTU, and the 20 mA output may represent 0.5 or 1.0 NTU for low level measurements.  Take care to setup the instrument to avoid situations where the turbidity measurement exceeds the 20 mA output setting, or the data capping may be experienced, in which the turbidity output maxes out and the exact turbidity reading may be unknown (in this case, the turbidity measurement may be obtained from the controller’s memory, although many controllers only store data for a limited period of time).  When setting up the outputs, ensure that the 4-20 mA scaling on the PLC matches the scaling programmed at the instrument.
  • Data Collection Frequency – Take care to ensure that the correct data measurement is being collected from the instrument at the correct frequency.  This may also include any event logs that are associated with instrument errors or maintenance.  For example, Presidents Award and Phase IV Partnership for Safe Water turbidity data reporting requires that data be collected from individual filters at a frequency no less than every 15 minutes.   

Additionally, consider the types of alarms that may be implemented through the SCADA system that may be configured in such a way as to help ensure data integrity.  These may include high/low alarms, rate of change alarms, loss of flow alarms, and others.


Data Validation

The definition of data validity provided earlier in this article referred to the use of error checking and data validation procedures to help ensure data accuracy.  This is also true for water quality measurements.  One basic means of verifying data accuracy from online instrumentation is through grab sample comparison.  Collect a sample of the water that the online instrument is measuring, carefully measure it using a laboratory instrument, and compare results.  Typically low level turbidity samples (<0.50 NTU) will agree to within +/- 0.05 NTU.  It is important that both lab and online instruments are accurately calibrated and in proper operating condition prior to performing this comparison.  Staff may also wish to utilize formal data validation procedures to further examine questionable data points.  For example, a decision tree type approach may be used to guide staff through a series of questions used to determine if a data point is valid or invalid.  This may involve consideration of issues such as agreement with other instruments (such as particle counters), maintenance activities, mechanical issues, or other factors.  Use of standard operating procedures for data validation, as well as for other sampling-related activities, can help to ensure consistent practices are followed by all staff.  Finally, consider the use of data trend charts, such as filter profiles, to visually examine data on a regular basis.


By implementing the actions described above water utility staff can help to ensure the integrity of data that is generated to support plant and distribution system processes, as well as for regulatory compliance.


Acknowledgements:  Many thanks to EPA’s Technical Support Center AWOP Team (Rick Lieberman), David Dani (CDPHE), Evan Hofeld (Oregon Health Authority), and Steve Deem (Washington State Department of Health) for the supporting information used to compile this article.