Quick™ Low Range Test Kit: Verification Report

August 2003 Environmental Technology Verification Report INDUSTRIAL TEST SYSTEMS, INC. QUICK™ LOW RANGE TEST KIT Prepared by Battelle Under a cooperative agreement with U.S. Environmental Protection Agency August 2003 Environmental Technology Verification Report ETV Advanced Monitoring Systems Center Industrial Test Systems, Inc. Quick™ Low Range Test Kit by Tim Kaufman Patty White Amy Dindal Zachary Willenberg Karen Riggs Battelle Columbus, Ohio 43201 ii Notice The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development, has financially supported and collaborated in the extramural program described here. This document has been peer reviewed by the Agency and recommended for public release. Mention of trade names or commercial products does not constitute endorsement or recommendation by the EPA for use. iii Foreword The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the nation’s air, water, and land resources. Under a mandate of national environmental laws, the Agency strives to formulate and implement actions leading to a compatible balance between human activities and the ability of natural systems to support and nurture life. To meet this mandate, the EPA’s Office of Research and Development provides data and science support that can be used to solve environmental problems and to build the scientific knowledge base needed to manage our ecological resources wisely, to understand how pollutants affect our health, and to prevent or reduce environmental risks. The Environmental Technology Verification (ETV) Program has been established by the EPA to verify the performance characteristics of innovative environmental technology across all media and to report this objective information to permitters, buyers, and users of the technology, thus substantially accelerating the entrance of new environmental technologies into the marketplace. Verification organizations oversee and report verification activities based on testing and quality assurance protocols developed with input from major stakeholders and customer groups associated with the technology area. ETV consists of seven environmental technology centers and two pilot programs. Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/. Effective verifications of monitoring technologies are needed to assess environmental quality and to supply cost and performance data to select the most appropriate technology for that assessment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA funding and support to plan, coordinate, and conduct such verification tests for “Advanced Monitoring Systems for Air, Water, and Soil” and report the results to the community at large. Information concerning this specific environmental technology area can be found on the Internet at http://www.epa.gov/etv/centers/center1.html. iv Acknowledgments The authors wish to acknowledge the support of all those who helped plan and conduct the verification test, analyze the data, and prepare this report. In particular we would like to thank Rosanna Buhl, Adam Abbgy, and Bea Weaver of Battelle, and Mike Madigan and Rick Linde of the Ayer, Massachusetts Department of Public Works Water Division. We also acknowledge the assistance of Jeff Adams of EPA and AMS Center stakeholders Vito Minei and Marty Link, who reviewed the verification reports. v Contents Page Notice.....................................................................................................................................................ii Foreword...............................................................................................................................................iii Acknowledgments ...............................................................................................................................iv List of abbreviations ..........................................................................................................................viii 1 Background.....................................................................................................................................1 2 Technology Description .................................................................................................................2 3 Test Design and Procedures...........................................................................................................4 3.1 Introduction ...........................................................................................................................4 3.2 Test Design ............................................................................................................................4 3.3 Test Samples..........................................................................................................................5 3.3.1 QC Samples...............................................................................................................5 3.3.2 PT Samples................................................................................................................7 3.3.3 Environmental Samples............................................................................................7 3.4 Reference Analysis................................................................................................................8 3.5 Verification Schedule............................................................................................................8 4 Quality Assurance/Quality Control...............................................................................................9 4.1 Laboratory QC for Reference Method.................................................................................9 4.2 Audits...................................................................................................................................11 4.2.1 Performance Evaluation Audit...............................................................................11 4.2.2 Technical Systems Audit........................................................................................12 4.2.3 Data Quality Audit..................................................................................................12 4.3 QA/QC Reporting ...............................................................................................................13 4.4 Data Review ........................................................................................................................13 5 Statistical Methods .......................................................................................................................14 5.1 Accuracy ..............................................................................................................................14 5.2 Precision...............................................................................................................................15 5.3 Linearity ..................................................................................................................15 5.4 Method Detection Limit .........................................................................................15 5.5 Matrix Interference Effects ....................................................................................15 5.6 Operator Bias ..........................................................................................................16 5.7 Inter-Unit Reproducibility......................................................................................16 5.8 Rate of False Positives/False Negatives................................................................16 vi 6 Test Results ...................................................................................................................................17 6.1 QC Samples .........................................................................................................................17 6.2 PT and Environmental Samples .........................................................................................22 6.2.1 Accuracy..................................................................................................................22 6.2.2 Precision ..................................................................................................................27 6.2.3 Linearity ..................................................................................................................27 6.2.4 Method Detection Limit .........................................................................................31 6.2.5 Matrix Interference Effects ....................................................................................32 6.2.6 Operator Bias ..........................................................................................................32 6.2.7 Inter-Unit Reproducibility......................................................................................33 6.2.8 Rate of False Positives/False Negatives................................................................33 6.3 Other Factors .......................................................................................................................36 6.3.1 Ease of Use..............................................................................................................37 6.3.2 Analysis Time .........................................................................................................37 6.3.3 Reliability ................................................................................................................37 6.3.4 Waste Material........................................................................................................37 6.3.5 Cost..........................................................................................................................37 7 Performance Summary.................................................................................................................38 8 References.....................................................................................................................................41 Figures Figure 2-1. Industrial Test Systems, Inc., Quick™ Low Range Test Kit ......................................2 Figure 2-2. Quick™Low Range Color Chart ...................................................................................3 Figure 6-1. Linearity of Quick™ Low Range Color Chart Results ..............................................30 Figure 6-2. Linearity of Quick™ Low Range Quick™ Arsenic Scan Results.............................31 Figure 6-3. Comparison of Quick™ Low Range Test Results for Technical and Non-Technical Operators .............................................................................................33 Figure 6-4. Comparison of Two Quick™ Arsenic Scan Units......................................................34 Tables Table 3-1. Test Samples for Verification of the Quick™ Low Range Test Kit ............................6 Table 3-2. Schedule of Verification Test Days ...............................................................................8 Table 4-1. Reference Method QCS Analysis Results...................................................................10 Table 4-2. Reference Method LFM Sample Results.....................................................................11 vii Table 4-3. Reference Method Duplicate Analysis Results...........................................................12 Table 4-4. Reference Method PE Audit Results ...........................................................................12 Table 4-5. Summary of Data Recording Process ..........................................................................13 Table 6-1a. RB Sample Results for the Technical Operator ..........................................................18 Table 6-1b. RB Sample Results for the Non-Technical Operator..................................................18 Table 6-2a. QCS Results for the Technical Operator .....................................................................19 Table 6-2b. QCS Results for the Non-Technical Operator.............................................................20 Table 6-3a. LFM Sample Results for the Technical Operator .......................................................21 Table 6-3b. LFM Sample Results for the Non-Technical Operator...............................................22 Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results...................................23 Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range Test Kits .................26 Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits ................28 Table 6-7. Precision Results for Quick™ Low Range Test Kits .................................................30 Table 6-8. Detection Limit Results for Quick™ Low Range Test Kit ........................................32 Table 6-9. Rate of False Positives for Quick™ Low Range Test Kits ........................................35 Table 6-10. Rate of False Negatives for Quick™ Low Range Test Kits ......................................36 Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results .......39 viii List of Abbreviations AMS Advanced Monitoring Systems EPA U.S. Environmental Protection Agency ETV Environmental Technology Verification HDPE high-density polyethylene ICPMS inductively coupled plasma mass spectrometry LFM laboratory-fortified matrix MDL method detection limit MSDS Material Safety Data Sheet NIST National Institute of Standards and Technology ppb parts per billion ppm parts per million PE performance evaluation PT performance test QA quality assurance QA/QC quality assurance/quality control QC quality control QCS quality control standard QMP Quality Management Plan RB reagent blank RPD relative percent difference RSD relative standard deviation TSA technical systems audit 1 Chapter 1 Background The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV) Program to facilitate the deployment of innovative environmental tech-nologies through performance verification and dissemination of information. The goal of the ETV Program is to further environmental protection by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-quality, peer-reviewed data on technology performance to those involved in the design, distribution, financing, permitting, purchase, and use of environmental technologies. ETV works in partnership with recognized testing organizations; with stakeholder groups consisting of buyers, vendor organizations, and permitters; and with the full participation of individual technology developers. The program evaluates the performance of innovative technologies by developing test plans that are responsive to the needs of stakeholders, con-ducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA) protocols to ensure that data of known and adequate quality are generated and that the results are defensible. The EPA’s National Exposure Research Laboratory and its verification organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center recently evaluated the performance of the Industrial Test Systems, Inc., Quick™ Low Range test kit for measuring arsenic in water. 2 Chapter 2 Technology Description The objective of the ETV AMS Center is to verify the performance characteristics of environ-mental monitoring technologies for air, water, and soil. This verification report provides results for the verification testing of the Quick™ Low Range test kit for arsenic in water (Figure 2-1). The following is a description of the test kit, based on information provided by the vendor. The information provided below was not verified in this test. To perform arsenic analyses with the Quick™ Low Range test kit, the water sample to be tested is mixed in the supplied reaction vessel with reagent #1 (tartaric acid with rate enhancers) to acidify the water sample. Reagent #2, an oxidizer (potassium peroxymonosulfate), is added to remove hydrogen sulfide inter-ference. The test tolerates up to 2 parts per million (ppm) hydrogen sulfide without interference. Zinc powder, reagent #3, is added to reduce inorganic arsenic compounds (As+3 and As+5) to arsine gas. As arsine gas is generated and comes in contact with the test strip, the mercuric bromide indicator on the test strip changes color from white to shades of yellow or brown. Material Safety Data Sheets (MSDSs) for all reagents and test strips are provided with each test kit. The MSDSs include information on how to safely handle the reagents and test strips, including instructions for exposure controls and personal protection. Once the reaction is completed, the test strip is removed and visually compared to a color chart to obtain a semi-quantitative measure of the arsenic concentration in the tested sample. The color chart consists of a series of color blocks (Figure 2-2). The color blocks correspond to concentrations ranging from 3 parts per billion (ppb) to >80 ppb. If the color on the test strip is between two color blocks, then the operator may estimate the concentration as between the values associated with the color blocks on either side. The test strip may also be read with the Quick™ Arsenic Scan hand-held instrument, which operates on the same principle as a colorimeter and provides a quantitative result. The Quick™ Arsenic Scan is calibrated weekly using a calibration card provided by the manufacturer. The Quick™ Arsenic Scan is not provided Figure 2-1. Industrial Test Systems, Inc., Quick™ Low Range Test Kit 3 with the test kit as a standard feature. The standard test kit with the color chart was the subject of this verification test; however, results for the Quick™ Arsenic Scan are also provided. The optimal detection range for the Quick™ Low Range test kit is below 20 ppb. Dilution instructions are provided for samples with arsenic levels above 30 ppb. The recommended temperature range for sample analysis is 24°C to 30°C. A modified testing protocol that specifies longer reaction times (up to 30 minutes longer for samples between 5°-15°C) is available for sample temperatures below this range. The kits are available in two sizes: two tests and 50 tests. The typical shelf life of the kits is 24 months. Figure 2-2. Quick™ Low Range Color Chart 4 Chapter 3 Test Design and Procedures 3.1 Introduction This verification test was conducted according to procedures specified in the Test/QA Plan for Verification of Portable Analyzers.(1) The verification was based on comparing the arsenic results from the Quick™ Low Range test kit to those from a laboratory-based reference method. The reference method for arsenic analysis was inductively coupled plasma mass spectrometry (ICPMS) performed according to EPA Method 200.8(2) Semi-quantitative results for arsenic were obtained from the Quick™ Low Range test kit by comparisons of test strips to a color chart provided with the test kit. Quantitative results also were obtained using a Quick™ Arsenic Scan instrument. The test kit performance was verified by analyzing laboratory-prepared performance test samples, treated and untreated drinking water, and fresh surface water. All samples were tested using both the test kit and the reference method. Both semi-quantitative and quantitative analyses were performed by the technical and non-technical operators. The test design and procedures are described below. 3.2 Test Design The Quick™ Low Range test kit was verified by evaluating the following parameters: § Accuracy § Precision § Linearity § Method detection limit (MDL) § Matrix interference effects § Operator bias § Inter-unit reproducibility § Rate of false positives/false negatives. All sample preparation and analyses were performed according to the manufacturer’s recommended procedures. All samples were warmed to 24°C prior to analysis using a hot water bath, which is at the lower end of the optimal temperature range listed in the test kit instructions. Color chart and Quick™ Arsenic Scan results were recorded manually. The results from the Quick™ Low Range test kits were compared to those from the reference method to assess accuracy and linearity. Multiple aliquots of performance test samples, drinking water samples, and surface water samples were analyzed to assess precision. Multiple aliquots of a low-level performance test sample were analyzed to assess the detection limit of the method. Potential 5 matrix interference effects were assessed by challenging the test kit with performance test samples of known arsenic concentrations that contained both low levels and high levels of interfering substances. Identical sets of samples were analyzed independently by a technical and a non-technical operator. The technical operator was a technician at Battelle with three years of field and laboratory experience and a Bachelors degree. The non-technical operator was a part-time temporary helper enrolled in undergraduate studies. Because the reagents of the Quick™ Low Range test kits were consumed in use, it was not feasible for the two operators to use the same kits; however, each operator used multiple kits in order to analyze all the samples and it is assumed that kit-to-kit variability was similar for both operators. Results of all analyses were statistically compared to evaluate operator bias. The technical operator analyzed all samples using two different Quick™ Arsenic Scan units to assess inter-unit reproducibility. The rate of false positive and false negative results were evaluated relative to the 10-ppb maximum contaminant level for arsenic in drinking water.(4) Other factors that were quail-tatively assessed during the test included time required for sample analysis, ease of use, and reliability. 3.3 Test Samples Three types of samples were analyzed in the verification test, as shown in Table 3-1: quality control (QC) samples, performance test (PT) samples, and environmental water samples. The QC and PT samples were prepared from National Institute of Standards and Technology (NIST) traceable standards purchased from a commercial supplier and subject only to dilution as appropriate. Under the Safe Drinking Water Act, the EPA lowered the maximum contaminant level for arsenic from 50 ppb to 10 ppb in January 2001; public water supply systems must comply with this standard by January 2006.(4) Therefore, the QC sample concentrations targeted the 10 ppb arsenic level. The PT samples ranged from 10% to 1,000% of the 10 ppb level (i.e., from 1 ppb to 100 ppb). The environmental water samples were collected from various drinking water and surface freshwater sources. Each sample was assigned a unique sample identification number when prepared in the laboratory or collected in the field. The PT and environmental samples were submitted blind to the technical and non-technical operators and were analyzed randomly to the degree possible. 3.3.1 QC Samples QC samples included laboratory reagent blank samples (RB), quality control samples (QCS), and laboratory-fortified matrix (LFM) samples (Table 3-1). The RB samples consisted of the same ASTM Type I water used to prepare all other samples and were subjected to the same handling and analysis procedures as the other samples. The RB samples were used to verify that no arsenic contamination was introduced during sample handling and analysis. RB samples were analyzed at a frequency of 10%. 6 Table 3-1. Test Samples for Verification of the Quick™ Low Range Test Kit Type of Sample Sample Characteristics Arsenic Concentration (a) No. of Replicates Reagent Blank (RB) ~ 0 ppb 10% of all Quality Control Sample (QCS) 10 ppb 10% of all Quality Control Laboratory Fortified Matrix (LFM) 10 ppb above native level 1 per site Prepared arsenic solution 1 ppb 4 Prepared arsenic solution 3 ppb 4 Prepared arsenic solution 10 ppb 4 Prepared arsenic solution 30 ppb 4 Prepared arsenic solution 100 ppb 4 Prepared arsenic solution for detection limit determination 15 ppb 7 Prepared arsenic solution spikedwith low levels of interfering substances 10 ppb 4 Performance Test Prepared arsenic solution spiked with high levels of interfering substances 10 ppb 4 Battelle drinking water <0.5 ppb 4 Ayer untreated water 64.8 ppb 4 Ayer treated water 1.39 ppb 4 Falmouth Pond water <0.5 ppb 4 Environmental Taunton River water 1.31 ppb 4 (a) Performance Test sample concentrations are target levels; environmental sample concentrations are actual (average of four replicate measurements). The QCS consisted of ASTM Type I water spiked in the lab to a concentration of 10 ppb arsenic with a NIST-traceable standard. QCS were used as calibration checks to verify that the Quick™ Low Range test kit was operating properly. QCS were analyzed at the beginning and end of each testing period, as well as after every tenth sample. Because the test kit utilized a color chart that could not be calibrated, no performance criteria were specified for the QCS. The LFM samples consisted of aliquots of environmental samples that were spiked in the field to increase the arsenic concentration by 10 ppb. The spike solution used for the LFM samples was prepared in the laboratory and brought to the field site. One LFM sample was prepared from each environmental sample. 7 3.3.2 PT Samples Three types of PT samples used in this verification test (Table 3-1): spiked samples ranging from 1 ppb to 100 ppb arsenic, a low-level spiked sample for evaluation of the test kit’s detection limit, and matrix interference samples that were spiked with potential interfering substances. All PT samples were prepared in the laboratory using ASTM Type I water and NIST-traceable standards. Five PT samples containing arsenic at concentrations from 1 ppb to 100 ppb were prepared to evaluate Quick™ Low Range test kit accuracy and linearity. Four aliquots of each of these samples were analyzed to assess precision. To determine the detection limit of the Quick™ Low Range test kit, a sample with an arsenic concentration five times the vendor’s estimated detection limit was prepared. Seven non-consecutive replicates of this 15 ppb arsenic sample were analyzed to provide precision data with which to estimate the MDL. The matrix interference samples were spiked with 10 ppb arsenic as well as potentially inter-fering species commonly found in natural water samples. One sample contained low levels of interfering substances that consisted of 1 ppm iron, 3 ppm sodium chloride, and 0.1 ppm sulfide. The second sample contained high levels of interfering compounds at the following concentrations: 10 ppm iron, 30 ppm sodium chloride, and 1.0 ppm sulfide. Four replicates of each of these samples were analyzed. 3.3.3 Environmental Samples The environmental samples listed in Table 3-1 included three drinking water samples and two surface water samples. All environmental samples were collected in 20-L high density poly-ethylene (HDPE) containers. The Battelle groundwater sample was collected directly from a tap without purging. Untreated and treated groundwater samples from the Ayer, Massachusetts Department of Public Works Water Treatment Plant were collected directly from spigots, also without purging. Four aliquots of each sample were analyzed using the Quick™ Low Range test kit in the Battelle laboratory as soon as possible after collection. One aliquot of each sample was preserved with nitric acid and submitted to the reference laboratory for reference analysis. One surface water sample was collected from a pond in Falmouth, Massachusetts and another was collected from the Taunton River near Bridgewater, Massachusetts. These samples were collected near the shoreline by submerging a 2-L HDPE sample container no more than one inch below the surface of the water, and decanting the water into a 20-L HDPE carboy until full. Each water body was sampled at one accessible location. These samples could not be analyzed at the field location as planned because of persistent, severe winter weather conditions. Therefore, the samples were returned to a storage shed at the Battelle laboratory, which was heated but was not serviced by running water. The storage shed was intended to simulate realistic field conditions under which the test kits might be used. Four aliquots of each surface water sample were analyzed in the storage shed as soon as possible after collection. One aliquot of each sample was preserved with nitric acid and submitted to the reference laboratory. 8 3.4 Reference Analysis The reference arsenic analyses were performed in a Battelle laboratory using a Perkin Elmer Sciex Elan 6000 ICPMS according to EPA Method 200.8, Revision 5.5.(2) The sample was introduced through a peristaltic pump by pneumatic nebulization into a radiofrequency plasma where energy transfer processes caused desolvation, atomization, and ionization. The ions were extracted from the plasma through a pumped vacuum interface and separated on the basis of their mass-to-charge ratio by a quadrupole mass spectrometer. The ions transmitted through the quadrupole were registered by a continuous dynode electron multiplier, and the ion information was processed by a data handling system. The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using a minimum of five calibration standards at concentrations ranging between 0.5 and 250 ppb, and a required correlation coefficient of a minimum of 0.999. Internal standards were used to correct for instrument drift and physical interferences. These standards were introduced in line through the peristaltic pump and analyzed with all blanks, standards, and samples. 3.5 Verification Schedule The verification test took place from January 29 through February 24, 2003. Table 3-2 shows the daily activities that were conducted during this period. The reference analyses were performed on March 7 and March 14, 2003, five to six weeks after sample collection. Table 3-2. Schedule of Verification Test Days Sample Analysis Date Sample Collection Date Tech. Op. Non-Tech. Op. Testing Location Activity 1/29/03-2/12/03 1/29/03-2/12/03 1/29/03-2/12/03 Battelle Laboratory Preparation and analysis of PT and associated QC samples. 2/12/03 2/13/03 2/14/03 Battelle Laboratory Collection and analysis of Ayer untreated and treated water and associated QC samples. 2/17/03 2/18/03 2/17/03 Battelle Laboratory Collection and analysis of Battelle drinking water and associated QC samples. 2/21/03 2/21/03 2/21/03 Battelle Storage Shed Collection and analysis of Falmouth Pond water and associated QC samples. 2/23/03 2/24/03 2/24/03 Battelle Storage Shed Collection and analysis of Taunton River water and associated QC samples. 9 Chapter 4 Quality Assurance/Quality Control Quality assurance/quality control (QA/QC) procedures were performed in accordance with the quality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verification test.(1) QA/QC procedures and results are described below. 4.1 Laboratory QC for Reference Method Reference analyses were conducted on March 7 and March 14, 2003. Laboratory QC for the reference method included the analysis of RB, QCS, LFM and analytical duplicate samples. Laboratory RB samples were analyzed to ensure that no contamination was introduced by the sample preparation and analysis process. The test/QA plan stated that if arsenic was detected in a RB sample above the MDL for the reference instrument, then the contamination source would be identified and removed and proper blank readings achieved before proceeding with the reference analyses. All of the laboratory RB samples analyzed were below the reporting limit for arsenic (i.e., below the concentration of the lowest calibration standard) except for several blanks that were analyzed at the end of the day on March 7. Three of the six test samples that were associated with these RB samples were re-analyzed on March 14, with acceptable RB sample results. The other three test samples had arsenic concentrations that were approximately twenty times higher than the RB sample concentrations; therefore, no action was taken. On March 7, the instrument used for the reference method was calibrated using nine calibration standards, with concentrations ranging from 0.5 to 250 ppb arsenic. On March 14, it was calibrated using eight standards ranging in concentration from 0.1 to 25 ppb arsenic for more accurate analysis of low level samples. The accuracy of the calibration was verified after the analysis of every 10 samples by analyzing a QCS of a known concentration. The percent recovery of the QCS was calculated from the following equation: 100´=sCR s where Cs is the measured concentration of the QCS and s is the spike concentration. If the QCS analysis differed by more than 10% from the true value of the standard, the instrument was recalibrated before continuing the test. As shown in Table 4-1, all QCS analyses were within the required range. (1) 10 Table 4-1. Reference Method QCS Analysis Results Sample ID Analysis Date Measured (ppb) Actual (ppb) Percent Recovery CCV 25 3/7/2003 24.96 25.00 100% QCS 25 3/7/2003 26.81 25.00 107% CCV 25 3/7/2003 24.50 25.00 98% CCV 25 3/7/2003 25.39 25.00 102% CCV 25 3/7/2003 25.73 25.00 103% CCV 25 3/7/2003 25.81 25.00 103% CCV 25 3/7/2003 25.64 25.00 103% CCV 25 3/7/2003 25.30 25.00 101% CCV 25 3/7/2003 24.90 25.00 100% CCV 25 3/7/2003 22.67 25.00 91% QCS 25 3/14/2003 24.90 25.00 100% CCV 2.5 3/14/2003 2.74 2.50 110% QCS 2.5 3/14/2003 2.70 2.50 108% CCV 2.5 3/14/2003 2.58 2.50 103% CCV 2.5 3/14/2003 2.65 2.50 106% CCV 2.5 3/14/2003 2.66 2.50 106% CCV 2.5 3/14/2003 2.61 2.50 104% CCV 2.5 3/14/2003 2.60 2.50 104% LFM samples were analyzed to assess whether matrix effects influenced the reference method results. The LFM percent recovery (R) was calculated from the following equation: 100´-=sCCR s where Cs is the measured concentration of the spiked sample, C is the measured concentration of the unspiked sample, and s is the spike concentration. If the percent recovery of an LFM sample fell outside the range from 85% to 115%, a matrix effect was suspected. As shown in Table 4-2, all of the LFM sample results were within this range. Duplicate samples were analyzed to assess the precision of the reference analysis. The relative percent difference (RPD) of the duplicate sample analysis was calculated from the following equation: 1002/)()(´+-=DDCCCCRPD where C is the concentration of the sample analysis, and CD is the concentration of the duplicate sample analysis. If the RPD was greater than 10%, the instrument was recalibrated before (3) (2) 11 Table 4-2. Reference Method LFM Sample Results Sample ID Matrix Analysis Date Unspiked (ppb) Spiked (ppb) Amount Spiked (ppb) Percent Recovery CAA-22 ASTM Type I water 3/7/2003 11.02 37.20 25.00 105% CAA-25 R4 ASTM Type I water 3/7/2003 0.95 22.76 25.00 87% CAA-28 R2 ASTM Type I water 3/7/2003 3.45 30.64 25.00 109% CAA-29 R4 ASTM Type I water 3/7/2003 34.98 60.37 25.00 102% CAA-37 R4 Drinking water 3/7/2003 0.52 28.20 25.00 111% CAA-41 R4 Drinking water 3/7/2003 1.24 28.88 25.00 111% CAA-48 Surface water 3/7/2003 12.26 39.40 25.00 109% CAA-47 R4 Surface water 3/7/2003 1.07 28.41 25.00 109% CAA-27 R1 ASTM Type I water 3/14/2003 2.56 4.73 2.50 87% CAA-37 R3 Drinking water 3/14/2003 0.45 3.11 2.50 107% CAA-47 R1 Surface water 3/14/2003 1.36 4.16 2.50 112% CAA-88 R3 Drinking water 3/14/2003 0.43 3.16 2.50 109% CAA-88 R4 Drinking water 3/14/2003 0.42 3.18 2.50 111% continuing the test. As shown in Table 4-3, the RPDs for the duplicate analyses were all less than 10%. The RPD for one duplicate pair was 9.5%; however, the sample concentrations were below the reporting limit for the reference method (i.e., below the concentration of the lowest calibration standard). 4.2 Audits Three types of audits were performed during the verification test: a performance evaluation (PE) audit of the reference method, a technical systems audit of the verification test performance, and a data quality audit. Audit procedures are described further below. 4.2.1 Performance Evaluation Audit A PE audit was conducted to assess the quality of the reference measurements made in this verification test. For the PE audit, an independent, NIST-traceable, reference material was obtained from a different commercial supplier than the calibration standards and the standard used to prepare the PT and field QCS samples. Accuracy of the reference method was verified by comparing the arsenic concentration measured using the calibration standards to that obtained using the independently-certified PE standard. Relative percent difference as calculated by Equation 3 was used to quantify the accuracy of the results. Agreement of the standard within 10% was required for the measurements to be considered acceptable. As shown in Table 4-4, the PE sample analysis was within the required range. 12 Table 4-3. Reference Method Duplicate Analysis Results Sample ID Analysis Date Sample concentration (ppb) Duplicate concentration (ppb) Relative Percent Difference CAA-4 3/7/2003 9.33 9.20 1.4% CAA-70 3/7/2003 10.93 10.82 1.0% CAA-26 R1 3/7/2003 1.14 1.13 1.4% CAA-28 R3 3/7/2003 3.49 3.45 1.1% CAA-31 R1 3/7/2003 111.89 112.20 0.3% CAA-38 3/7/2003 11.96 11.90 0.5% CAA-42 3/7/2003 13.02 13.06 0.3% CAA-48 3/7/2003 12.26 12.22 0.4% CAA-23 3/14/2003 3.03 2.99 1.3% CAA-27 R2 3/14/2003 2.64 2.61 0.9% CAA-37 R4 3/14/2003 0.44 0.43 2.3% CAA-47 R2 3/14/2003 1.31 1.32 0.2% CAA-88 R4 3/14/2003 0.42 0.38 9.5% Table 4-4. Reference Method PE Audit Results Sample ID Date of Analysis Measured Arsenic Concentration (ppb) Actual Arsenic Concentration (ppb) Percent Difference PE-1 3/24/03 9.63 10.0 4 4.2.2 Technical Systems Audit An independent Battelle quality staff conducted a technical systems audit (TSA) from February 3 to 6 to ensure that the verification test was being conducted in accordance with the test/QA plan(1) and the AMS Center QMP.(3) A TSA of the reference method performance was con-ducted by the Battelle Quality Manager on March 5, 2003, when the reference analyses were initiated. As part of the TSA, test procedures were compared to those specified in the test/QA plan, data acquisition and handling procedures were reviewed, and the reference standards and method were reviewed. Observations and findings from the TSA were documented and sub-mitted to the Battelle Verification Test Coordinator for response. None of the TSA findings required corrective action. TSA records are permanently stored with the Battelle Quality Manager. 4.2.3 Data Quality Audit At least 10% of the data acquired during the verification test was audited. Battelle’s Quality Manager traced the data from the initial acquisition, through reduction and statistical analysis, to 13 final reporting to ensure the integrity of the reported results. All calculations performed on the data undergoing the audit were checked. 4.3 QA/QC Reporting Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV AMS Center.(3) Once the audit reports were prepared, the Battelle Verification Test Coordinator ensured that a response was provided for each adverse finding or potential problem and imple-mented any necessary follow-up corrective action. The Battelle Quality Manager ensured that follow-up corrective action was taken. The results of the TSA and the data quality audit were submitted to the EPA. 4.4 Data Review Records generated in the verification test received a review before these records were used to calculate, evaluate, or report verification results. Table 4-5 summarizes the types of data recorded and reviewed. All data were recorded by Battelle staff. Data were reviewed by a Battelle technical staff member involved in the verification test, but not the staff member that originally generated the record. The person performing the review added his/her initials and the date to a hard copy of the record being reviewed. Review of some of the test data sheets occurred outside of the two week period specified in the test/QA plan. Table 4-5. Summary of Data Recording Process Data to be Recorded Where Recorded How Often Recorded Disposition of Data(a) Dates, times of test events ETV field data sheets Start/end of test event Used to organize/check test results; manually incorporated in data spreadsheets as necessary Test parameters (temperature, analyte/ interferant identities, and Quick™ Low Range test kit and Quick™ Arsenic Scan results) ETV field data sheets When set or changed, or as needed to document test Used to organize/check test results, manually incorporated in data spreadsheets as necessary Reference method sample analysis, chain of custody, and results Laboratory record books, data sheets, or data acquisition system, as appropriate Throughout sample handling and analysis process Transferred to spreadsheets (a) All activities subsequent to data recording are carried out by Battelle. 14 Chapter 5 Statistical Methods The statistical methods used to evaluate the performance factors listed in Section 3.2 are presented in this chapter. Qualitative observations were also used to evaluate verification test data. 5.1 Accuracy All samples were analyzed by both the Quick™ Low Range test kit and reference methods. For each sample, accuracy was expressed in terms of a relative bias (B) as calculated from the following equation: 100xCdBR= where d is the average difference between the reading from the Quick™ Low Range test kit and those from the reference method, and RC is the average of the reference measurements. An additional assessment of accuracy was conducted for the color chart results because of the semi-quantitative nature of the visual comparisons. Each color in the chart represents a concentration range. Performance was assessed by determining whether the result falls within the expected concentration range as measured by the reference analysis. Overall agreement was assessed by calculating the percent of results that fell within the correct range, calculated from the following equation: 100´=nYA where A is the percent of measurements in agreement, Y is the number of measurements within the expected color range, and n is the total number of measurements. Readings below the vendor-stated detection limit of the test kit (e.g., <3 ppb) were judged to be in agreement with the reference result if the reference value was in the specified “less than” range. (4) (5) 15 5.2 Precision When possible, the standard deviation (S) of the results for the replicate samples was calculated and used as a measure of Quick™ Low Range test kit precision at each concentration. Standard deviation was calculated from the following equation: 2112)(11úûùêëé --= å=nkk CCnS where n is the number of replicate samples, Ck is the concentration measured for the kth sample, and C is the average concentration of the replicate samples. Precision was reported in terms of the relative standard deviation (RSD) as follows: 100´=CSRSD 5.3 Linearity Linearity was assessed by performing a linear regression of Quick™ Low Range test kit results against the reference results, with linearity characterized by the slope, intercept, and correlation coefficient (R). Linearity was tested using the five PT samples over the range of 1 ppb to 100 ppb arsenic. Samples with results below the vendor-stated detection limit were not included. Both color chart results and Arsenic Quick™ Scan results were plotted against the corresponding mean reference concentrations and separate regressions were performed. 5.4 Method Detection Limit The MDL for the Quick™ Low Range test kit was assessed using results from both detection methods (color chart and Quick™ Arsenic Scan) for seven replicate analyses of a sample spiked with approximately 15 ppb arsenic. The standard deviation of the seven replicate samples was calculated using Equation (6). The MDL was calculated using the following equation: StMDL ´= where t is the Student’s t value for a 99% confidence level and S is the standard deviation of the seven replicate samples. 5.5 Matrix Interference Effects The potential effect of interfering substances on the sensitivity of the Quick™ Low Range test kit was evaluated by the calculating accuracy (expressed as bias) using Equation 4. These results (6) (7) (8) 16 were qualitatively compared with accuracy results for PT samples containing only arsenic to assess whether there was a positive or negative effect due to matrix interferences. 5.6 Operator Bias Potential operator bias for the Quick™ Low Range test kit was assessed by performing a linear regression of sample results above the detection limit generated by the technical and non-technical operator. Color chart and Quick™ Arsenic Scan results were evaluated. The slope, intercept, and correlation coefficient were used to evaluate the degree of operator bias. A paired t-test was also conducted to evaluate whether the two sets of sample results were significantly different at a 95% confidence level. 5.7 Inter-Unit Reproducibility Inter-unit reproducibility for the Quick™ Low Range test kit with the Quick™ Arsenic Scan device was assessed by performing a linear regression of sample results generated by the two units that were used by the technical operator. The slope, intercept, and correlation coefficient were used to evaluate the degree of inter-unit reproducibility. A paired t-test was also conducted to evaluate whether the two sets of sample results were significantly different at a 95% confidence level. 5.8 Rate of False Positives/False Negatives The rates of false positives and false negatives produced by the Quick™ Low Range test kit were assessed relative to the 10-ppb target arsenic level. A false positive result is defined as any result reported to be greater than or equal to the guidance level (10 ppb) and greater than 125% of the reference value, when the reference value is less than that guidance level. Similarly, a false negative result is defined as any result reported below the guidance level and less than 75% of the reference value, when the reference value is greater than the guidance level. The rates of false positives and false negatives were expressed as a percentage of total samples analyzed for each type of sample. 17 Chapter 6 Test Results The results of the verification test of the Quick™ Low-Range Arsenic test kits are presented in this section. 6.1 QC Samples As described in Section 3.3.1, the QC samples analyzed with the Quick™ Low Range test kit included RB samples, QCS, and LFM samples (these QC samples were different than those analyzed in conjunction with the reference analyses). The RB samples were analyzed at a frequency of 10% and results were used to verify that no arsenic contamination was introduced during sample handling and analysis. QCS were analyzed at the beginning and end of each test period, and after every tenth sample. The QCS results were used to verify that the test kit was operating properly. One LFM sample was prepared from each environmental sample to evaluate potential matrix interferences. Acceptance criteria for test kit QC samples were not specified in the test/QA plan because modifications to the technology would not be made during testing. RB sample results for the technical and non-technical operators are presented in Tables 6-1a and 6-1b, respectively. Unique sample identification codes were assigned to each container of ASTM Type I water that was used. The RB samples were analyzed at the required frequency except on the first day of testing, when the RB sample was inadvertently omitted by the technical operator. The technical operator recorded one detected value for a RB sample using the color chart; how-ever, all Quick™ Arsenic Scan readings were below detection. The non-technical operator reported detected values for two RB samples using the color chart on the first two days of testing; however, all Quick™ Arsenic Scan readings were below detection. The RB samples were not analyzed by the reference laboratory. Because the color chart detections for several RB samples were not confirmed by the Quick™ Arsenic Scan, it was concluded that arsenic contamination resulting from sample handling and analysis did not occur. QCS results for the technical and non-technical operators are presented in Tables 6-2a and 6-2b, respectively. The QCS were analyzed at the required frequency except on the first day of testing, when the technical operator inadvertently omitted these samples. The percent recovery of the QCS was calculated using Equation 1 (Section 4.1). The QCS percent recovery for the technical operator ranged from 100% to 130% for the color chart and from 64% to 144% for the Quick™ Arsenic Scan. The QCS percent recovery for the non-technical operator ranged from 18% to 150% for the color chart and from 40% to 90% for the Quick™ Arsenic Scan. Overall the test kits and Quick™ Arsenic Scan appeared to be operating as expected. 18 Table 6-1a. RB Sample Results for the Technical Operator(a) Sample ID Replicate Analysis Date Color chart (ppb) Quick™ Arsenic Scan #1 (ppb) Quick™ Arsenic Scan #2 (ppb) CAA-6 1 1/30/2003 <3 <1 <1 CAA-12 1 2/4/2003 <3 <1 <1 CAA-13 1 2/6/2003 3 <1 <1 CAA-49 1 2/6/2003 <3 <1 <1 CAA-52 1 2/10/2003 <3 <1 <1 CAA-54 1 2/13/2003 <3 <1 <1 CAA-54 2 2/13/2003 <3 <1 <1 CAA-57 1 2/18/2003 <3 <1 <1 CAA-58 1 2/21/2003 <3 <1 <1 CAA-59 1 2/24/2003 <3 <1 <1 (a) The technical operator inadvertently omitted the RB sample on the first day of testing (1/29/03). Table 6-1b. RB Sample Results for the Non-Technical Operator Sample ID Replicate Analysis Date Color Chart (ppb) Quick™ Arsenic Scan #1 (ppb) CAA-3 1 1/29/2003 8 <1 CAA-6 1 1/30/2003 6 <1 CAA-10 1 2/3/2003 <3 <1 CAA-13 1 2/6/2003 <3 <1 CAA-49 1 2/6/2003 <3 <1 CAA-53 1 2/11/2003 <3 <1 CAA-55 1 2/14/2003 <3 <1 CAA-55 2 2/14/2003 <3 <1 CAA-56 1 2/17/2003 <3 <1 CAA-58 1 2/21/2003 <3 <1 CAA-59 1 2/24/2003 <3 <1 19 Table 6-2a. QCS Results for the Technical Operator(a) Sample ID Replicate Analysis Date Color chart (ppb) Quick™ Arsenic Scan #1 (ppb) Quick™ Arsenic Scan #2 (ppb) Amount Spiked (ppb) Percent Recovery Color Chart Percent Recovery Quick™ Arsenic Scan #1 Percent Recovery Quick™ Arsenic Scan #2 CAA-14 1 1/30/2003 11 10.4 10.4 10.0 110% 104% 104% CAA-14 2 1/30/2003 10 10.4 9.7 10.0 100% 104% 97% CAA-16 1 2/4/2003 11 10.4 10.4 10.0 110% 104% 104% CAA-16 2 2/4/2003 12 11.2 14.4 10.0 120% 112% 144% CAA-17 1 2/6/2003 11 8.0 9 10.0 110% 80% 90% CAA-17 2 2/6/2003 11 9.7 10.4 10.0 110% 97% 104% CAA-19 1 2/10/2003 11 8.0 9.7 10.0 110% 80% 97% CAA-19 2 2/10/2003 11 6.4 7.3 10.0 110% 64% 73% CAA-20 1 2/13/2003 12 7.3 9 10.0 120% 73% 90% CAA-20 2 2/13/2003 12 7.3 9 10.0 120% 73% 90% CAA-21 1 2/18/2003 10 9.7 12.8 10.0 100% 97% 128% CAA-21 2 2/18/2003 13 9.7 10.4 10.0 130% 97% 104% CAA-71 1 2/21/2003 12 9.7 10.4 10.0 120% 97% 104% CAA-71 2 2/21/2003 13 9.0 9 10.0 130% 90% 90% CAA-72 1 2/24/2003 11 11.2 11.2 10.0 110% 112% 112% CAA-72 2 2/24/2003 12 9.0 9 10.0 120% 90% 90% (a) The technical operator inadvertently omitted the QCS on the first day of testing (1/29/03). 20 Table 6-2b. QCS Results for the Non-Technical Operator Sample ID Replicate Analysis Date Color Chart (ppb) Quick™ Arsenic Scan #1 (ppb) Amount Spiked (ppb) Percent Recovery Color Chart Percent Recovery Quick™ Arsenic Scan #1 CAA-4 1 1/29/2003 15 5.6 10.0 150% 56% CAA-4 2 1/29/2003 15 5.6 10.0 150% 56% CAA-14 1 1/30/2003 15 7.3 10.0 150% 73% CAA-16 1 2/3/2003 15 9 10.0 150% 90% CAA-16 2 2/3/2003 15 9 10.0 150% 90% CAA-17 1 2/6/2003 10 7.3 10.0 100% 73% CAA-17 2 2/6/2003 10 4.8 10.0 100% 48% CAA-19 1 2/11/2003 10 5.6 10.0 100% 56% CAA-19 2 2/11/2003 10 6.4 10.0 100% 64% CAA-20 1 2/14/2003 10 4 10.0 100% 40% CAA-20 2 2/14/2003 10 5.6 10.0 100% 56% CAA-21 1 2/17/2003 8 4 10.0 80% 40% CAA-21 2 2/17/2003 8 5.6 10.0 80% 56% CAA-70 1 2/21/2003 1.8 6.4 10.0 18% 64% CAA-70 2 2/21/2003 1.8 4 10.0 18% 40% CAA-72 1 2/24/2003 10 8 10.0 100% 80% CAA-72 2 2/24/2003 10 4.8 10.0 100% 48% 21 The LFM sample results for the technical and non-technical operators are presented in Tables 6-3a and 6-3b. The percent recovery associated with each LFM sample was calculated using Equation 2 (Section 4.1). Reference method results are also provided for comparison. The lowest recoveries measured by both operators were associated with the Ayer untreated water LFM sample. The reference method results also indicated a low recovery. The low recoveries indicate that a matrix interference may be adversely affecting the detection of arsenic. Consequently, test kit results for this sample may be biased low. Table 6-3a. LFM Sample Results for the Technical Operator Description Unspiked(a) (ppb) Spiked (ppb) Amount spiked (ppb) Percent Recovery Battelle drinking water LFM Color Chart <3 12 10 120% Quick™ Arsenic Scan #1 <1 7.3 10 73% Quick™ Arsenic Scan #2 <1 8 10 80% Reference <0.5 11.96 10 120% Ayer untreated water LFM Color Chart 40 50 10 100% Quick™ Arsenic Scan #1 10 5 10 -50% Quick™ Arsenic Scan #2 14.5 10 10 -45% Reference 64.82 69.74 10 49% Ayer treated water LFM Color Chart <3 10 10 100% Quick™ Arsenic Scan #1 <1 4 10 40% Quick™ Arsenic Scan #2 <1 4 10 40% Reference 1.39 13.02 10 116% Falmouth Pond LFM Color Chart <3 11 10 110% Quick™ Arsenic Scan #1 <1 9.7 10 97% Quick™ Arsenic Scan #2 <1 10.4 10 104% Reference <0.5 11.50 10 115% Taunton River LFM Color Chart <3 12 10 120% Quick™ Arsenic Scan #1 1.3 10.4 10 92% Quick™ Arsenic Scan #2 2.0 9.7 10 78% Reference 1.31 12.26 10 109% (a) Average of four replicates. Non-detects were assigned a value of zero. 22 Table 6-3b. LFM Sample Results for the Non-Technical Operator Description Unspiked(a) (ppb) Spiked (ppb) Amount Spiked (ppb) Percent Recovery Battelle drinking water LFM Color Chart <3 8 10 80% Quick™ Arsenic Scan #1 <1 4 10 40% Reference <0.5 11.96 10 120% Ayer untreated water LFM Color Chart 80 80 10 0% Quick™ Arsenic Scan #1 10 10 10 0% Reference 64.82 69.74 10 49% Ayer treated water LFM Color Chart <3 8 10 80% Quick™ Arsenic Scan #1 <1 2 10 20% Reference 1.39 13.02 10 116% Falmouth Pond LFM Color Chart <3 2.2 10 22% Quick™ Arsenic Scan #1 2.3 16 10 137% Reference <0.5 11.50 10 115% Taunton River LFM Color Chart <3 10 10 100% Quick™ Arsenic Scan #1 <1 10.4 10 104% Reference 1.31 12.26 10 109% (a) Average of four replicates. Non-detects were assigned a value of zero. 6.2 PT and Environmental Samples Table 6-4 presents the sample results for the PT and environmental samples. The table includes the Low Range Quick™ test kit results and the reference method results. The test kit results are shown for both the technical and non-technical operators, and the Quick™ Arsenic Scan Units #1 and #2. Sample results that were obtained from diluted samples are noted. Test kit results below the detection limit were assigned a value of <3 ppb for the color chart and <1 ppb for the Quick™ Arsenic Scan. The reporting limit for the reference analyses was 0.5 ppb, which corresponds to the lowest calibration standard used. Results for each performance factor are presented below. 6.2.1 Accuracy Table 6-5 presents the accuracy results for the Quick™ Low Range test kit, expressed as percent bias as calculated by Equation 4 (Section 5.1). Percent bias was not calculated for results below the detection limit. The four replicate analyses for each sample were averaged in the calculation of bias. The relative bias for the color chart ranged from -38% to 239% for the technical operator and -81% to 579% for the non-technical operator. The relative bias for the Quick™ Arsenic Scan 23 Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results Description Sample ID Replicate Technical Operator Color Chart (ppb) Technical Operator Quick™ Arsenic Scan #1 (ppb) Technical Operator Quick™ Arsenic Scan #2 (ppb) Non-Technical Operator Color Chart (ppb) Non-Technical Operator Quick™ Arsenic Scan #1 (ppb) Reference (ppb) CAA-25 1 3 <1 1 6 1 0.91 CAA-25 2 3 <1 <1 6 1 0.86 CAA-25 3 3 <1 <1 6 1 0.90 PT - 1 ppb As CAA-25 4 3 1 <1 6 1 0.86 CAA-27 1 6 2 1 6 2 2.56 CAA-27 2 6 2 1 6 2 2.64 CAA-27 3 6 3 1 6 2 2.50 PT - 3 ppb As CAA-27 4 6 3 2 6 3 2.71 CAA-1 1 10 9.7 8 15 8 9.09 CAA-1 2 10 10.4 9.7 12 5.6 8.95 CAA-1 3 10 11.2 9.7 19 12.8 8.83 PT - 10 ppb As CAA-1 4 10 8 7.3 12 5.6 8.99 CAA-29 1 30 (a) 24.4 (a) 35.2 (a) 24 22.4 33.96 CAA-29 2 30 (a) 35.2 (a) 28.8 (a) 24 24.1 34.39 CAA-29 3 30 (a) 29.4 (a) 33.6 (a) 24 24.9 34.51 PT - 30 ppb As CAA-29 4 30 (a) 27.2 (a) 28.8 (a) 24 26.6 34.98 CAA-31 1 95 (b) 91.5 (b) 96 (b) 120 (b) 96 (b) 111.89 CAA-31 2 95 (b) 104 (b) 91.5 (b) 120 (b) 76. 5 (b) 115.57 CAA-31 3 100 (b) 96 (b) 112 (b) 95 (b) 88 (b) 114.65 PT - 100 ppb As CAA-31 4 100 (b) 99.5 (b) 96 (b) 95 (b) 52 (b) 113.83 CAA-24 1 12 9.70 9.70 8 4.8 CAA-24 2 13 12.80 12.8 8 3 CAA-24 3 13 10.40 11.2 8 4.8 CAA-24 4 12 11.20 12.20 8 7.3 CAA-24 5 13 10.40 11.2 12 9 CAA-24 6 13 12.80 13.6 12 9 Detection limit CAA-24 7 15 12.20 12.8 12 7.3 14.18 24 Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results (continued) Description Sample ID Replicate Technical Operator Color Chart (ppb) Technical Operator Quick™ Arsenic Scan #1 (ppb) Technical Operator Quick™ Arsenic Scan #2 (ppb) Non-Technical Operator Color Chart (ppb) Non-Technical Operator Quick™ Arsenic Scan #1 (ppb) Reference (ppb) CAA-33 1 13 10.4 10.4 3 1 CAA-33 2 11 9 9 8 9.7 CAA-33 3 11 9 9.7 10 4.8 PT - 10 ppb As + low level interferents CAA-33 4 11 10.4 11.2 10 4.8 9.90 CAA-35 1 24 19.2 27.4 24 17.6 CAA-35 2 22 20.8 22.4 24 19.2 CAA-35 3 24 19.9 24.9 24 22.4 PT - 10 ppb As + high level interferents CAA-35 4 22 17.6 17.6 24 17.6 11.59 CAA-37 1 <3 <1 <1 <3 <1 <0.5 CAA-37 2 <3 <1 <1 <3 <1 <0.5 CAA-37 3 <3 <1 <1 <3 <1 <0.5 Battelle drinking water CAA-37 4 <3 <1 <1 <3 <1 <0.5 Battelle drinking water LFM CAA-38 1 12 7.3 8 8 4 11.96 CAA-39 1 40 (b) 10 (b) 15 (b) 80 (c) 10 (c) 65.61 CAA-39 2 45 (b) 20 (b) 28 (b) 80 (c) 10 (c) 62.73 CAA-39 3 40 (b) 10 (b) 15 (b) 80 (c) 10 (c) 67.47 Ayer untreated water CAA-39 4 35 (b) <1 (b) <1 (b) 80 (c) 10 (c) 63.48 Ayer untreated water LFM CAA-40 1 50 5 10 80 10 69.74 CAA-41 1 <3 <1 <1 <3 <1 1.36 CAA-41 2 <3 <1 <1 <3 <1 1.45 CAA-41 3 <3 <1 <1 <3 <1 1.44 Ayer treated water CAA-41 4 <3 <1 <1 <3 <1 1.32 Ayer treated water LFM CAA-42 1 10 4 4 8 2 13.02 25 Table 6-4. Quick™ Low Range Test Kit and Reference Sample Results (continued) Description Sample ID Replicate Technical Operator Color Chart (ppb) Technical Operator Quick™ Arsenic Scan #1 (ppb) Technical Operator Quick™ Arsenic Scan #2 (ppb) Non-Technical Operator Color Chart (ppb) Non-Technical Operator Quick™ Arsenic Scan #1 (ppb) Reference (ppb) CAA-43 1 <3 <1 <1 <3 <1 <0.5 CAA-43 2 <3 <1 <1 <3 1 <0.5 CAA-43 3 <3 <1 <1 <3 7.3 <0.5 Falmouth Pond water CAA-43 4 <3 1 <1 <3 1 <0.5 Falmouth Pond water LFM CAA-46 1 11 9.7 10.4 2.2 16 11.50 CAA-47 1 <3 1 1 <3 1 1.36 CAA-47 2 <3 <1 2 <3 1 1.31 CAA-47 3 <3 4 4.8 <3 1 1.31 Taunton River water CAA-47 4 <3 <1 <1 <3 <1 1.26 Taunton River water LFM CAA-48 1 12 10.4 9.7 10 10.4 12.26 Note: (a) = diluted 1:2; (b) = diluted 1:5; (c) = diluted 1:10. 26 Table 6-5. Quantitative Evaluation of Accuracy for Quick™ Low Range Test Kits (a) Bias Description Technical Operator Color Chart Technical Operator Quick™ Arsenic Scan #1 Technical Operator Quick™ Arsenic Scan #2 Non-Technical Operator Color Chart Non-Technical Operator Quick™ Arsenic Scan #1 Performance Test Samples 1 ppb As 239% NA NA 579% 13.1% 3 ppb As 131% -3.9% -51.9% 131% -13.5% 10 ppb As 11.6% 9.6% -3.2% 61.8% -10.7% 30 ppb As -12.9% -15.7% -8.3% -30.4% -28.9% 100 ppb As -14.5% -14.2% -13.3% -5.7% -31.5% 10 ppb As + low level interferents 16.2% -2.0% 1.8% -21.7% -48.7% 10 ppb As + high level interferents 98.4% 67.1% 99.1% 107% 65.6% Environmental Samples Battelle drinking water NA NA NA NA NA Battelle drinking water LFM 0.31% -39.0% -33.1% -33.1% -66.6% Ayer untreated water -38.3% NA NA 23.4% -84.6% Ayer untreated water LFM -28.3% -92.8% -85.7% 14.7% -85.7% Ayer treated water NA NA NA NA NA Ayer treated water LFM -23.2% -69.3% -69.3% -38.5% -84.6% Falmouth Pond water NA NA NA NA NA Falmouth Pond water LFM -4.4% -15.7% -9.6% -80.9% 39.1% Taunton River water NA NA NA NA NA Taunton River water LFM -2.1% -15.2% -20.9% -18.5% -15.2% (a) Percent bias calculated according to Equation 4, Section 5.1. NA: one or more replicates below detection limit 27 ranged from -93% to 99% for the technical operator and -86% to 66% for the non-technical operator. Negative biases for the Ayer untreated water sample (with the exception of the color chart result for the non-technical operator) confirm the apparent matrix effect observed in the associated LFM sample (see Section 6.1). Table 6-6 presents accuracy results for each PT and environmental replicate sample according to whether the color chart result agreed with the reference value for that sample. The reference sample result was assigned to the correct corresponding color block. A test kit result was considered to be in agreement with the reference method result if it fell within the range of plus or minus one color block (i.e., the concentration range spanning three adjacent color blocks). If the color chart test result for a given sample was within this range, then a “Y” was reported in Table 6-6. If the color chart result was outside this range, then an “N” was reported. Overall agreement was determined by calculating the total percent of results in agreement for the technical and non-technical operators. The total percent agreement using this method was 81% for the technical operator and 74% for the non-technical operator. 6.2.2 Precision Precision results for the Quick™ Low Range test kit are presented in Table 6-7. The RSD was determined according to Equation 7 (Section 5.2). The RSD was not calculated if any of the results for a set of replicates were below the detection limit (i.e., <3 ppb for the color chart or <1 ppb for the Quick™ Arsenic Scan). For the technical operator, RSDs ranged from 0% to 10% for the color chart and 5% to 23% for the Quick™ Arsenic Scan. For the non-technical operator, RSDs ranged from 0% to 23% for the color chart and 0% to 42% for the Quick™ Arsenic Scan. 6.2.3 Linearity The linearity of the Quick™ Low Range test kit readings was assessed by performing a linear regression of the test kit results against the reference method results for the five PT samples ranging from 1 ppb to 100 ppb arsenic. In these regressions, results reported as below the detection limit by the test kit were not used. Figures 6-1 and 6-2 present the results of the linear regressions for the color chart and Quick™ Arsenic Scan results, respectively. The slope, intercept and correlation coefficient for each equation are shown on the charts. For the color chart, the results for the technical operator were more linear than the results for the non-technical operator; however, the non-technical operator results showed an overall closer correspondence with the reference method results. For the Quick™ Arsenic Scan, the results for the technical operator using Unit #1 showed the greatest degree of linearity. The technical operator results for Unit #2 showed the closest overall correspondence with the reference method results. 28 Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits Description Sample ID Replicate Within Range (Y/N) Technical Operator Color Chart Within Range (Y/N) Non-Technical Operator Color Chart Performance Test Samples CAA-25 1 Y N CAA-25 2 Y N CAA-25 3 Y N 1 ppb As CAA-25 4 Y N CAA-27 1 Y Y CAA-27 2 Y Y CAA-27 3 Y Y 3 ppb As CAA-27 4 Y Y CAA-1 1 Y N CAA-1 2 Y Y CAA-1 3 Y N 10 ppb As CAA-1 4 Y Y CAA-29 1 Y Y CAA-29 2 Y Y CAA-29 3 Y Y 30 ppb As CAA-29 4 Y Y CAA-31 1 Y Y CAA-31 2 Y Y CAA-31 3 Y Y 100 ppb As CAA-31 4 Y Y CAA-33 1 N N CAA-33 2 Y Y CAA-33 3 Y Y 10 ppb As + low level interferents CAA-33 4 Y Y CAA-35 1 N N CAA-35 2 N N CAA-35 3 N N 10 ppb As + high level interferents CAA-35 4 N N Environmental Samples CAA-37 1 Y Y CAA-37 2 Y Y CAA-37 3 Y Y Battelle drinking water CAA-37 4 Y Y Battelle drinking water LFM CAA-38 1 Y N CAA-39 1 N Y CAA-39 2 N Y CAA-39 3 N Y Ayer untreated water CAA-39 4 N Y Ayer untreated water LFM CAA-40 1 N Y 29 Table 6-6. Qualitative Evaluation of Agreement for Quick™ Low Range Test Kits (continued) Description Sample ID Replicate Within Range (Y/N) Technical Operator Color Chart Within Range (Y/N) Non-Technical Operator Color Chart CAA-41 1 Y Y CAA-41 2 Y Y CAA-41 3 Y Y Ayer treated water CAA-41 4 Y Y Ayer treated water LFM CAA-42 1 Y N CAA-43 1 Y Y CAA-43 2 Y Y CAA-43 3 Y Y Falmouth Pond water CAA-43 4 Y Y Falmouth Pond water LFM CAA-46 1 Y N CAA-47 1 Y Y CAA-47 2 Y Y CAA-47 3 Y Y Taunton River water CAA-47 4 Y Y Taunton River water LFM CAA-48 1 Y Y Percent Agreement(a) 81% 74% (a) Percent of sample “Y”, within range, divided by total number of samples. 30 Table 6-7. Precision Results for Quick™ Low Range Test Kits RSD Description Technical Operator Color Chart Technical Operator Quick™ Arsenic Scan #1 Technical Operator Quick™ Arsenic Scan #2 Non-Technical Operator Color Chart Non-Technical Operator Quick™ Arsenic Scan #1 Reference Method Performance Test Samples 1 ppb As 0% NA NA 0% 0% 3% 3 ppb As 0% 23% 40% 0% 22% 4% 10 ppb As 0% 14% 14% 23% 42% 1% 30 ppb As 0% 16% 10% 0% 7% 1% 100 ppb As 3% 5% 9% 13% 25% 1% Environmental Samples Battelle drinking water NA NA NA NA NA NA Ayer untreated water 10% NA NA 0% 0% 3% Ayer treated water NA NA NA NA NA 4% Falmouth Pond water NA NA NA NA NA NA Taunton River water NA NA NA NA 0% 3% NA: one or more replicates below detection limit y = 0.83x + 2.61R = 0.9992y = 0.90x + 2.78R = 0.98050204060801001201400 20 40 60 80 100 120 140Reference concentration (ppb)Color chart concentration (ppb)Technical OperatorNon-technical OperatorLinear (Technical Operator)Linear (Non-technical Operator)1:1 line Figure 6-1. Linearity of Quick™ Low Range Color Chart Results 31 y = 0.85x + 0.83R = 0.9972y = 0.87x + 0.41R = 0.9939 y = 0.68x + 0.99R = 0.96600204060801001200 20 40 60 80 100 120 140Reference concentration (ppb)Quick Arsenic Scan concentration (ppb)Technical Operator Unit #1Technical Operator Unit #2Non-techncial operator Unit #1Linear (Technical Operator Unit #1)Linear (Technical Operator Unit #2)Linear (Non-techncial operator Unit #1)1:1 line Figure 6-2. Linearity of Quick™ Low Range Quick™ Arsenic Scan Results 6.2.4 Method Detection Limit The MDL was assessed by analyzing seven replicates of a sample spiked at approximately five times the lowest concentration shown on the Quick™ Low Range test kit color chart (i.e., 3 ppb X 5 = 15 ppb arsenic). Table 6-8 provides the standard deviation for the seven replicate samples for the technical and non-technical operator for both the color chart and Quick ™ Arsenic Scan results, and the calculated MDLs. 32 Table 6-8. Detection Limit Results for Quick™ Low Range Test Kit Sample ID Technical Operator Color Chart (ppb) Technical Operator Quick™ Arsenic Scan #1 (ppb) Technical Operator Quick™ Arsenic Scan #2 (ppb) Non-Technical Operator Color Chart (ppb) Non-Technical Operator Quick™ Arsenic Scan #1 (ppb) CAA-24 Rep 1 12 9.70 9.70 8 4.8 CAA-24 Rep 2 13 12.80 12.8 8 3 CAA-24 Rep 3 13 10.40 11.2 8 4.8 CAA-24 Rep 4 12 11.20 12.20 8 7.3 CAA-24 Rep 5 13 10.40 11.2 12 9 CAA-24 Rep 6 13 12.80 13.6 12 9 CAA-24 Rep 7 15 12.20 12.8 12 7.3 Standard Deviation 1.00 1.26 1.32 2.14 2.30 Method Detection Limit (ppb) 3.1 4.0 4.1 6.7 7.2 6.2.5 Matrix Interference Effects Matrix interference effects were assessed by comparing the calculated bias for the samples containing low-level and high-level concentrations of interferences with the bias reported for the other PT samples containing arsenic only (Table 6-5). An examination of these results indicates that low levels of interfering compounds did not appear to affect the detection of arsenic, with biases ranging from -48.7% to 16.2% for both the technical and non-technical operators, and color chart and Quick™ Arsenic Scan methods. However, high levels of interferences appear to have affected the arsenic levels measured by the Quick™ Low Range test kit for both the color chart and Quick™ Arsenic Scan results, as evidenced by positive biases ranging from 65.6% to 107%. 6.2.6 Operator Bias Operator bias was evaluated by comparing the color chart and Quick™ Arsenic Scan Unit #1 results above the detection limit produced by the technical and non-technical operators for all PT and environmental samples (the non-technical operator did not use the Quick™ Arsenic Scan Unit #2). Linear regression results for the two sets of data are shown in Figure 6-3. The slopes of the regressions made from measurements by the non-technical operator suggest that the color chart results tended to be higher, and the Quick™ Arsenic Scan results made by the technical operator tended to be higher. Paired t-tests of the two sets of data indicated that the color chart results were not significantly different at a 5% significance level; however, the Quick™ Arsenic Scan results were significantly different. 33 y = 1.20x - 1.81R = 0.94y = 0.79x + 0.26R = 0.950204060801001201400 20 40 60 80 100 120Technical OperatorNon-technical OperatorColor ChartArsenic Quick Scan Unit #1Linear (Color Chart)Linear (Arsenic Quick Scan Unit #1)1:1 line Figure 6-3. Comparison of Quick™ Low Range Test Results for Technical and Non-Technical Operators 6.2.7 Inter-Unit Reproducibility Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan units used by the technical operator. Only results above the detection limit were included in the analysis. A linear regression of the two sets of data is shown in Figure 6-4. The regression line almost exactly corresponded to the 1:1 line, indicating that the performance of the two units was very similar. A paired t-test of the two sets of data indicated that the results were not significantly different at a 5% significance level. 6.2.8 Rate of False Positives/False Negatives Tables 6-9 and 6-10 show the data and results for the rates of false positives and false negatives, respectively, obtained from the Quick™ Low Range test kit. All PT and environmental samples were included in this evaluation. 34 y = 1.00x + 1.17R = 0.98630204060801001200 20 40 60 80 100 120Unit #1Unit #2 Quick Arsenic ScanLinear (Quick Arsenic Scan)1:1 line Figure 6-4. Comparison of Two Quick™ Arsenic Scan Units As shown in Table 6-9, 32 samples had arsenic concentrations at or below 10 ppb as measured by the reference analyses. For these samples, the Quick™ Low Range test kit color chart results were >10 ppb and greater than 125% of the reference measurement for one sample for the technical operator and four samples for the non-technical operator, yielding false positive rates of 3% and 12.5% respectively. The rates of false positives for the Quick™ Arsenic Scan units were 3% and 0% for the technical operator (Units #1 and #2, respectively) and 3% for the non-technical operator (Unit #1 only). Twenty-two samples had arsenic concentrations above 10 ppb as measured by the reference analyses (Table 6-10). For these samples, the Quick™ Low Range test kit color chart results were <10 ppb and less than 75% of the reference measurement for none of the samples for the technical operator and three samples for the technical operator, yielding false negative rates of 0% and 14%, respectively. The rates of false negatives for the Quick™ Arsenic Scan units were 19% and 14% for the technical operator (Units #1 and #2, respectively), and 9.5% for the non-technical operator (Unit #1 only). 35 Table 6-9. Rate of False Positives for Quick™ Low Range Test Kits False Positive (Y/N) Description Sample ID Replicate Technical Operator Color Chart Technical Operator Quick™ Arsenic Scan #1 Technical Operator Quick™ Arsenic Scan #2 Non-Technical Operator Color Chart Non-Technical Operator Quick™ Arsenic Scan #1 CAA-25 1 N N N N N CAA-25 2 N N N N N CAA-25 3 N N N N N 1 ppb As CAA-25 4 N N N N N CAA-27 1 N N N N N CAA-27 2 N N N N N CAA-27 3 N N N N N 3 ppb As CAA-27 4 N N N N N CAA-1 1 N N N Y N CAA-1 2 N N N Y N CAA-1 3 N Y N Y Y 10 ppb As CAA-1 4 N N N Y N CAA-37 1 N N N N N CAA-37 2 N N N N N CAA-37 3 N N N N N Battelle drinking water CAA-37 4 N N N N N CAA-41 1 N N N N N CAA-41 2 N N N N N CAA-41 3 N N N N N Ayer treated water CAA-41 4 N N N N N CAA-43 1 N N N N N CAA-43 2 N N N N N CAA-43 3 N N N N N Falmouth Pond water CAA-43 4 N N N N N CAA-47 1 N N N N N CAA-47 2 N N N N N CAA-47 3 N N N N N Taunton River water CAA-47 4 N N N N N CAA-33 1 Y N N N N CAA-33 2 N N N N N CAA-33 3 N N N N N 10 ppb As + low level interferents CAA-33 4 N N N N N Total number of samples 32 32 32 32 32 Total false positive 1 1 0 4 1 Percent false positive 3% 3% 0% 12.5% 3% 36 Table 6-10. Rate of False Negatives for Quick™ Low Range Test Kits False Negative (Y/N) Description Sample ID Replicate Technical Operator Color Chart Technical Operator Quick™ Arsenic Scan #1 Technical Operator Quick™ Arsenic Scan #2 Non-Technical Operator Color Chart Non-Technical Operator Quick™ Arsenic Scan #1 CAA-29 1 N N N N N CAA-29 2 N N N N N CAA-29 3 N N N N N 30 ppb As CAA-29 4 N N N N N CAA-31 1 N N N N N CAA-31 2 N N N N N CAA-31 3 N N N N N 100 ppb As CAA-31 4 N N N N N Battelle drinking water LFM CAA-38 1 N Y Y Y Y CAA-39 1 N N N N N CAA-39 2 N N N N N CAA-39 3 N N N N N Ayer untreated water CAA-39 4 N Y Y N N Ayer untreated water LFM CAA-40 1 N Y N N N Ayer treated water LFM CAA-42 1 N Y Y Y Y Falmouth Pond water LFM CAA-46 1 N N N Y N Taunton River water LFM CAA-48 1 N N N N N CAA-35 1 N N N N N CAA-35 2 N N N N N CAA-35 3 N N N N N 10 ppb As + high level interferents CAA-35 4 N N N N N Total number of samples 21 21 21 21 21 Total number of false negatives 0 4 3 3 2 Percent of false negatives 0% 19% 14% 14% 9.5% 6.3 Other Factors During testing activities, the technical and non-technical operators were instructed to keep a record of their comments on ease of use, reliability, portability, and generation of waste materials. This section summarizes these observations and other comments pertaining to any problems encountered during testing. Cost information is also presented. 37 6.3.1 Ease of Use The technical and non-technical operator both reported that the Quick™ Low Range test kit was very easy to use. The test kit instructions were clear and easy to follow. Although the manu-facturer provided instructions for diluting samples above the 30 ppb arsenic level, the non-technical operator sometimes had difficulty successfully performing dilutions and correctly converting the results to a final concentration. The three scoops used to sequentially add reagents were color coordinated, which facilitated the efficient operation of the test kit. The sample bottles were of moderate size and were relatively easy to handle, although the narrow neck sometimes caused spillage during the addition of reagents. The test kit materials were readily transported to the Battelle storage shed where environmental samples were tested. 6.3.2 Analysis Time The average total analysis time for a sample was about 15 minutes at a sample temperature of 24°C. The manufacturer provided a modified protocol that specified increased reaction times for samples below 24°C. The test kit enabled two samples to be run concurrently without any confusion. 6.3.3 Reliability The Quick™ Low Range test kits operated reliably throughout the period of the test. Extra care had to be taken to ensure that the caps to the reaction vessels were completely dry before proceeding with further analyses, and in some cases the presence of water droplets on the test pad prevented an accurate reading and required the reanalysis of the sample. After the analysis of some of the environmental surface water samples, black particulate matter was noted on the test pad, presumably because of the presence of organic material in the water sample. 6.3.4 Waste Material The waste generated by the Quick™ Low Range test kit was manageable. The vendor’s instructions provide a warning that hydrogen and arsine are generated during the test and recommend that testing be conducted in a well-ventilated area away from open flames and other sources. MSDSs should be reviewed before handling any chemicals. Instructions for the disposal of residual materials were clear and complete. The residual liquid in the reaction vessel was allowed to settle before disposal in order to let particulates accumulate on the bottom. A dilute hydrochloric acid solution was used to clean the reaction vessel prior to subsequent analyses. Disposal of this waste in an appropriate manner must be taken into consideration. 6.3.5 Cost The listed price for a Quick™ Low Range test kit for the analysis of 50 samples is $179.99. The Quick™ Arsenic Scan is available as an option for an additional cost of $1,599.99. 38 Chapter 7 Performance Summary The Quick™ Low Range test kit was verified by evaluating the following parameters: § Accuracy § Precision § Linearity § MDL § Matrix interference effects § Operator bias § Inter-unit reproducibility § Rate of false positives/false negatives. The quantitative assessment of accuracy indicated that the relative bias for the color chart ranged from -38% to 239% for the technical operator and -81% to 579% for the non-technical operator. The relative bias for the Quick™ Arsenic Scan ranged from -93% to 99% for the technical operator and -86% to 66% for the non-technical operator. The overall agreement for the color chart results based on an assessment of whether the result was assigned to the correct color block indicated that the total percent agreement was 81% for the technical operator and 74% for the non-technical operator. Precision was assessed by analyzing four replicates of each sample. For the technical operator, precision expressed as a RSD ranged from 0% to 10% for the color chart and 5% to 23% for the Quick™ Arsenic Scan. For the non-technical operator, RSDs ranged from 0% to 23% for the color chart and 0% to 42% for the Quick™ Arsenic Scan. These results exclude samples where one or more of the replicate results was not detected by the Quick™ Low Range test kit. The linearity of response was evaluated by plotting the test kit results against the reference analysis results for the PT samples. The equations for the linear regressions that were performed to evaluate linearity are summarized in Table 7-1. The slope, y-intercept, and correlation coefficient corresponding to a linear response that exactly matched reference concentrations would be 1.0, 0, and 1.0, respectively. The MDL was assessed by analyzing seven replicates of a sample spiked at a level approximately five times the manufacturer’s estimated detection limit for the color chart (i.e., 3 ppb X 5 = 15 ppb). The MDLs calculated using the precision data from these replicates ranged from 3.1 ppb to 6.7 ppb for the color charts and 4.0 ppb to 7.2 ppb for the Quick™ Arsenic Scan. 39 Table 7-1. Summary of Linear Regression Equations for Test Kit and Reference Results Description Slope Intercept Correlation Coefficient (R) Color chart, technical operator 0.83 2.61 0.9992 Color chart, non-technical operator 0.90 2.78 0.9805 Quick™ Arsenic Scan #1, technical operator 0.85 0.83 0.9972 Quick™ Arsenic Scan #2, technical operator 0.87 0.41 0.9939 Quick™ Arsenic Scan #1, non-technical operator 0.68 0.99 0.9660 Results for samples containing low and high levels of interfering compounds indicated that low levels of interferents did not appear to affect the detection of arsenic, with biases ranging from -48.7% to 16.2%. However, high levels of interferences appear to have affected the arsenic levels measured by the Quick™ Low Range test kit for the color chart and Quick™ Arsenic Scan results, as evidenced by positive biases ranging from 65.6% to 107%. An evaluation of Quick™ Low Range test kit results for the technical and non-technical operators suggested that the color chart measurements made by the non-technical operator tended to be higher and the Quick™ Arsenic Scan measurements made by the technical operator tended to be higher. Paired t-tests of the two sets of data indicated that the color chart results were not significantly different at a 5% significance level; however, the Quick™ Arsenic Scan results were significantly different for the two operators. The regression equations were as follows: Color chart y = 1.20x – 1.81, R = 0.94 Quick™ Arsenic Scan y = 0.79x + 0.26, R = 0.95 where x is the technical operator and y is the non-technical operator. Inter-unit reproducibility was evaluated by comparing the data for the two Quick™ Arsenic Scan units used by the technical operator. A linear regression of the two sets of data indicated that the results closely corresponded. A paired t-test of the two sets of data indicated that the results were not significantly different at a 5% significance level. The regression equation was as follows: Quick™ Arsenic Scan y = 1.00x + 1.17, R = 0.99 where x is Unit #1 and y is Unit #2. A false positive was defined as a test kit result that was greater than 10 ppb and greater than 125% of the reference concentration, when the reference concentration is less than or equal to 10 ppb. The rates of false positives for the technical and non-technical operators using the color charts were 3% and 12.5%, respectively. The rates of false positives for the Quick™ Arsenic Scan units were 3% and 0% for the technical operator (Units #1 and #2, respectively) and 3% for the non-technical operator (Unit #1 only). A false negative was defined as a test kit result that was less than or equal to 10 ppb and less than 75% of the reference concentration, when the 40 reference concentration was greater than 10 ppb. The false negative rates for the technical and non-technical operators using the color charts were 0% and 14%, respectively. The rates of false negatives for the Quick™ Arsenic Scan units were 19% and 14% for the technical operator (Units #1 and #2, respectively) and 9.5% for the non-technical operator (Unit #1 only). The Quick™ Low Range test kits were easy to use and readily transportable to the field. The time to analyze one sample was approximately 15 minutes at a temperature range of 24°C to 30°C (longer reaction times are required for samples below this temperature range). Two samples were run concurrently without difficulty. The sample bottles were of moderate size and were relatively easy to handle, although the narrow neck sometimes caused spillage during the addition of reagents. The cost for a 50-sample test kit with the color chart is listed as $179.99. Replacement reagents and supplies are not available; kits are provided as a complete set because reagents, test strips, and color charts are made to perform optimally with each other, according to the vendor. The Quick™ Arsenic Scan is available as an option for an additional cost of $1,599.99. 41 Chapter 8 References 1. Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2.0. December 8, 2000. 2. U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma Mass Spectrometry, Revision 5.5, October 1999. 3. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot, Version 4.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio, December, 2002. 4. U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B.