Users of national network data can also use BSP data as a quantitative tool for assessing the measurement bias and variability of selected laboratory methods and stream water-quality data during more recent periods of operation of the national networks. The CD- ROMs include BSP data for 34 chemical constituents analyzed at the USGS NWQL during the water years 1985 to 1995.
Estimates of the variability of laboratory analytical measurements provide quantitative information about the extent to which laboratory methods contribute to random variations in measurements of ambient stream water quality. The measurement variability indicates how well measurements can be reproduced with repeated applications of an analytical method. Random errors in measurements are intrinsic to the measurement process. These errors are by definition independent of one another, and are not reproducable over time. One important concern for a data user is whether random variations in measurements related to methods are large enough to prevent the detection of variations related to chemical processes in the environment. BSP estimates of measurement variability, determined through repeated measurements of each reference sample, are available for a wide range of concentrations, and can be used to examine this issue.
Estimates of the bias of laboratory analytical measurements provide quantitative information about the extent to which laboratory methods contribute to systematic variations in measurements of ambient stream water quality. In contrast to random measurement errors, systematic errors (i.e., measurement bias) are reproducable over time. Repeated applications of a biased analytical method would be expected to produce correlated measurement errors of a similar sign and magnitude. Such errors may be caused, for example, by sample contamination or matrix interferences. BSP data can be used to identify time periods during which laboratory analytical methods may have produced measurements with a positive or negative bias. A sufficiently large bias might lead a data user to disregard data processed at the laboratory during certain time periods or might lead a user to become overly cautious about the interpretation of environmental records that might potentially contain such errors. A data user may also question whether trends in environmental concentrations result from or are masked by changes in the measurement bias of a laboratory analytical method. BSP estimates of measurement bias can also be used to evaluate the possible effects of measurement bias on observed trends in water-quality data.
The following two sub-sections describe the characteristics of BSP samples and how estimates of the laboratory measurement bias and variability are obtained using BSP samples. A final sub-section describes the BSP data selected for publication on the CD-ROMs. The next major section of the report discusses various ways of using BSP data in the interpretation of national network water-quality data. Throughout this discussion, use of the term "environmental" data refers to stream water-quality data collected at national network stations.
Standard reference water samples (SRWS) are used in the preparation of BSP samples (Skougstad and Fishman, 1975; Schroder and others, 1980; Janzer, 1985; Long and Farrar, 1992). SRWS are obtained from natural waters, which principally include Colorado streams. In preparing BSP samples, SRWS are used directly or are diluted with deionized water or mixed in varying proportions with other SRWS. This sample-mixing procedure produces a larger number of unique quality-control samples than would be available from the SRWS program alone.
SRWS are filtered during preparation; therefore, all constituents in the BSP are in the dissolved phase. Filtration removes most particulate material in the samples, and thus avoids subsampling problems that could produce non-representative results. Whole-water recoverable (WWR) methods are applied to the filtered reference samples in the SRWS program for selected constituents (Fishman and Friedman, 1989); however, any differences between dissolved analyses and WWR analyses are due to the digestion process associated with the WWR method rather than differences in sample source or sampling techniques.
BSP reference samples are designed to appear as much like environmental samples as possible to reduce the possibility that analysts will identify the quality-assurance samples. As with regular environmental samples, analytical request forms are completed to ensure that the user-selected analytical tests are performed on the samples.
The number of analytical determinations requested for each laboratory analytical procedure is proportional to the number of requests for the procedure from all environmental samples submitted. The BSP follows the guidelines of Friedman and Erdmann (1982) by setting the rate of submissions of external quality- assurance samples to approximately five percent of the laboratory workload for each analytical procedure. The annual workload for each analytical procedure is estimated from sample login records for the previous year. The number of BSP samples analyzed each year varies, but is approximately 200, 300, and 350 for trace elements, nutrients, and major ions, respectively.
The concentration of each SRWS is estimated semi-annually through a round-robin evaluation program involving upwards of 150 laboratories [see Long and Farrar (1992) for a more detailed description]. A statistical summary of the round-robin results is prepared for each set of samples (see for example, Janzer, 1983), and includes estimates of the known concentration of each SRWS and estimates of the inter-laboratory variability in SRWS measurements. Until 1988, the SRWS concentration and inter-laboratory variability in measurements were estimated using parametric statistics of the mean and standard deviation. After 1988, nonparametric statistics of the median and F-pseudosigma (Hoaglin and others, 1983) were used. The F-pseudosigma gives an unbiased estimate of the standard deviation when the data are normally distributed. The estimate of the SRWS concentration according to the mean or median of all laboratory measurements is reported as the "most probable value" (MPV) for each constituent in SRWS summary reports. The F- pseudosigma (or standard deviation prior to 1988) is referred to as the "minimum probable deviation" (MPD) in the estimate of the known reference sample concentration.
In the Blind Sample Program, estimates of the mean or median concentration of mixtures of SRWS or mixtures of SRWS and deionized water are based on the proportion and the most probable values of the SRWS used. When deionized water is used to dilute the SRWS, a sample concentration of zero is applied to estimate the most probable value (MPV) for the proportion used.
The MPD is estimated for each SRWS chemical constituent over a continuous range of concentrations by regressing the most probable value on the MPD using the method of ordinary least squares [see Maloney and others (1994) for additional details]. SRWS program data for semi-annual round-robin studies from the previous seven years were used to derive equations for each constituent. The regression equations were updated and applied to each water year, taking into account the most recently released SRWS data.
Estimates of measurement accuracy
The laboratory analytical measurement error (E) is estimated
as the difference between a laboratory measurement of a BSP
reference sample (M) and the known concentration (MPV) of the
sample, and contains both systematic and random sources of error.
The error may be written as
E = M - MPV (1)
Estimates of analytical measurement error provide a quantitative measure of the accuracy (bias and variability) of a laboratory method.
USGS laboratory procedures for reporting analytical measurements specify rounding rules that can affect the accuracy of BSP estimates of measurement error. All water-quality measurements reported by the laboratory are rounded to the nearest whole number or decimal fraction according to the sensitivity of analytical methods as specified by the method reporting limit. As a result, the accuracy of individual estimates of error would be expected to decline as the magnitude of the error becomes smaller (i.e., as the magnitude of the error decreases relative to the size of the reporting limit). The highest levels of inaccuracy occur for error estimates that are computed to be less than the method reporting limit. For these cases, the uncertainty in water-quality measurements exceeds the magnitude of the error. Error values that are less than the method reporting limt are flagged in the CD-ROM data base. In such cases, the sign of the error value is not known with certainty, however, we report the sign resulting from differences in concentration computed according to equation 1. See the section "Statistical analysis of measurement error estimates" (p.55) for additional discussion of the effects of rounding on error estimates.
An assessment of the accuracy of analytical methods can be conducted by comparing observed error estimates with the specified limits of acceptable variability in measurement error. The Blind Sample Program defines the acceptable limits as a multiple of the inter-laboratory variability (i.e., the MPD) of the reference sample concentration. Performance of an analytical method is judged as "out-of-control" in cases where errors exceed this threshold. For purposes of assessing measurement accuracy, the BSP expresses the measurement error from equation 1 in standardized units of variability (i.e., mean of zero and standard deviation of one) called a z-value or NSD (number of standard deviations). The NSD is estimated as
NSD = E / MPD (2)
The Blind Sample Program considers values of NSD within +/- 2 of the MPV of acceptable quality (Maloney and others, 1994). For purposes of internal laboratory quality-assurance, the NWQL specifies control limits based on values of NSD within +/- 1.5 of the MPV (Pritt and Raese, 1992).
For reference samples with very low known concentrations, located close to the method reporting limits, the reported NSD may be unrealistically large. This situation can arise because SRWS values of the most probable value are frequently reported with greater numerical precision than measurements reported by the NWQL. Thus, the regression estimates of the MPD may reflect higher precision than the NWQL analytical methods can achieve. For example, the NWQL reports values for many trace metals (based on AA analytical methods) to the nearest 10 mg/L for determinations below 100 mg/L. In contrast, the SRWS program reports values to the nearest 1 mg/L. Therefore, estimates of the MPD at very low reference sample concentrations may produce tolerance limits that are too small to be attained by the NWQL.
For constituents where laboratory rounding rules would produce a minimum NSD above one, Blind Sample Program procedures set the minimum MPD at 75 percent of the laboratory reporting limit, thereby ensuring that the minimum NSD falls within +/- 1 of the MPV. For example, manganese is reported by the NWQL to the nearest 10 mg/L. Therefore, the most accurate measurement value for a reference sample with an MPV of 25.2 mg/L would be a value of 20 or 30 mg/L. According to available regression estimates of precision, the closest reportable values of 20 and 30 mg/L would be -1.33 and 1.19 NSD units from the MPV, respectively (measurement values within +/- 1.0 NSD units of the MPV would correspond to 21.1 and 29.1 mg/L). When the minimum MPD is set to 75 percent of the laboratory reporting limit (or 7.5 mg/L), the closest reportable values of 20 and 30 mg/L would be -0.73 and 0.60 NSD units from the MPV, respectively, and thus fall within +/- 1 of the MPV.
In an extremely few cases, the laboratory has reported measurements of reference samples as "less than" the method reporting limit. This has only occurred infrequently for selected BSP samples with very low chemical concentrations. Prior to 1991, these less-than values were replaced by substituting the value of the method reporting limit. After this year, these determinations were not included as part of the BSP data base.
Estimates of measurement bias
BSP estimates of measurement error (equation 1) reflect both
systematic and random sources of error. Systematic error, or
measurement bias, can be isolated and quantified separately from
random error by computing the mean of multiple error estimates.
Measurement bias (B) is defined as
B = (1/n) [summation(i=1,n)] Ei (3)
where i is the ith value of laboratory measurement error for a BSP reference sample, and n is the total number of values of laboratory measurement error for a specified time period. The bias, therefore, describes a static, persistent measurement error associated with an analytical method for a specified length of time. The next major section of the report entitled "Relation of BSP Data to National Network Water-Quality Data" discusses graphical techniques for identifying bias, methods for estimating changes in bias over time, and methods for testing whether estimated bias is statistically different from zero.
Estimates of measurement variability
Estimation of random variability in analytical measurement error relies on the use of replicate measurements of a water sample. By computing the difference between replicate measurements of identical water samples spaced closely in time, systematic errors cancel, and random errors can be isolated and quantified.
In the BSP, measurement variability is estimated from replicate determinations on reference samples prepared from identical mixes of SRWS. Each BSP sample mix is analyzed repeatedly in the laboratory for approximately one year with the exception of nutrient mixes that are analyzed over about a six month period. For each unique BSP mixture and MPV concentration, the measurement variability, or variance (s2), is estimated from all replicate determinations made during the 6 to 12 month time period as the sum of the squared differences between each measurement and the mean concentration for the period divided by the number of measurements minus one (Taylor, 1987). This estimate of measurement variability excludes any persistent long-term bias signal (i.e., mean) that may be present in analytical measurements during this time period. Any short-term variations in systematic error (bias) that may occur within this time period are included in the estimates of measurement variability. The analytical precision can be computed from estimates of measurement variability, and is defined as the inverse of the square root of the variance.
The BSP also expresses the measurement variance relative to the mean concentration of the reference sample by computing the relative standard deviation (RSD). This expression in percent is the coefficient of variation multiplied by 100, and is written as
RSD = (sqrt(s2) / X) 100 (4)
where X is the mean of the replicate determinations. The variance is expressed relative to the mean of NWQL determinations rather than the MPV of the reference samples to provide a consistent intra- laboratory estimate of measurement variability.
The BSP data published on the CD-ROMs include NWQL analyses (laboratory analysis agency code of 80020) of 20 trace elements and 9 major dissolved ions during the water years 1985-95 (October 1, 1984 to September 30, 1995) and 5 nutrients during the water years 1986-95. These BSP data apply to all national network stream water-quality analyses made after April 1986. From October 1, 1984 to April 1986, the NWQL BSP data apply predominantly to national network stream water-quality analyses made for stations west of the Mississippi River (samples from other stations were analyzed at the Atlanta laboratory during this period; see the section entitled "National Network Laboratory Techniques" for a description of the jurisdiction of USGS laboratories). BSP data collected for water years 1981-84 at NWQL and for water years 1981-86 at the Atlanta laboratory (laboratory analysis agency code 80010) are currently being re-evaluated and are not published on the CD-ROMs. All BSP data published on CD-ROM were obtained from a USGS internal BSP data repository located in Reston, Virginia.
The BSP data published on CD-ROM were analyzed according to the NWQL method codes listed in Table 9 for the indicated water years. These methods were used in the analysis of national network stream water-quality samples. Some WQN stream water-quality data published on CD-ROM may have been analyzed in other laboratories as part of local investigations conducted at national network stations. The laboratory analysis agency code and the laboratory analytical method codes for WQN water-quality data should be examined by the user prior to use of the BSP data (see the section below "Selecting the Appropriate BSP Data").
The BSP data selected for publication on the CD-ROMs include measurement error data for assessing the accuracy (bias and variability) of laboratory methods and national network water- quality data. For each constituent, the measurement error data include the NWQL measurement of the BSP reference sample, the MPV of the reference sample, the NSD, the measurement error expressed as percent of the MPV, and an error remark code ("<") identifying estimates of error that are less-than the analytical method reporting limit. The measurement error is expressed as a percentage of the MPV by dividing by the MPV and multiplying by 100. Where the error is remarked as less than the reporting limit, estimates of the percent error are computed by expressing the reporting limit as a percentage of the MPV (a negative sign is assigned to error values where a negative difference is obtained between the laboratory measurement and the MPV). For each BSP sample, the laboratory login date and the BSP sample mixture number are reported.
For BSP measurement variability data computed from replicate samples, the earliest and latest dates for laboratory analysis (i.e., login) of each BSP sample mixture are recorded. For each constituent, the data files include the MPV of the reference sample, the mean, variance, and relative standard deviation (RSD) of NWQL measurements of the BSP reference sample, and the number of observations used to compute the mean and variability statistics for the indicated time interval.