The channel networks of 3 watersheds and 125 streamflow sampling sites on the Wyoming Range in southwestern Wyoming (fig. 1) were used for modeling monthly streamflow. The three watersheds are identified by the following USGS 12-digit hydrologic unit codes (HUC12; U.S. Geological Survey, 2016): 140401010907 (Lower South Piney Creek), 140401011102 (North Fork Dry Piney Creek), and 140401011103 (Dry Piney Creek). In this report, these three HUC12s are referred to as watersheds 1–3, respectively. The three watersheds are in the western WLCI area, which covers much of southwestern Wyoming (fig. 1).

The three watersheds are in a sparsely populated region and have many similar physical properties. Watersheds 1 and 2 are similar in drainage area (34 and 26 square miles [mi²], respectively) and mean elevation (8,428 and 8,455 feet, respectively), whereas watershed 3 is larger (55 mi²) and lower (8,031 feet) (U.S. Geological Survey, 2019a). The watersheds have a similar semiarid climate characterized by warm summers with precipitation from occasional thunderstorms and by cold winters with snow typically covering the montane-steppe landscape from November through March. Watersheds 1 and 2 have a mean annual temperature of 38.1 degrees Fahrenheit and a total annual precipitation of 10.6 inches, whereas watershed 3 is cooler (37.2 degrees Fahrenheit) and wetter (13.7 inches) (PRISM Climate Group, 2018). Geology of the watersheds is mostly sedimentary rocks of Quaternary and Tertiary age (Oriel and Platt, 1980). The watersheds are on the eastern Overthrust Belt (not shown), and the surficial units in the upper elevations and Cretaceous Mountain (not shown; to the east of watersheds 1 and 2) are much older, dating to the Cambrian. Additionally, the Overthrust Belt has many surficial and buried faults that transmit groundwater into the watersheds through springs and seeps (Zielinski and others, 1985). Land cover in the watersheds is mostly shrub/scrub (66–72 percent) and forest (20–25 percent), and irrigated land use ranges from about 0.5 to 2.5 percent (Multi-Resolution Land Characteristics Consortium, 2017). Oil and gas development began in the region in the early 1900s and is ongoing. The energy development footprint increases from watershed 1 to 3 (Walters and others, 2019). Total length of streams in the three watersheds is 372 miles, and about 75 percent (277 miles) consisted of first- and second-order streams (U.S. Geological Survey, 2019a). Annual hydrographs are characterized by snowmelt high flows in late spring and early summer (May and June) followed by low flows during summer through the following spring; the lowest flows are in fall and winter.

The only USGS streamgage close to the study area and operating during the study period (2012–17) was on Fontenelle Creek near Herschler Ranch, near Fontenelle, Wyoming (USGS 09210500; fig. 1), which has a drainage area of 152 mi² and an elevation of 6,950 feet (above the National Geodetic Vertical Datum of 1929; U.S. Geological Survey, 2021). The watershed monitored by this streamgage has a larger drainage area and lower mean elevation than all three watersheds of the study area. This difference in drainage area and elevation and the distance of this streamgage from the study area made the streamgage unfit for calibrating and validating a streamflow model.

A machine learning approach was developed in R (R Core Team, 2021) for modeling monthly streamflow from 2012 through 2017 in three watersheds on the Wyoming Range. This study was intended to investigate the utility and limitations of applying a machine learning approach to modeling streamflow with sparse data (for example, streamflow measurements made at different times and places instead of a time series of streamflow observations made at a streamgage). This study was not intended to investigate performance differences among various machine learning models used for modeling streamflow. Methodology of the modeling approach is described, including temporal and spatial conditioning processes applied to the input data and calibration and validation of the MLFLOW model. Results of the MLFLOW model are presented with discussion of the model output, including predictive performance at the sampling sites (with comparisons to other streamflow modeling studies), variable importance for explaining monthly streamflow, sensitivity to the predictor variables and streamflow observations used in fitting the model, and monthly streamflow predictions on the channel networks. The input data, model output, and R scripts for the MLFLOW model are available in an accompanying data release (McShane and Eddy-Miller, 2021).

No USGS streamgages, with time series of streamflow, were available in the study area for use in developing the MLFLOW model. One nearby streamgage (less than 20 miles from the study area) was operational during the study period (2012–17), but the streamgage was on a river with a larger drainage area and lower mean elevation than of the three watersheds of the study area. Therefore, this study relied on available discrete streamflow observations to calibrate and validate the MLFLOW model.

Streamflow measurements were made at 125 sites (fig. 1) in 35 months during 2012–17, totaling 971 discrete observations (table 1); only a single measurement was made at any site in any month. These observations were not made on a consistent spatiotemporal basis but together are representative of the annual hydrograph on a monthly timestep, including high flows in late spring and early summer (fig. 2B–E) and low flows from late summer through early spring (fig. 2F–H, A). Therefore, the observations were assumed to qualitatively represent monthly streamflow. Ecologists with the Wyoming Cooperative Fish and Wildlife Research Unit (University of Wyoming and USGS) made 763 observations at 101 sites in 30 months during 2012–17 for various research on aquatic species distribution and abundance. Most (736) of the observations were made during May through August of 2012–17, and the remainder of the observations were made during April of 2012 and 2017 and September of 2013–16. USGS hydrologists made 208 observations at 24 sites in 10 months during 2015–17 for research on groundwater/surface-water relations. Most (184) of the observations were made during June, August, and November of 2015 and 2016 and July and September of 2017; the remainder of the observations were made during April 2016 and February 2017. Streamflow measurements were made following standard USGS techniques and methods for streamflow measurement and computation using current-velocity meters, portable flumes, or pressure transducers (Rantz and others, 1982; Sauer and Turnipseed, 2010). The monthly streamflow observation data are available in the accompanying data release (McShane and Eddy-Miller, 2021).

The number of sites where streamflow measurements were made was different among the three watersheds and among the 6 years of sampling, whereas the number of months when streamflow measurements were made was similar among watersheds and among years. On average, sampling involved 42 sites and 33 months per watershed and 88 sites and 6 months per year (table 1). No sites were sampled in every month, but every site was sampled in at least 2 months, and of the 35 months sampled during 2012–17, sites were sampled in 8 months on average (McShane and Eddy-Miller, 2021). The sites sampled in the greatest number of months were sites 25, 27, and 31 (fig. 1), which were sampled in 28, 25, and 28 months, respectively. Additionally, the watershed (3) or year (2013) with the most streamflow observations had about 100 more observations than the watershed (1) or year (2014) with the fewest streamflow observations (table 2).

Observed flows at the sampling sites ranged from 0 to 19.67 cubic feet per second (ft³/s) and averaged 2.39 ft³/s (table 2), and drainage area of the sampling sites ranged from 0.64 to 78.1 mi² and averaged 14.3 mi² (fig. 2). A robust streamflow/drainage-area relation among the sampling sites was not readily apparent in every month sampled (fig. 2A–H), most likely because sites with different drainage areas were sampled in different months, confounding the relation between streamflow and drainage area with the relation between streamflow and month. Sites with the largest drainage areas (greater than 67 mi²) were sampled only in watershed 3 and only during summer (June through August, fig. 2D–F). Furthermore, sites with smaller drainage areas (less than 41 mi²) were sampled in every month (fig. 2A–H) but only in 1 or 2 years in February, April, and November (table 1; fig. 2A, B, H). Additionally, in some months (for example, May, fig. 2C), the streamflow-drainage area relation seemed to be more robust by watershed, where streamflow increased with increasing drainage area, than by year, when streamflow decreased with drainage area in some years but increased in others.

The distribution in values of streamflow observations varied more among the 6 years of sampling than among the three watersheds. For example, mean streamflow differed by only 31 percent among watersheds but by 740 percent among years (table 2). In general, streamflow values were low—median streamflow ranged from 0.4 to 4.01 ft³/s among watersheds and years; furthermore, two watersheds and 3 years had zero-flow observations (table 2). In addition, climatically, 2012 and 2013 were dry years, whereas 2017 was a wet year. These 2 dry years and 1 wet year had very low and very high mean streamflow, respectively, compared with the other 3 years (2014–16, table 2).

Based on the potential to affect streamflow, 24 variables describing physiographic, anthropogenic, and climatic characteristics were chosen as predictors in the MLFLOW model—20 variables described static physiographic and anthropogenic conditions, and 4 variables described dynamic climatic conditions (table 3). Vector data (surficial geology permeability index and number of surficial geology contacts, bedrock geology faults, springs, water diversions, roads, and water wells, table 3) were converted to raster data. Raster data describing topography (drainage area, elevation, slope-area ratio, and slope, table 3) were derived from the 30-meter (m) resolution National Elevation Dataset digital elevation model (DEM) in the USGS National Hydrography Dataset (U.S. Geological Survey, 2019a). All variables were snapped to the 30-m DEM extent. Any variable with lower spatial resolution than the 30-m DEM resolution, including base-flow index, depth to soil restrictive layer, depth to water table, and every climate variable (table 3), was resampled, which was necessary for processing the data for use in the MLFLOW model. This resampling produced data with higher spatial resolution than the original data resolution but with minimal change in the distribution of values of the original data. Each climate variable was provided as a monthly value—total for evapotranspiration and precipitation, mean for temperature, and first day of the month for snow water equivalent. Values of each predictor variable were produced throughout the channel networks of the three watersheds, which consisted of every cell of the flow accumulation grid in the medium resolution (30 m) National Hydrography Dataset with a value of 100 or more, equivalent to a drainage area of 0.09 square kilometer or more. Data on the predictor variables are available in the accompanying data release (McShane and Eddy-Miller, 2021), including raw values of the variables and values of the variables with temporal and spatial conditioning—the conditioning processes are described in the following section.

The MLFLOW model used a gradient boosting machine (Friedman, 2001, 2002; also known as boosted trees or generalized boosted models) because of the potential for the machine learning model to fit complex relations between the predictor variables (24 geology, hydrology, topography, vegetation, human, and climate variables, table 3) and the response variable (monthly streamflow; streamflow observations, table 1). Gradient boosting machines are one of many machine learning models, including random forests, support vector machines, and neural networks, that are used in various hydrologic modeling (Shen and others, 2018), including modeling of streamflow (Bellos and Carbajal, 2020). Gradient boosting machines were applied in this study using the “caret” (Kuhn 2008, 2020) and “gbm” (Greenwell and others, 2020; Ridgeway, 2020) packages in R (R Core Team, 2021).

A gradient boosting machine is a machine learning model used to solve regression or classification problems. The gradient boosting machine produces a predictive model that is an ensemble of many weaker models, which are typically implemented as decision trees. Gradient boosting is treated statistically as a numerical optimization problem with an objective of minimizing the loss of the model by adding weak learners using a functional gradient descent (Friedman, 2002). Gradient boosting machines are applied with the following features in R (Ridgeway, 2020): a loss function (squared error) to optimize; a weak learner (decision trees) for predicting; and an additive model (stochastic gradient descent) for adding weak learners to minimize the loss function. However, gradient boosting is a greedy model, meaning gradient boosting can readily overfit the data (Friedman, 2001). To limit the model from overfitting the data, the following four hyperparameters of gradient boosting machines can be tuned using packages in R (Kuhn, 2020, Greenwell and others, 2020): (1) the learning rate (shrinkage) of the decision trees; (2) the maximum interaction depth (number of branches, or intermediate nodes) of a tree; (3) the minimum number of observations per leaf, or terminal node, of a tree; and (4) the number of iterations (trees) fit. The hyperparameters of the models in this study were tuned as follows: 0.005, 0.01, or 0.05 for shrinkage; 1 or 5 for maximum interaction depth; 5 or 10 for minimum number of observations; and 500 for number of trees. More detail on modeling with gradient boosting machines in general and specifically in R is available in Kuhn and Johnson (2013).

Development of the MLFLOW model proceeded in an iterative process of model calibration and validation. Streamflow observations (monthly streamflow) used to fit the model were split into training and testing samples using repeated k-fold cross-validation (Hastie and others, 2009). The data were randomly split into 10 folds, with each fold containing 10 percent of the data. The model was iteratively trained with 9 folds and tested against 1 fold left out. The splitting of data into 10 folds was repeated 5 times, totaling 50 iterations (resamples) of training and testing. For each resample, the model fitted 500 trees, with each tree iteratively learning from the previously fitted tree.

The MLFLOW model fitted to all data had satisfactory agreement between observed and predicted streamflow (R²=0.80, NSE=0.79, logNSE=0.82, and PBIAS=0.7 percent, fig. 6). The equivalence between NSE (0.79) and logNSE (0.82) indicated the MLFLOW model performed equally well for high and low flows. PBIAS (0.7 percent) indicated the MLFLOW model did not overpredict or underpredict monthly streamflow in general. For streamflow observations less than 10 ft³/s, the model on average overpredicted by less than 1 percent. For streamflow observations toward 20 ft³/s, the model on average underpredicted by about 5 percent; however, the model had fewer streamflow observations to fit toward the upper range of values.

The MLFLOW model performed equally well for all months with streamflow observations. For each month, the mean of the distribution of residuals (observed minus predicted streamflow) of the model was about zero (fig. 7), indicating model predictions for months with fewer observations (such as February and April) were not biased by the model potentially overfitting data for months with more observations (such as June and July). For most months, the median of model predictions was higher than the median of streamflow observations, and the distribution of model predictions for all months was more limited than the distribution of streamflow observations (fig. 7), suggesting the MLFLOW model was fitting toward the mean of the data.

Goodness-of-fit metrics computed independently for every site with at least eight streamflow observations also indicated acceptable performance of the MLFLOW model (table 4). Medians of the goodness-of-fit metrics for the 85 sites analyzed were 0.81 for R², 0.74 for NSE, 0.77 for logNSE, and 2.3 percent for PBIAS (table 4), not much different from the metrics for the MLFLOW model fitted to all data (R²=0.80, NSE=0.79, logNSE=0.82, and PBIAS=0.7 percent, fig. 6). Median absolute deviation (MAD) of the metrics indicated some variation in model performance among the sites analyzed (table 4). However, the difference between median and MAD for PBIAS was less than 25 percent (median plus or minus MAD), so the MLFLOW model performed satisfactorily even for sites toward the tails of the distribution (table 4). The other three metrics were all greater than or equal to 0.5 (median minus MAD) toward the lower distributional tail, also indicating satisfactory model performance (table 4).

Model performance at the three sites with the most streamflow observations in this study (sites 25, 27, and 31, fig. 1) was acceptable (fig. 8A–C). All four goodness-of-fit metrics indicated site 31 (fig. 8C) had the best performance in general (R²=0.88, NSE=0.88, logNSE=0.91, and PBIAS=0.8 percent). Site 25 (fig. 8A) had R² of 0.89 and NSE of 0.84, comparable with values for site 31, but had worse logNSE (0.77) and PBIAS (−3.6 percent) than for site 31. Site 27 (fig. 8B) had the best PBIAS (−0.6 percent) and had logNSE (0.75) similar to the value for site 25, but R² (0.62) and NSE (0.61) were more than 25 percent less than for sites 25 or 31. Qualitatively, the MLFLOW model tended to underpredict the highest flows in most years, whereas lower flows were overpredicted in some years but underpredicted in others (fig. 8A–C). On average for the three sites, NSE (0.78) was lower than logNSE (0.81) (fig. 8A–C), indicating performance was only about 3 percent better for low flows than for high flows, the same as for the MLFLOW model fitted to all data. The tendency to underpredict the highest flows probably resulted from fewer observations of high flows than of low flows available for use in fitting the MLFLOW model.

In comparison, Miller and others (2018) used a random forest model fitted to USGS streamgage data to predict monthly streamflow for the conterminous United States. The approach of Miller and others (2018) included temporal conditioning of dynamic climatic predictor variables, as in this study. Median NSE ranged from 0.5 to 0.9 and median PBIAS ranged from −15 to 5 percent (Miller and others, 2018), compared with median NSE of 0.74 and median PBIAS of 2.3 percent for every site with at least eight observations in this study (table 4). In addition, the model in Miller and others (2018) was fitted to few data from low-order streams, unlike the MLFLOW model in this study.

Additionally, the Precipitation-Runoff Modeling System, a physically based hydrologic model, was used to model monthly streamflow in seven watersheds (about HUC12 in size) in Montana (Chase and others, 2016). USGS streamgage data were used by Chase and others (2016) to calibrate and validate the model, whereas discrete streamflow observations were used in this study. The model performance in Chase and others (2016) ranged from negative values for NSE to about 0.75, whereas in this study, median NSE was 0.74 at sites with at least eight observations (table 4) and NSE was 0.84 or more at the two sites with the most observations (fig. 8A, C).

The most important variables (statistically important in the MLFLOW model) for explaining monthly streamflow were the 6-month moving average of precipitation and the 3-month moving average of snow water equivalent (fig. 9). The top 20 important variables were populated by the following 14 moving-average variants: 4 precipitation variants, 4 snow water equivalent variants, 3 evapotranspiration variants, and 3 temperature variants. Forest cover was the only static variable among the top five variables (fig. 9). Elevation, drainage area, depth to water table, number of diversions, and number of surficial geology contacts—also static variables—also were important variables for explaining monthly streamflow.

The 20 most important variables in the MLFLOW model had simple to more complex relations with monthly streamflow as interpreted with partial dependence plots. Many of the relations between the predictor variables and monthly streamflow were intuitive. Monthly streamflow increased with increasing drainage area (fig. 10O) and number of surficial geology contacts (fig. 10P) and decreased with increasing elevation (fig. 10R) and depth to water table (fig. 10T). Monthly streamflow also increased with increasing 6-, 9-, and 12-month moving averages of precipitation (fig. 10E–G, respectively) and 3- and 6-month moving averages of snow water equivalent (fig. 10J, K, respectively) and decreased with increasing 9-month moving averages of evapotranspiration (fig. 10C) and temperature (fig. 10N). Other relations between the predictor variables and monthly streamflow were counterintuitive. Monthly streamflow decreased with increasing number of water diversions (fig. 10Q) and forest cover (fig. 10S). However, in the study area, most forest cover was at higher elevations where channels had smaller drainage areas, and most water diversions were on channels with larger drainage areas at lower elevations. Additionally, each climate variable had relations with monthly streamflow that reversed when proceeding from shorter to longer moving-average variants. For example, monthly streamflow increased with increasing current month and 1-month moving average of evapotranspiration (fig. 10A, B, respectively) and current month and 6-month moving average of temperature (fig. 10L, M, respectively), but monthly streamflow decreased with 9-month moving averages of evapotranspiration (fig. 10C) and temperature (fig. 10N). In addition, intermediate values of some predictor variables produced the highest monthly streamflow (such as depth to water table, fig. 10T) or the lowest (such as 1-month moving average of snow water equivalent, fig. 10I), whereas for other predictor variables, the relation with monthly streamflow had more than one inflection. For example, as current month of snow water equivalent increased (fig. 10H), monthly streamflow increased, then decreased, and then increased again.

The most important variables in models fitted to data from each watershed did not differ substantially among the watersheds or from the MLFLOW model fitted to all data. For each watershed, moving-average variants of precipitation and snow water equivalent were still the most important variables (fig. 11A–C), suggesting insubstantial spatial variability in the relation between monthly streamflow and the predictor variables. The effect of precipitation and snow water equivalent on monthly streamflow was apparently as important across small or large areas (each watershed modeled separately; fig. 11A–C) as across the study area (all watersheds modeled together; fig. 9). However, forest cover, contrary to its ranking as third most important variable in the model fitted to all data (fig. 9), was the fifth most important variable for watershed 1 (fig. 11A) and was not among the top five important variables for the other two watersheds (fig. 11B, C). In contrast, the 9-month moving average of snow water equivalent was not among the top 20 important variables in the model fitted to all data (fig. 9) but was the third most important variable for watershed 3 (fig. 11C). Additionally, drainage area was among the top five important variables for watersheds 2 and 3 (fig. 11B, C, respectively), but drainage area was only the 11th most important variable in the model fitted to all data (fig. 9). The increased importance of drainage area was understandable for watershed 3, which has the largest area (55 mi²) of the three watersheds, but the increased importance of drainage area was surprising for watershed 2, which has the smallest area (26 mi²) (U.S. Geological Survey, 2019a).

The most important variables in models fitted to data from each year differed considerably among the years and from the MLFLOW model fitted to all data. Six static variables (cropland cover, number of surficial geology contacts, number of water diversions, forest cover, depth to water table, and drainage area)—from one to three variables for any year—were among the top five most important variables (fig. 12A–F), which suggests considerable temporal variability in the relation between the predictor variables and monthly streamflow. In addition, moving-average variants of precipitation (fig. 12A, C) and snow water equivalent (fig. 12D, F) were among the top five important variables in only 2 years, whereas moving-average variants of evapotranspiration and temperature were among the top five important variables in all but 1 year (fig. 12B–F). During a shorter period (each year modeled separately; fig. 12A–F), the effect of evapotranspiration and temperature seemed more important, whereas the effect of precipitation and snow water equivalent seemed more important during the longer study period (all years modeled together; fig. 9). For 3 years, a dynamic variable was the most important—the 1-month moving average of snow water equivalent for 2015 (fig. 12D) and the 9-month moving average of evapotranspiration for 2016 (fig. 12E) and 2017 (fig. 12F). However, for the other 3 years, a static variable was the most important—cropland cover for 2012 (fig. 12A) and 2013 (fig. 12B) and forest cover for 2014 (fig. 12C). Although the varying importance of static variables from 1 year to another year may seem illogical, several reasons are possible. Static variables can interact differently with dynamic variables from year to year, resulting in the increased importance of a static variable. Dynamic variables alone can decrease in importance from year to year, which relatively increases the importance of a static variable. Dynamic variables may vary less within 1 year than during several years, resulting in the decreased importance of dynamic variables for a single year. Lastly, one static variable may substitute, in relative importance, for many correlated dynamic variables.

Temporal and spatial conditioning intensified the relation of many predictor variables with monthly streamflow, resulting in more information that the MLFLOW model could use for predicting streamflow. For many of the variables, such as forest cover or surficial geology contacts, unconditioned values of many cells on the channel networks were zero (figs. 4A, 5A), but with spatial conditioning, values of most cells on the channel networks were nonzero (figs. 4B, 5B), providing the MLFLOW model with more diversely valued variables that might better explain variation in streamflow. Temporal conditioning increased r for the dynamic variables by as much as 0.34 compared with the equivalent variable without conditioning (table 5). For example, current month of precipitation had r of −0.02 (ppt_00, table 5), whereas the 6-month moving average of precipitation had r of 0.36 (ppt_06, table 5). Spatial conditioning increased r for the moving-average variants of the dynamic variables by as much as 0.25 (table 5). For example, the 1-month moving average of snow water equivalent without conditioning had r of 0.10 (swe_01, table 5), but the 1-month moving average of snow water equivalent with conditioning had r of 0.35 (swe_01_a, table 5). For the static variables, spatial conditioning increased r by as much as 0.15, such as for surficial geology contacts (contacts, table 5). Also, for some static variables, correlation between the unconditioned variable and monthly streamflow was not possible to compute because all values of the variable were zero for any streamflow observation, such as for water diversions (diversions, table 5). However, spatial conditioning produced a correlation of the variable with streamflow; for example, r of 0.09 for water diversions (diversions_d, table 5).

Models fitted to different combinations of predictor variables were used to qualitatively assess sensitivity of the MLFLOW model to selection and conditioning of the predictor variables—static versus dynamic; conditioned versus unconditioned. Performance improved progressively from model 1 to model 5 (table 6). All models had acceptable performance, although model 1 was marginal for three goodness-of-fit metrics (R²=0.54, NSE=0.52, and logNSE=0.49, table 6). However, model 5 (the model fitted to all data) had the best performance (R²=0.80, NSE=0.79, logNSE=0.82, and PBIAS=0.7 percent, table 6).

Related to this study, Jaeger and others (2019) used a machine learning approach to modeling streamflow permanence (presence or absence of streamflow) in the Columbia River Basin. Jaeger and others (2019) also applied spatial conditioning by area-averaged accumulation to the predictor variables, as in this study. Model performance was about 80 percent (out-of-bag error rate of 20 percent; Jaeger and others, 2019), which was comparable with this study (R²=0.80, NSE=0.79, and logNSE=0.82, table 6). Updated streamflow permanence modeling has been progressing in the upper Missouri River Basin (not shown), including temporal conditioning of dynamic predictor variables and spatial conditioning by distance-decayed accumulation (Roy Sando, U.S. Geological Survey, oral commun., 2020), similar to this study.

Different combinations of static or dynamic variables and unconditioned or temporally or spatially conditioned variables, or both, were used to fit the MLFLOW model, which resulted in considerable improvements in performance. Temporal conditioning of dynamic variables in model 2 increased R², NSE, and logNSE by as much as 0.09 compared with the unconditioned dynamic variables in model 1 (table 6). Adding unconditioned physiographic and anthropogenic variables in model 3 increased R², NSE, and logNSE by as much as 0.15 compared with just the climatic variables in model 1 (table 6). Spatial conditioning of all variables in model 4 increased R², NSE, and logNSE by as much as 0.05 compared with the unconditioned dynamic and static variables in model 3 (table 6). Spatial conditioning of all variables together with temporal conditioning of dynamic variables in model 5 increased R², NSE, and logNSE by as much as 0.13 compared with just the spatial conditioning in model 4 (table 6). PBIAS varied less than 1 percent among the five models (table 6), indicating the MLFLOW model did not overpredict or underpredict monthly streamflow in general within the study area during the study period. In addition, logNSE was worse than NSE by 0.03 in model 1 but was better by 0.03 in model 5 (table 6). This 0.06 improvement of logNSE relative to NSE indicates selection of all variables with all temporal and spatial conditioning improved prediction of low flows relative to prediction of high flows.

The MLFLOW model was most sensitive to selection of dynamic climatic variables. Unconditioned dynamic climatic variables (model 1) alone explained 54 percent of the variance (R²=0.54, table 6) in monthly streamflow, whereas adding static physiographic and anthropogenic variables (model 3) only explained 12 percent more of the variance (R²=0.66, table 6), indicating the greater importance of the time series data. Also, spatial conditioning of all variables together with temporal conditioning of dynamic variables (model 5) increased the variance explained in the MLFLOW model by another 14 percent (R²=0.80, table 6). Kratzert and others (2019) used another machine learning approach (long short-term memory networks) with different combinations of dynamic and static variables that performed similarly to this study. Models developed with only dynamic climatic variables performed well (NSE=0.63) but performed moderately better with the inclusion of static physiographic variables (NSE=0.74) (Kratzert and others, 2019). However, model performance indicating that the dynamic variables had greater explanatory power than the static variables in the MLFLOW model was not surprising because time series data intrinsically have more variation, temporal and spatial, to use in fitting the model to the streamflow observations, which also are innately variable, temporally and spatially.

Sensitivity of the MLFLOW model to using data from a different area (watershed) or period (year) was qualitatively assessed by differentially grouping streamflow observations by watershed or year for use in model training and testing. For models trained with all watersheds and years, performance was better in testing on observations from each watershed than from each year separately (table 7), indicating the MLFLOW model had greater sensitivity to temporal than to spatial differences in the data. Means of the goodness-of-fit metrics were 0.79 for R², 0.79 for NSE, 0.82 for logNSE, and 1.4 percent for absolute value of PBIAS for models trained with all watersheds but were 0.68 for R², 0.67 for NSE, 0.67 for logNSE, and 3.5 percent for absolute value of PBIAS for models trained with all years (table 7). However, in contrast, models trained with all except 1 year left out sequentially and tested on the left-out year performed better than models trained with all except one watershed left out sequentially and tested on the left-out watershed. Means of the goodness-of-fit metrics were 0.51 for R², 0.49 for NSE, 0.50 for logNSE, and 11 percent for absolute value of PBIAS for models trained with a watershed left out but were 0.56 for R², 0.53 for NSE, 0.56 for logNSE, and 12 percent for absolute value of PBIAS for models trained with 1 year left out (table 7).

In general, performance for models trained with all except one watershed or 1 year left out sequentially was satisfactory for all the test watersheds and years (table 7). Model performance was only unsatisfactory (for one goodness-of-fit metric) for watershed 3 (logNSE=0.37) and 2013 (logNSE=0.37) (table 7), indicating the relations between the predictor variables and monthly streamflow at low flows in watershed 3 and in 2013 were difficult to explain with data from the other watersheds or years. Less similarity of data among the three watersheds than of data among the 6 years may explain the better performance of the model in testing on 1 year left out from model training than on one watershed left out from model training. However, another explanation may be fewer data were excluded from model training by leaving out 1 year than by leaving out one watershed, and consequently, more data (with more variance) were included in model testing on the left-out watershed than on the left-out year—on average, each watershed and year had 324 and 162 observations, respectively.

Sensitivity of the MLFLOW model to the quantity of data used was qualitatively assessed by progressively reducing streamflow observations by percentage of sites or months available for use in model fitting. Performance was better for models fitted to fewer sites than to fewer months of observations (table 8), which indicated the MLFLOW model was more sensitive to temporal than to spatial differences in the data. The goodness-of-fit metrics for models fitted to 50 percent fewer months were worse than for models fitted to 50 percent fewer months by 0.12 for R², 0.14 for NSE, 0.17 for logNSE, and 6.9 percent for absolute value of PBIAS (table 8). In addition, further reduction in sites and months from 50 to 15 percent affected the goodness-of-fit metrics more for models fitted to fewer months than to fewer sites. On average, for three goodness-of-fit metrics (R², NSE, and logNSE), values decreased by 0.21 for models fitted to 85 percent fewer sites but decreased by 0.27 for models fitted to 85 percent fewer months (table 8). Performance was still satisfactory for models fitted to 50 percent of sites and was marginally satisfactory for models fitted to 50 percent of months but was no longer satisfactory for models fitted to only 15 percent of sites or months (table 8).

The MLFLOW model fitted to all data performed very well (R²=0.80, NSE=0.79, logNSE=0.82, and PBIAS=0.7 percent, table 8), but performance in general progressively decreased for models fitted to progressively reduced percentages of sites or months (table 8). However, the reductions in sites and months were applied using random samples of the data, which may explain the poor performance of models fitted to only 15 percent of sites or months. The 15-percent samples of sites or months used to train the MLFLOW model may not have been adequately representative of the variance in the remaining 85 percent of the data used to test the model. If sites could be sampled more uniformly across a watershed, focusing on proportional representation of small, medium, and large streams (by drainage area), or if months could be sampled more intentionally throughout the year, focusing on key moments of the annual hydrograph for streams at different elevations (low, medium, and high), then data for fewer sites and months might be sufficient for the MLFLOW model to perform as well as the model performed with 971 streamflow observations.

The MLFLOW model predicted spatially and temporally continuous monthly streamflow for the study area (17,518 cells; fig. 1) and study period (72 months; 2012–17), using 971 discrete streamflow observations (table 1). Spatial and temporal variations in the streamflow predictions are discussed in this section using, as an example, a subset of the study area (part of the channel network of watershed 2; figure area, fig. 1) and a subset of the study period (every month of 2017 and every year in August).

Intra-annual variation in streamflow was simulated realistically by the MLFLOW model—seasonality of streamflow was well characterized. Predicted flows in 2017 were much lower from January through March (fig. 13A–C) and from August through December (fig. 13H–L), months before or after the snowmelt high flows during April through June (fig. 13D–F). Using February, May, August, and November to represent winter, spring, summer, and fall, respectively, mean of cells on the channel network was 8.9 ft³/s in May (fig. 13E) but only 1.0, 1.8, and 0.51 ft³/s in February, August, and November, respectively (fig. 13B, H, K, respectively). Furthermore, in May, some cells had predicted flows of almost 17 ft³/s (fig. 13E), whereas in February and November, many cells on the channel networks had predicted flows of 0 ft³/s (fig. 13B, K, respectively). However, because 2017 was a wetter year, no cells on the channel network in August had predicted flows of 0 ft³/s (zero flow; fig. 14F), whereas in August of 2012, which was a drier year, about 45 percent of cells on smaller channels had zero-flow predictions (fig. 14A). In February, predicted flows were higher in smaller channels than in larger channels (fig. 13B), but in May, larger channels had considerably higher predicted flows (fig. 13E), and in August and November, streamflow receded, remaining higher in more intermediate channels (fig. 13H, K, respectively). Higher flows were observed at sites with smaller drainage areas and higher snow water equivalent in February, at sites with larger drainage areas in May, and at sites with larger drainage areas or higher precipitation in August and November (McShane and Eddy-Miller, 2021).

The MLFLOW model also realistically simulated interannual variation in streamflow, well representing yearly hydroclimatic conditions. Predicted flows in August were low in 2012 and 2013 (fig. 14A, B, respectively), which were drier years, but warmer in 2012 and cooler in 2013 (McShane and Eddy-Miller, 2021), whereas predicted flows were high in 2014 and 2017 (fig. 14C, F, respectively), which were years with more normal temperature but wetter, with more rainfall in 2014 and more snowpack in 2017 (McShane and Eddy-Miller, 2021). Although 2015 was a drier, warmer year similar to 2012, predicted flows were higher in 2015 (fig. 14D) than in 2012 (fig. 14A), most likely because 2015 followed 2014, a wetter year. Similarly, although 2016 was a wetter year similar to 2014 and 2017, predicted flows in 2016 were more intermediate (fig. 14E) similar to 2015 (fig. 14D), most likely because the preceding year, 2015, was drier. Using 2012 and 2017 to represent a range of hydroclimatic conditions from warmer and drier to cooler and wetter, respectively, mean predicted flow was only 0.12 ft³/s in 2012 but 1.8 ft³/s in 2017. Moreover, maximum predicted flow in 2012 was only 0.61 ft³/s, whereas minimum predicted flow in 2017 was 0.19 ft³/s. In 2017, a year with more snowpack, mean predicted flow in August was 1.8 ft³/s, whereas in 2014, a year with more rainfall, mean predicted flow in August was 2.4 ft³/s, indicating the responsiveness of the model to recent-month precipitation versus earlier-year snow water equivalent (McShane and Eddy-Miller, 2021). In August, observed flows at sites on larger channels were lower in 2012 than in 2013 because these sites had lower snow water equivalent in 2012 than in 2013. Observed flows were higher at sites on larger channels in 2017 but were higher at sites on more intermediate channels in 2014. This difference resulted from higher snow water equivalent at sites on larger channels in 2017 than in 2014 and higher precipitation at sites on more intermediate channels in 2014 than in 2017 (McShane and Eddy-Miller, 2021).

The greatest utility of the modeling approach is the ease of use and the speed of processing input data, running the model, and interpreting the model output, whereas the greatest limitation is the need for spatially and temporally representative streamflow observations to drive the model. Not only is the quantity of observations an important limitation but the quality of observations also is an important limitation. Streamflow measurements might be made by poorly trained persons and errors could be proportionally larger at low flows than at high flows; however, in this study, all measurements were made by well trained personnel following standard USGS techniques and methods (Rantz and others, 1982; Sauer and Turnipseed, 2010). In addition, the MLFLOW model performed well with the available streamflow measurements, which were made at a spatially representative number of sites and in a temporally representative number of months.

Streamflow predictions seemed to well represent the annual hydrograph within the study area during the study period. However, streamflow measurements were not made in January, March, October, or December, so model performance was not computable for those 4 months. In addition, for the other 8 months, streamflow measurements were not made at every site in every month of all 6 years, so the evaluation of predictive performance was more limited than what would be possible for a model developed using time series data from a USGS streamgage. Also, model predictions in this study were assumed representative of natural streamflow. Although three anthropogenic predictor variables (water diversions, roads, and water wells) were used in the MLFLOW model, the variables were merely informative of potential hydrologic alteration in a qualitative sense. Therefore, the MLFLOW model did not quantitatively characterize any actual streamflow regulation that was due to diversion, release, or return of water.

These observations, although single measurements in a month, were treated as representative of mean monthly streamflow for use in the MLFLOW model. Because the annual hydrograph of the study area is characterized by a relatively short period (about 3 months) of higher flows and a much longer period (about 9 months) of lower flows, a streamflow measurement made on any day of the month is likely representative of the mean monthly streamflow for most months of the year. However, during the rise and fall of the high-flow period, measurements made earlier and later in a month may be quite different from each other. Measurements may be lower in early May than in late May, and measurements may be higher in early June than in late June. Therefore, the magnitude of the highest flow may not be accurately represented with the streamflow observations, but the magnitude of the shorter high-flow period (2 months) relative to the magnitude of the longer low-flow period (8 months) was evident with the available streamflow observations.

The MLFLOW model, fitted as is with the data used in this study, is not transferable to another watershed unless the other watershed has similar statistical distributions of the physiographic, anthropogenic, and climatic variables. Transfer of the model to another watershed with values of these variables at the margins of the variables’ statistical distributions would require judgment on the reliability of the model’s application. However, because of the data-driven nature of the machine learning approach in this study, new streamflow observations in addition to the current observations can be used to refit the model. Nonetheless, transferring the MLFLOW model, fitted as is, to a watershed with a different hydroclimatic regime would not be as reasonable as fitting a new model to data for the new regime.

The modeling approach in this study was not a process-based model with mathematical functions derived from first principles or empirical research. Instead, temporal and spatial conditioning processes were applied to the predictor variables as a means to substitute simple properties of the data—temporally and spatially averaged or decayed values of climatic, physiographic, and anthropogenic variables—for some of the complex process-based functions (requiring much parameter optimization) for climate and land surface elements in a physically based hydrologic model.

Temporal conditioning was an effective and efficient way of increasing the information content of the dynamic climatic variables. A moving average was computationally simple and provided a simplified means for simulating short to long periods of the water storage change in a runoff model. For monthly streamflow as modeled in this study, the effect of climate is less immediate than in an event-based model. Generating a progression of moving average values of the time series data provided the model with more dimensionality for exploring the relation of climate to streamflow. However, the multiplicity of moving-average variants of the dynamic variables may have increased the use of these variables in the MLFLOW model because of the greater abundance of the dynamic variables relative to the static variables. In addition, too many moving-average variants of the climate variables can complicate evaluation of variable importance, including interpretation of partial dependence plots.

Spatial conditioning also had an important effect on the information content of the dynamic and static variables. Spatial conditioning was especially important to the static variables with binary data because no or few nonzero values of water diversions, springs, roads, water wells, surficial geology contacts, or bedrock geology faults corresponded to any sampling site, meaning these six variables (without spatial conditioning) had no variance for the MLFLOW model to explain. However, spatial conditioning of the unconditioned variables altered them from discrete (0 or 1) to continuous (ranging from 0 to 1) data for each cell (with only nonzero values on the channel network). Consequently, all sampling sites had a nonzero value of the six spatially conditioned variables—variance was generated in these variables for the MLFLOW model to explain. Area-averaged or distance-decayed accumulation of the spatial data not only increased explanatory power of the variables but also modified incompatible spatially discrete variables into applicable spatially continuous variables.

However, conditioning of the variables was less important than selection of dynamic variables for use in the MLFLOW model. The variance of the time series data was more attuned to the variance of the streamflow observations because every monthly streamflow observation had a corresponding monthly value of evapotranspiration, precipitation, snow water equivalent, and temperature. Therefore, the selection of additional dynamic variables might increase performance more than any static variable already selected for use in the MLFLOW model.

Gradient boosting machines used in this study are just one of many available machine learning models, and other circumstances might make a different model better. However, most machine learning models can be readily applied in R, such as neural networks, support vector machines, and random forests. The “caret” package in R has more than 100 different models (Kuhn, 2020). Carlisle and others (2016) evaluated five machine learning models, including random forests and gradient boosting machines, and determined that random forests performed more ably than the other four machine learning models. However, in this study, the MLFLOW model also was developed with a random forest, but performance was not as good as for the MLFLOW model developed with the gradient boosting machine.

The modeling approach in this study is relatively simple to apply. Other predictor variables can be used in the MLFLOW model, and variables that were most important in the model are available for larger areas and longer periods than were applicable to this study. The R scripts are adaptable—other machine learning models can be substituted for the gradient boosting machine used in the MLFLOW model. Although familiarity with R is necessary, only a working knowledge of hydrology (for selecting appropriate predictor variables and evaluating the quality of streamflow observations) and a rudimentary understanding of machine learning models are needed. Therefore, this modeling approach is practicable for other scientists who work with water but who are not hydrologists.