For more information on the USGS—the Federal source for science about the Earth, its natural and living resources, natural hazards, and the environment—visit
For an overview of USGS information products, including maps, imagery, and publications, visit
Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government.
Although this information product, for the most part, is in the public domain, it also may contain copyrighted materials as noted in the text. Permission to reproduce copyrighted items must be secured from the copyright owner.
Invasive annual grasses (IAGs)—including
Maps are generally developed based on remotely sensed inputs, including satellite or aerial photos, which are measured over time (
For over 20 spatial data products representing annual grasses in the western United States published since 2010, refer to the following resources:
The choice of IAG spatial products is not always easy. Consider these seven key criteria in the order listed when selecting maps for IAG management.
Determine if your geographic area is fully covered in the IAG map, which can sometimes be time consuming. Refer to our IAG product compendium for at-a-glance lists of States and thumbnails depicting the geographic extent of each map product. More detailed information, such as the range of latitude and longitude coordinates the product covers, can be found in our detailed product database.
Different map outputs and data types are better suited for different IAG management objectives. Spatial data describing species occurrence (presence or absence) are suitable for identifying areas where invasion has already occurred. Maps of habitat suitability predict the potential for invasion. Abundance metrics like percent cover or total biomass describe the intensity of invasion and are useful for targeting actions designed for early or fully established invasions (
Higher-resolution maps provide estimates of IAGs using smaller pixels (for example, 30×30 meters). A 30-m map will have 9 pixels for every pixel in a 90-m map. If accurate, higher-resolution maps can allow you to distinguish fine-scale differences in IAG occurrence or cover across the landscape. High resolution does not necessarily mean high accuracy: resolution describes the scale at which the map tries to distinguish pixels, and accuracy describes how well the map actually does this.
The distribution and cover of IAGs are dynamic and highly variable within the growing season and across years. The timeframe represented in the map can affect interpretation and confidence in using the map for management applications. Recent data may provide a better snapshot of recent change than an older product. Time-series maps can estimate IAG expansion and contraction over time. Products that combine several years of data to create one map may better map areas with persistent IAG infestation, but they may not highlight recent changes (
Developers of IAG maps often indicate how their products are intended to be used and note key limitations. To ensure map data are used appropriately, users should consider whether their management needs align with the developer’s recommendations and examples of the product being used for similar purposes. Our product compendium and product database describe any recommendations and caveats stated by the authors of each map product.
Accuracy describes how well the map’s estimated locations and percent cover for IAGs represents the actual locations and percent cover on the ground. Although technically complex, a basic understanding of how spatial data products are evaluated is important when comparing and selecting products. A diversity of methods and metrics with varying statistical rigor are used to evaluate accuracy of products (Liu and others, 2009). Refer to “Evaluating the Accuracy of Spatial Products” for more details.
When making a final decision on a map product, it is best to look at the technical details of the model approach. The relevance and quality of model inputs (for example, aerial or satellite imagery or field measurements) can influence model limitations and applications (
Narrow down the full set of map products to five or fewer that might meet your needs by assessing basic information (considerations A–D above). Then, move towards your final selection by considering the complex details in depth (steps 1–2). Finally, assess the product’s overall applicability by visualizing it with other information (steps 3–5).
Learn how each product was developed and applied (considerations E and F), then scrutinize the details important for your application. Our product database characterizes more than 40 attributes for each product that we reviewed, and our product compendium provides simplified comparison tables and 2-page summaries.
Most models provide an accuracy score for model predictions (consideration F), but comparisons among products can be challenging owing to the variety of evaluation methods and metrics used. Accuracy scores themselves may have inaccuracy. Products with moderate accuracy scores estimated with more rigorous tests (for example, evaluated with fully independent data) may be as good as products with high accuracy scores estimated with less rigorous methods (for example, evaluated with nonindependent data). Tests that included data from your region of interest (consideration G) may yield higher confidence in the accuracy score than tests that did not (
Overlay IAG maps with other data (for example, IAG point locations, maps of other landscape features) and evaluate the completeness of coverage. Use complementary vegetation maps and high-resolution aerial imagery to assess the degree of agreement among data. Individually, all such information will be incomplete and have limitations (
Local information (for example, field and monitoring data) and field knowledge can help identify how well map products represent on-the-ground conditions. Regional map data are general characterizations of IAGs and should not always be expected to map small locations with high accuracy. Incorporating local information can help identify and account for discrepancies between regional and local patterns. Pairing IAG maps with high-resolution aerial imagery (
To resolve questions about the limitations of IAG spatial data, consider reading the original documentation or contacting the map developer. Use-case examples can also provide insight into common applications and highlight potential limitations.
Prediction of invasive annual grass (IAG) distribution (left; from Dahal and others [2020],
Accuracy evaluation is a complex topic, and a rigorous comparison requires more than simply determining whether one product's accuracy score is higher than another product’s accuracy score. It is important to examine whether and how the scoring measures can be compared and what dataset was used to estimate them.
Many different accuracy measures are used to evaluate IAG models (Liu and others, 2009). Accuracy measures differ between products that map continuous outputs (for example, percent cover) and those that map categorical outputs (for example, occurrence;
Evaluations using independent data are more rigorous than those that reuse model building data for validation (
[All measures listed could be applied to either within-sample or fully independent data. IAG, invasive annual grass; %, percent; R2, coefficient of determination; MAE, mean of absolute error; RMSE, root mean squared error; nMAE, normalized mean of absolute error; nRMSE, normalized root mean squared error; >, greater than; <, less than; PCC, percent correctly classified; ROC, receiver operating characteristic; AUC, area under the curve; AUC-ROC, area under the curve receiver operating characteristic; AUPRC, area under the precision recall curve]
Metric | Definition and interpretation | Possible range; Interpretation | Observed range |
Mapped IAG output type—Continuous |
|||
---|---|---|---|
R2 | R2 is a measure of how well the model fits the data, which describes the proportion (0–100%) of the variance in the dependent variable that is predictable from the explanatory variables. An R2 of 0.36 indicates 36% of the variation in the data is explained by the covariates (explanatory variables). | 0–1 or 0–100%; larger is better | 0.340–0.980 |
MAE and RMSE | MAE and RMSE are estimates of the error between predicted and observed data. They are calculated from the average difference between the model’s predicted value for a data point and the actual value. For example, an RMSE value of 5% means on average, the true value will fall within 5% of the model’s prediction. | 0–20% or more; smaller is better | 0.87–14.00% |
nMAE and nRMSE | nMAE or nRMSE are rescaled to make the measures comparable across studies by normalizing them to range from 0 (perfect fit) to 1 (no better than random chance). | 0–1; smaller is better | 0.130–0.700 |
Mapped IAG output type—Categorical |
|||
PCC | PCC is the percentage of pixels correctly predicted to be IAG (true positives) or correctly predicted to not have IAG (true negatives). | 0.5–1.0; larger is better | 0.520–0.975 |
Kappa (κ) | Cohen's Kappa coefficient is a more statistically robust version of PCC that accounts for the probability of correct classification occurring by random chance. | 0–1.0; larger is better | 0.511–0.945 |
ROC | ROC is a graphed curve showing a tradeoff between the rate of true positives versus false positives for all possible thresholds of probability. | Interpreted visually: the further the line of the curve bends into the upper-left corner of the graph, the better. | |
AUC and AUPRC | AUC (sometimes called AUC-ROC) is a metric that summarizes the ROC curve. Larger values indicate a better ability to accurately predict both IAG occurrence and non-occurrence. Some papers report AUPRC, an alternative that puts more emphasis on the true positives to focus on detecting all invaded areas. | 0.5–0.7 = poor |
0.854–0.994 |
More than one spatial product may be suitable for an intended application, with potential tradeoffs (
[IAG, invasive annual grass; R2, coefficient of determination; AUC, area under the curve; %, percent]
Consideration | Product A | Product B | Tradeoffs for application |
Percent-cover map | Presence-absence map | The percent-cover map of product A may provide information on low to moderate percent cover of invasion in areas that could be targeted for herbicide application (for example, areas representing early infestation; |
|
IAGs (any species) | Cheatgrass | Both products are suitable if manager does not need species-specific information. If product B can reliably distinguish cheatgrass from other IAGs, it may provide species-specific information, but may include other IAGs as cheatgrass if discrimination is limited. | |
Desert ecoregions of the western U.S.A. | Entirety of western U.S.A. | Both products cover the smaller extent. Examining the model details and evaluation may help determine whether the smaller-extent model, with spatially constrained environmental inputs, better represents IAGs than the larger-extent model representing a wider range of environmental conditions ( |
|
30 meters | 250 meters | Product A supports spatial targeting of treatment areas using higher-resolution maps. Higher spatial resolution can more accurately map smaller patches of IAGs, but not necessarily. Examine for tradeoffs between resolution and accuracy (see “Evaluation accuracy”). | |
Single map of average condition, 2016–2020 | Annual maps that create a time series, 1990–2016 | The composite map of product A can highlight recent IAG infestations and may better map all the areas suitable for the species by accounting for annual variation in IAG patches owing to weather. The time series of annual maps can be used to map where IAGs are changing through time, such as invasion fronts, but product B does not characterize recent coverage of IAGs. | |
Spectral imagery, topography, soil, climate, vegetation | Spectral imagery, topography, climate | Additional input variables may increase local accuracy of product A, but not necessarily, since some models with very few variables have high accuracy. Note that overreliance on indirect predictors like topography can reduce model performance ( |
|
Within sample R2 = 0.71 | Cross-validation sample AUC = 0.91 | Because product A is a continuous variable and product B is a categorical variable, they use different accuracy measures that cannot be directly compared. Product A’s R2 indicates good accuracy (71% of the variability in IAG percent cover is explained by their model) but it is based on within-sample validation which is less rigorous and likely to overestimate accuracy. Product B’s cross-validation sample is more rigorous, and 0.91 AUC value gives it an outstanding accuracy rating, but users should check region-specific accuracy (if possible). |
[IAG, invasive annual grass; R2, coefficient of determination; AUC, area under the curve]
Consideration | Product A | Product B | Tradeoffs for application |
Percent cover map | Presence-absence map | Both products are suitable to assess areas affected by IAGs at some scale. Percent cover (product A) may better represent level of habitat degradation at a local or patch scale. Presence or absence (product B) can show habitat degradation at a landscape scale. | |
IAGs (any species) | Cheatgrass | The species-level maps may not be needed if the wildlife species of interest responds similarly to different IAG species and management options are similar among IAGs. | |
Desert ecoregions of the western U.S.A. | Entirety of western U.S.A. | Broader-extent maps may support comparisons of IAG threats across the full extent of the wildlife species’ range, whereas ecoregion-specific maps may not cover the species’ range. | |
30 meters | 250 meters | If the wildlife species selects habitat and responds to habitat degradation at scales generally greater than 250 meters, both resolutions should represent the scale of habitat selection well. If it is a small-bodied species that selects at fine spatial scales, product A may be more suitable. | |
Single map of average condition, 2016–20 | Annual maps that create a time series, 1990–2016 | The more recent years (2016–2020) of product A describe current coverage, and the multiyear composite may better capture the full range of climatic conditions that can support IAG in different years. The time series (product B) could be useful to assess long-term trends in habitat degradation and support assessments that link IAGs with population change. | |
Spectral imagery, topography, soil climate, vegetation | Spectral imagery, topography, climate | Additional input variables may increase local accuracy of product A, but not necessarily. Fewer input variables could increase the generality of the IAG map, allowing extrapolation and comparison across the full extent of wildlife habitat. | |
Within sample R2 = 0.71 | Cross-validation sample AUC = 0.91 | See discussion in |
Nathan D. Van Schmidt, Jessica E. Shyvers, D. Joanne Saher, Bryan C. Tarbox, Julie A. Heinrichs, Cameron L. Aldridge. For more information contact
Center Director, Fort Collins Science Center
U.S. Geological Survey
2150 Centre Ave., Building C
Fort Collins, CO 80526-8118