U.S. Geological Survey Resources for Comparing and Selecting Spatial Data

For over 20 spatial data products representing annual grasses in the western United States published since 2010, refer to the following resources:

Product database.—Compares spatial data product attributes in a sortable, filterable spreadsheet: https://doi.org/10.5066/P9VW97AO.

Product compendium.—Includes 2-page spatial product summaries with key attributes, comparison tables, and web resources: https://doi.org/10.3133/dr20221152.

Journal article.—A review bridging science, spatial data, and action to manage invasive annual grasses: https://doi.org/10.1016/j.rama.2022.01.006.

A. Geographic Extent

Determine if your geographic area is fully covered in the IAG map, which can sometimes be time consuming. Refer to our IAG product compendium for at-a-glance lists of States and thumbnails depicting the geographic extent of each map product. More detailed information, such as the range of latitude and longitude coordinates the product covers, can be found in our detailed product database.

B. Type of Map Outputs

Different map outputs and data types are better suited for different IAG management objectives. Spatial data describing species occurrence (presence or absence) are suitable for identifying areas where invasion has already occurred. Maps of habitat suitability predict the potential for invasion. Abundance metrics like percent cover or total biomass describe the intensity of invasion and are useful for targeting actions designed for early or fully established invasions (He and others, 2015). Many regional-scale map products do not distinguish among different species of annual grasses.

C. Spatial Resolution

Higher-resolution maps provide estimates of IAGs using smaller pixels (for example, 30×30 meters). A 30-m map will have 9 pixels for every pixel in a 90-m map. If accurate, higher-resolution maps can allow you to distinguish fine-scale differences in IAG occurrence or cover across the landscape. High resolution does not necessarily mean high accuracy: resolution describes the scale at which the map tries to distinguish pixels, and accuracy describes how well the map actually does this.

D. Recentness

The distribution and cover of IAGs are dynamic and highly variable within the growing season and across years. The timeframe represented in the map can affect interpretation and confidence in using the map for management applications. Recent data may provide a better snapshot of recent change than an older product. Time-series maps can estimate IAG expansion and contraction over time. Products that combine several years of data to create one map may better map areas with persistent IAG infestation, but they may not highlight recent changes (fig. 1; Rocchini and others, 2015).

E. Recommended Use and Caveats

Developers of IAG maps often indicate how their products are intended to be used and note key limitations. To ensure map data are used appropriately, users should consider whether their management needs align with the developer’s recommendations and examples of the product being used for similar purposes. Our product compendium and product database describe any recommendations and caveats stated by the authors of each map product.

F. Evaluation of Accuracy

Accuracy describes how well the map’s estimated locations and percent cover for IAGs represents the actual locations and percent cover on the ground. Although technically complex, a basic understanding of how spatial data products are evaluated is important when comparing and selecting products. A diversity of methods and metrics with varying statistical rigor are used to evaluate accuracy of products (Liu and others, 2009). Refer to “Evaluating the Accuracy of Spatial Products” for more details.

G. Model Approach and Inputs

When making a final decision on a map product, it is best to look at the technical details of the model approach. The relevance and quality of model inputs (for example, aerial or satellite imagery or field measurements) can influence model limitations and applications (Sofaer and others, 2019). Ideally, inputs represent the full range of environmental conditions that exist in the area of interest (He and others, 2015). Models that use dated inputs or exclude some environmental predictors of where IAGs occur may limit how maps can be interpreted and used to support management activities (Uden and others, 2015; Sofaer and others, 2019).

Step 1—Compare and Contrast the Details

Learn how each product was developed and applied (considerations E and F), then scrutinize the details important for your application. Our product database characterizes more than 40 attributes for each product that we reviewed, and our product compendium provides simplified comparison tables and 2-page summaries.

Step 2—Compare Model Evaluation Results

Most models provide an accuracy score for model predictions (consideration F), but comparisons among products can be challenging owing to the variety of evaluation methods and metrics used. Accuracy scores themselves may have inaccuracy. Products with moderate accuracy scores estimated with more rigorous tests (for example, evaluated with fully independent data) may be as good as products with high accuracy scores estimated with less rigorous methods (for example, evaluated with nonindependent data). Tests that included data from your region of interest (consideration G) may yield higher confidence in the accuracy score than tests that did not (Smith and others, 2019). See “Evaluating the Accuracy of Spatial Products” for a detailed guide to accuracy.

Step 3—Examine the Map Product within Your Management Area

Overlay IAG maps with other data (for example, IAG point locations, maps of other landscape features) and evaluate the completeness of coverage. Use complementary vegetation maps and high-resolution aerial imagery to assess the degree of agreement among data. Individually, all such information will be incomplete and have limitations (Rocchini and others, 2015; Smith and others, 2019), but together these other data types can indicate areas of higher or lower confidence in IAG maps.

Step 4—Pair the Map Product with Local Information

Local information (for example, field and monitoring data) and field knowledge can help identify how well map products represent on-the-ground conditions. Regional map data are general characterizations of IAGs and should not always be expected to map small locations with high accuracy. Incorporating local information can help identify and account for discrepancies between regional and local patterns. Pairing IAG maps with high-resolution aerial imagery (fig. 2), field data, and local knowledge can help guide the appropriate use of regional spatial products for local applications (Sofaer and others, 2019).

Step 5—Reassess Limitations

To resolve questions about the limitations of IAG spatial data, consider reading the original documentation or contacting the map developer. Use-case examples can also provide insight into common applications and highlight potential limitations.

Types of Measures

Many different accuracy measures are used to evaluate IAG models (Liu and others, 2009). Accuracy measures differ between products that map continuous outputs (for example, percent cover) and those that map categorical outputs (for example, occurrence; table 1). Accuracy may vary locally, but metrics are often reported only for the mapped area overall (Rocchini and others, 2015). For categorical maps, models estimate continuous probabilities, and a threshold cutoff is often selected to classify the landscape (for example, greater than 90-percent probability of presence yields “presence”). These accuracy measures report the prediction’s success dependent on the chosen threshold. Models are designed to optimize accuracy, but they do so in different ways. Some measures focus on correct prediction of actual IAG presence (optimizing true positives or false negatives) because these are relevant to early detection of invasion (Uden and others, 2015). Some models are designed to avoid overpredicting IAG where it is really absent (optimizing true negatives and false positives; Sofaer and others, 2019).

Rigor of Evaluation Dataset

Evaluations using independent data are more rigorous than those that reuse model building data for validation (Sofaer and others, 2019). Within-sample metrics compare model-building data to values predicted by the model and can overestimate accuracy. Cross-validation or bootstrapping uses all data to fit the final published model, but it assesses accuracy by refitting the model just to subsets of the data and seeing how well those models can predict other subsets of the data that are held out from fitting. This approach is less likely to overestimate accuracy. Fully independent evaluations compare model predictions to data never used in model building and are the most rigorous test type.