Illinois State Geological Survey
615 East Peabody Drive
Champaign, IL 61820
Telephone: (217) 244-2513
Fax: (217) 333-2830
email: nelson@isgs.uiuc.edu
The Illinois Natural Resources Geospatial Data Clearinghouse project is a multi-agency effort, led by the Illinois State Geological Survey (ISGS), to make available on the Internet digital geospatial data and associated documentation (metadata) concerning Illinois natural resources. The project was initiated in September 1996, and the prototype Illinois Clearinghouse began operation on July 1, 1997. The primary goal of this on-going effort is to foster a climate for the cooperative development of a statewide clearinghouse network in Illinois by promoting the advantages of the National Spatial Data Infrastructure (NSDI) via a highly visible, operational prototype. The NSDI is a network of computers and agencies that work to reduce redundant data collection and increase data distribution by cooperatively producing and sharing uniform, searchable metadata catalogs about available geospatial data. The growth of the NSDI is guided by the Federal Geographic Data Committee (FGDC), whose charge is the development of standards, data clearinghouses, local frameworks for data sharing, and cooperative partnerships. Thus far, the Illinois Clearinghouse has received a great deal of attention, and, in the spirit of the NSDI and FGDC, is providing unprecedented access to Illinois digital data. The Illinois Clearinghouse is available at http://www.isgs.uiuc.edu/nsdihome/ISGSindex.html.
The participants in this project are the ISGS, the Illinois Natural History Survey (INHS), the Illinois State Water Survey (ISWS), the Illinois Waste Management and Research Center (WMRC), collectively known as the Illinois Scientific Surveys, and the Illinois State Museum (ISM), the Office of Mines and Minerals (OMM), and the Office of Realty and Environmental Planning (OREP) of the Illinois Department of Natural Resources (DNR). Each participant has contributed metadata and digital geospatial data to the Illinois Clearinghouse, making data available on topics such as the Public Land Survey System, bedrock and Quaternary geology, wetlands and streams, landfills, fish and wildlife areas, land cover, political boundaries, municipal boundaries, roads, and railroads.
The ISGS and several of the other participants have been using Geographic Information System (GIS) technology since 1983 to develop and analyze natural resource information. The system has been used for a wide variety of projects, many in partnership with other government agencies, academia, industry, and citizen groups. Typically, the work has been done using Arc/Info software from Environmental Systems Research Institute, Inc. (ESRI, 1997). One result of these projects has been the accumulation of a large collection of digital geospatial data appropriate for addressing a wide range of natural resource issues.
Until recently, most of these data sets were accessible only to agency staff and a limited number of specialized data consumers. During the last few years, however, decreasing costs of computer hardware and GIS software, increasing Internet connectivity, and the advent of new, user-friendly software have significantly increased the use of GIS technology in the participating agencies and their external customers. This has fostered an unprecedented demand for digital geospatial data. The project partners initially addressed the new demand in two primary ways: by independent direct distribution to end-users, and by cooperative development of a two-volume CD-ROM set of Illinois digital databases (Illinois DNR, 1996). A prototype Illinois Natural Resources Geospatial Data Clearinghouse was the next logical step. The intent was not only to meet existing data distribution needs, but also to provide an operational example of the NSDI clearinghouse system in Illinois that would encourage other agencies to recognize the value of participation in data development and sharing initiatives. The ideal result would be a coordinated statewide clearinghouse system involving the wide variety of organizations that generate, manage and use digital geospatial data. While great progress has been made, such a goal will require several more years of effort. We consider this project the first phase, providing initial impetus and a successful model for subsequent efforts.
The fundamental component of a NSDI clearinghouse is a detailed, searchable metadata database that makes data easily discoverable and accessible. Thus, the primary project objectives were: (1) to adopt a specific format for metadata compliant with the Content Standard for Digital Geospatial Metadata (CSDGM) (FGDC, 1995), (2) to implement the clearinghouse with a robust metadata database and downloadable data available through searching and browsing functions, and (3) to develop Fgdcmeta.aml, a streamlined variant of the Arc/Info metadata collection program Document.aml (ESRI, 1995). The purpose of developing Fgdcmeta.aml was to produce an Arc/Info-based metadata collection program for the efficient generation of metadata files that would conform with existing FGDC metadata preparation, indexing, and server tools.
Methods for metadata collection and dissemination are in the early stages of development. We wished to provide as much metadata as possible on our NSDI node using a flexible approach that could accommodate the evolution of this young discipline. Three major issues were (1) the adoption of a specific metadata format, (2) the amount and type of training to provide, and (3) the metadata tools to choose. The following discussion of these issues is taken in large part from the interim project report (Nelson, 1997).
The CSDGM has been in revision over the past two years and indications are that it will not change drastically. Some metadata elements will be redesignated as "core," "recommended if applicable," and "optional," or something similar, and a standard method of adding "user-defined" metadata elements will be instituted. These changes will give users of the standard more freedom in the way they choose to apply it, while maintaining uniformity and utility. Existing CSDGM-compliant metadata should comply "as is" with the revised standard. Nonetheless, the project partners decided that to formally adopt a specific metadata format based on a soon-to-be-replaced standard would be ill-advised. They informally agreed to produce FGDC-compliant metadata using the set of elements that had previously been identified for use with the Illinois DNR digital data CD-ROM set. This metadata format consists of the Identification Information and Metadata Reference sections of the CSDGM, and substantial parts of other sections as applicable. Although the difference is subtle, proceeding in this manner leaves the participants in a position to better assess and recommend a formal metadata format for Illinois data after the revisions to the CSDGM are complete.
Project staff both received and provided training. They attended a half-day orientation on clearinghouse mission and goals, and how clearinghouse software and data structures utilize properly formatted metadata files for search and presentation. Core project staff attended intensive training classes and have given presentations at several local and regional meetings where the FGDC, clearinghouse and metadata were primary issues. Project partners have participated in several meetings to promote the clearinghouse concept. Most recently, a hands-on metadata training class highlighting the Wisconsin Metadata Primer (Hart and Phillips, 1997) was provided to staff of the Illinois Scientific Surveys.
However, large-scale training of staff not directly involved with the project has been kept to a minimum. The tools and techniques needed by metadata developers are not necessarily those needed by data developers or data users. It was considered prudent to first establish the prototype clearinghouse, assess the results, and refine the product. Then the response of data developers and users could be evaluated to determine the type and scope of training required. Also, the CSDGM is being revised and training of non-project staff was judged not to be an immediate necessity. It was deemed more efficient to wait for the release of the revised metadata standard than to provide training in the current version only to re-tool and retrain for the subsequent version.
Tools for metadata creation were chosen based on the previous experiences of the various participants. Many metadata files were produced prior to the project using tools such as word processing templates, Document.aml, and Xtme (Xt Metadata Editor) (Schweitzer, 1997c). Existing metadata files that were originally produced in this manner for the Illinois DNR digital data CD-ROMs were reformatted using cns (Chew and Spit) (Schweitzer, 1997a) and augmented to comply with clearinghouse requirements. Most of the tools tested were used on a UNIX computer system, but some PC tools were tested also. Limitations experienced with these various approaches led to the development of the Fgdcmeta.aml metadata collection program.
Fgdcmeta.aml was derived using programming code extracted from Document.aml. It was created because Document.aml has some fundamental problems related to text editing processes. In essence, Fgdcmeta.aml uses the core data extraction routines of Document.aml to derive descriptive information from an Arc/Info data set, but employs a much simpler text editing interface for ancillary data entry. Fgdcmeta.aml has proven to be straightforward and efficient in the generation of metadata from Arc/Info data sets for the Illinois project. Several other organizations have also found it useful, including the United States Geological Survey (USGS), the Geological Survey of Alabama, and the Bureau of Land Management (BLM). Unfortunately, and in-depth discussion of the development of Fgdcmeta.aml cannot be presented here. For more information, refer to the article entitled Arc/Info Solutions to Metadata Problems: Building a Solid NSDI Clearinghouse Node on a Shifting Metadata Landscape (Nelson and others, 1997). The article and the Fgdcmeta.aml program are available at the Illinois Clearinghouse web site. An independent review of the program (Phillips, 1997) is available at http://badger.state.wi.us/agencies/wlib/sco /metatool/mtools.htm.
Through trial and error, the following metadata generation process was developed: Fgdcmeta.aml is used to gather initial, data-specific information from Arc/Info data sets and produce a CSDGM-compliant template file in ASCII format. The template contains generalized institutional, distribution and contact information, as well as headings and input fields for additional metadata elements that require manual data entry. Any text editor can be used to provide the additional information; Xtme is recommended. The files are then processed with mp (Metadata Parser) (Schweitzer, 1997b) to generate SGML- and HTML-formatted files in preparation for indexing and serving the metadata. Files are indexed using Isite software (CNIDR, 1997) and served on the Internet with the WAIS Z29.50 protocol. The software tools mentioned here (other than Fgdcmeta.aml) are available free of charge at the FGDC web site (http://www.fgdc.gov) for most of the common computer platforms.
The clearinghouse was brought on-line on July 1, 1997, with over 1,800 downloadable GIS data sets described by over 100 complete metadata documents. An additional 100 documents containing minimal information comprise a metadata working list. The data and metadata are accessible through browse and search functions. Z-server (WAIS) software was brought on-line simultaneously, giving users the ability to search the metadata database either remotely from the primary NSDI gateway or locally by navigating directly to the Illinois Clearinghouse. The downloadable data have since been augmented with a catalog of short abstracts, metadata links, and over 110 GIF images giving graphic portrayals of the data. Links to various other geospatial data applications in state government have been added.
The most significant result for the participating agencies is that we are providing unprecedented access to digital data. Our participation in the NSDI Clearinghouse System gives us a very visible forum through which to further distribute information, data, and methodologies. The technical advantages of a Clearinghouse allow us to easily and significantly increase distribution and to provide most of the data free of charge. From July 1997 to March 1998, the Illinois Clearinghouse had 180,000 hits and 9,450 user sessions, averaging 39 users per day. Metadata documents were accessed over 5,000 times and 15,000 data sets (7.5 gigabytes) were downloaded. The metadata database was searched several hundred times through the Z-server housed at the primary NSDI clearinghouse node at the USGS. Although Internet statistics of this sort are imprecise, they indicate that the Illinois Clearinghouse is receiving a great deal of attention and use. For existing customers, we have provided easier data access and a larger data catalog than previously available. The real impact, however, is for those new customers who, until now, had no idea of the breadth and depth of our digital data holdings.
Also, in some of the participating organizations this effort has provided the impetus to greatly reorganize and enhance existing GIS databases. As a result, data maintenance, update and control policies have been improved, and our geospatial data sets are better prepared to move into the mainstream of traditional digital information systems.
Perhaps the most significant result in terms of the overall NSDI clearinghouse system is that NSDI-node-in-a-box works; the Illinois Clearinghouse is a textbook example. "Node-in-a-box" is a FGDC catch-phrase that is intended to convey the relative ease with which a NSDI clearinghouse can be assembled. The implication is that a great deal of the initial experimental work has been done and that interested parties need only download the required software via the Internet and follow step-by-step instructions. For the Illinois project it was almost that easy. There were, of course, some technical difficulties; every computer system has its unique challenges. Technically adept personnel are essential, and a robust computer network with high-speed Internet connectivity is an absolute requirement. Ultimately, though, thanks to node-in-a-box technology, project staff spent minimal time bringing the metadata indexing and server software up to operational status. This allowed for maximum concentration on the truly important aspects of the project: metadata creation, catalog development, and digital data sharing.
In terms of metadata creation, there is an important and simple lesson to be learned, and it is echoed frequently by novice metadata compilers. The CSDGM is complex. It takes a great deal of time and effort to gain expertise. Without in-depth study, the meanings of many metadata elements often seem unclear or redundant. The Illinois project was fortunate in this regard because some of the participants had prior metadata experience. Less time had to be spent learning the standard, so more time could be spent implementing it. Nonetheless, a warning to new users of the CSDGM is appropriate: when learning to use the metadata standard, recognize that it is indeed complex, as are most powerful tools. Be prepared, at first, to write and rewrite the same metadata documents reiteratively. After all, few people can ride a bike on the first try, or write the perfect report in the first draft. Repetition is an unavoidable feature of the learning process. It is not unique to metadata creation and, if planned for, should not be a source of frustration. As expertise with metadata increases, so does an understanding of the dynamic nature of metadata and the relationships between individual metadata documents. Such insights often suggest modifications to existing metadata. A constructive way to approach this situation is to institute periodic reviews of data and the associated metadata. This is not only good database maintenance practice, but is also an excellent application of metadata!
The greatest challenge of metadata and clearinghouse implementation is convincing those with no budget for it that it is a worthwhile activity. Many approaches have been suggested to address this issue. In this project the approach was simple: partner with supportive organizations, build the prototype clearinghouse, and use it to promote its own utility. If the data are useful, the demand will be demonstrable and the benefits compelling. In fact, the demand may be too great. One contributor to the Illinois Clearinghouse asked that his metadata be temporarily removed, stating, "I'm getting too many requests for the data, and I'm not quite ready to distribute it!"
The Illinois initiative is now positioned for the next phase. We intend to use the clearinghouse as a self-promotional tool in pursuit of a broader base of institutional support and participation, and new funding for maintenance and expansion. We will continue to build our metadata database and add downloadable digital data sets. We have plans to add 10-30 gigabytes of storage space from which we will serve, free of charge, raw and modified USGS Digital Raster Graphic (DRG) files for every 7.5 minute quadrangle in the state. We also intend to complement digital data with finished cartographic products by creating an on-line browsing gallery of published and openfile maps. Ultimately, we hope the Illinois Natural Resources Geospatial Data Clearinghouse will be the first step toward a cooperative and comprehensive geospatial data framework in Illinois. Such an "Illinois Geospatial Data Framework" would provide uniform map bases and techniques with which spatial data developers and users would build shared solutions to common natural resource and civil planning needs.
The following individuals made significant contributions to this project: Galen Arnold and Sally Denhart of the Illinois State Geological Survey; Jill Blanchar, National Center for Supercomputing Applications (formerly of the Illinois Waste Management and Research Center); Mark Joselyn, Illinois Natural History Survey; Kingsley Allan, Illinois State Water Survey; Erich Schroeder, Illinois State Museum; Ray Druhot, Illinois Office of Mines and Minerals; and Will Hinsman and Sheryl Oliver, Illinois Office of Realty and Environmental Planning. Funding for this project was provided by the United States Geological Survey under the auspices of the FGDC Competitive Cooperative Agreements Program (CCAP), and by the Illinois State Geological Survey. The Illinois Clearinghouse participants owe a great debt of thanks to the grantees of the 1994 and 1995 CCAP awards for their efforts in laying the foundation for our work. The authors thank Jon Goodwin, Jennifer Hines, Don McKay, and Colin Treworgy of the ISGS, and David Soller of the USGS for their helpful comments in reviewing this manuscript.
Center for Networked Information Discovery and Retrieval, 1997, Isite software: Research Triangle Park, North Carolina, CNIDR, http://cnidr.org/welcome.html.
Environmental Systems Research Institute, Inc., 1995, Document.aml (several versions through ESRI version 7.0.4): Redlands, California, ESRI, original programming by D. Nebert and M. Negri (United States Geological Survey), and M. Hoel (ESRI).
Environmental Systems Research Institute, Inc., 1997, Arc/Info GIS software: Redlands, California, ESRI, http://www.esri.com.
Federal Geographic Data Committee, 1995, Content standards for digital geospatial metadata workbook (March 24): Washington, D.C., Federal Geographic Data Committee.
Hart, D., and Phillips, H., 1997, Metadata primer: National States Geographic Information Council and University of Wisconsin-Madison, Madison, Wisconsin, http://www.lic.wisc.edu/metadata/metahome.htm.
Illinois Department of Natural Resources, 1996, Illinois Geographic Information System CD-ROM of Digital Datasets of Illinois: Springfield, Illinois, Illinois Department of Natural Resources, 2 vols.
National States Geographic Information Council, 1997, A practical guide to metadata implementation for GIS/LIS professionals: A national satellite videoconference: Hanover, New Hampshire, National States Geographic Information Council, http://www.lic.wisc.edu/metadata/metasat.htm.
Nelson, D.O., 1997, Six-month project report for the 1996 Competitive Cooperative Agreements Program (CCAP) project entitled: Illinois Natural Resources -- A prototype National Geospatial Data Clearinghouse Node: Champaign, Illinois, Illinois State Geological Survey, unpublished.
Nelson, D.O., Krumm, R.J., Denhart, S.L., and Beaverson, S.K., 1997, Arc/Info solutions to metadata problems: Building a solid NSDI Clearinghouse Node on a shifting metadata landscape: Proceedings of the 1997 ESRI Annual User Conference, San Diego, California.
Phillips, H., 1997, Metadata tools for geospatial data: Madison, Wisconsin, Wisconsin NSDI Clearinghouse and National States Geographic Information Council, http://badger.state.wi.us/agencies/wlib/sco/metatool/mtools.htm.
Schweitzer, Peter, 1997a, Tools for creation of formal metadata -- cns [chew and spit], a pre-parser for formal metadata: U.S. Geological Survey web site (http://geology.usgs.gov/tools/metadata) (web site last updated July 20, 1998).
Schweitzer, Peter, 1997b, Tools for creation of formal metadata -- mp, a compiler for formal metadata: U.S. Geological Survey web site (http://geology.usgs.gov/tools/metadata) (web site last updated July 20, 1998).
Schweitzer, Peter, 1997c, Tools for creation of formal metadata -- Xtme, an editor for formal metadata: U.S. Geological Survey web site (http://geology.usgs.gov/tools/metadata) (web site last updated July 20, 1998).
U.S.Department of the Interior, U.S. Geological Survey
<https://pubs.usgs.gov/openfile/of98-487/nelson.html>
Maintained by Dave Soller
Last updated 10.06.98