Skip past header information
USGS - science for a changing world

U.S. Geological Survey Open-File Report 2011-1156

Carolinas Coastal Change Processes Project Data Report for Observations near Diamond Shoals, North Carolina, January-May 2009


Data Processing

Skip past contents information

After the tripods were recovered, proprietary software from each specific instrument manufacturer was used to download sensor measurements from each instrument and decoded to instrument-specific data formats. The data were then converted from instrument-specific formats and calibrated. Units were modified to scientific EPIC-standard units (http://www.pmel.noaa.gov/epic/), and the data were stored in Network Common Data Format (netcdf). EPIC is a set of standards that allow researchers from different organizations to share oceanographic data without having to translate "foreign" data types into the local vernacular. netcdf is a very general, self-documenting, machine-transportable data format created and supported by UCAR ( http://www.unidata.ucar.edu/software/netcdf/). After the conversion, the data were carefully checked for inconsistencies due to instrument malfunctions and biological fouling and then edited to remove these spurious points. Also, the beginning and end of each data series were truncated to remove data collected out of water. The data were carefully checked at each stage of processing. After final editing, the data are considered to be the "best basic version" and include all variables recorded at the basic sampling interval. Best basic versions of most data files in netcdf format are included in this report (see the Digital Data Files section). Some data types (such as ABS and sonar) were processed onlyto the calibration version. Best basic version methods for these data types are being developed.

Details below describe the conversion and calibration process for the equipment specific to this deployment. Additional documentation of the data processing for all instruments as well as a description of the quality-control and review process can be found in Montgomery and others (2008).

Jump to:RD Instruments ADCP
             SonTek ADV
             Paroscientific Digiquartz Pressure Sensor
             Sea-Bird SEACAT
             Sea-Bird MicroCAT
             D&A Instruments OBS
             Aquatec ABS
             Imagenex Rotating Sonar

RD Instruments (RDI) Acoustic Doppler Current Profiler (ADCPTM)

The RDI ADCP water-flow observations were processed using Matlab®-based U.S. Geological Survey software (available at http://woodshole.er.usgs.gov/operations/stg/Pubs/ADCPtools/adcp_index.htm). The ADCP's were normally configured to record data in beam coordinates (rather than earth coordinates). Upon recovery, the ADCP data were transferred to a personal computer using a PCMCIA flash memory card. These data were converted to netcdf format using software available from the ADCP Toolbox (above). Matlab® routines were used to check for data quality, flag values, truncate the data to remove out-of-water data at the beginning and end of the deployment, and discard bins that were always exposed above the water surface. Some near-surface bins were not discarded even though the side-beam reflection at times of low tide renders these data invalid, so near-surface ADCP data must be interpreted with care. Normally, a four-beam solution was used to rotate the data to earth coordinates. When one beam was flagged as bad (according to the instruments data quality indicators), a three-beam solution was used. If two or more beams were bad, the output file has a fill value.

The ADCP wave observations were processed using the RDI proprietary software, WavesMon (see www.rdinstruments.com), and Matlab®-based Wave Data Processing Toolbox software (available at https://pubs.usgs.gov/of/2005/1211/). The WavesMon software package was used to process the raw binary ADCP file and produce a series of data files from which the directional and non-directional wave-energy spectra were calculated. These spectra, in addition to the time-series of wave parameters, were output as a series of ASCII files. These files were converted to netcdf format using the Wave Data Processing Toolbox. This toolbox used a series of Matlab® routines to load the data from the WavesMon-generated ASCII-files, flag values, and truncate data to remove out-of-water data at the beginning and end of the deployment.

SonTek Acoustic Doppler Velocimeter (ADV)

The SonTek ADV water-flow data were also processed using the Hydratools Toolbox developed by the U.S. Geological Survey. The software organized the data files, gathered metadata, converted the data from the SonTek raw binary format to netcdf format, checked data quality, removed noise and poor quality data (according to the instrument data-quality indicators), and rotated the water flow data from beam to geographic coordinates.

Paroscientific (Paros) Digiquartz® Pressure Sensors

The Paros Digiquartz® Pressure sensors deployed for this report were external sensors that were attached to the ADVs (above) and PCADPs (above). Therefore, their data were processed simultaneously with the ADV and PCADP data. The U.S. Geological Survey Hydratools software checked the pressure data for quality and removed any outlying data. The data were then written to the EPIC-standard netcdf files for the instrument (ADV or PCADP) to which the sensors were attached.

Sea-Bird SEACAT and MicroCAT

SEACAT data are stored internally. After recovery of the SEACAT, SEASOFT programs (Sea-Bird Electronics, Inc.) were used to read the data into a file on a personal computer, convert the data to calibrated oceanographic units, calculate salinity and density, and write the data to ASCII files. ASCII files were translated to EPIC-standard netcdf files, and the data were edited for outlying data and truncated to remove out-of-water data at the beginning and end of the deployment.

D&A Instruments Optical Backscatter Sensor (OBS)

The D&A Instruments OBS-3s deployed for this report were external sensors that were attached to the ADVs (above) and PCADPs (above). As such, their data were processed simultaneously with the ADV and PCADP data. The U.S. Geological Survey Hydratools software checked the OBS data for quality and removed any outlying data. The data were then written to the EPIC-standard netcdf files for the instrument (ADV or PCADP) to which the sensors were attached.

Aquatec Acoustic Backscatter Sensor (ABS)

The Aquatec ABS data were processed using Matlab®-based software (available on the Matlab® m-files section) developed by the U.S. Geological Survey. This software decoded the Aquatec raw binary file, flagged data values, and truncated out-of-water data at the beginning and end of the deployment.

Imagenex Rotating Sonar

The Imagenex rotating sonar data were processed using Matlab®-based software (available on the Matlab® m-files section) developed by the U.S. Geological Survey. This software decoded the Imagenex raw binary file, created images from the data, and converted the images to an EPIC-standard netcdf file. The images were then animated using the VideoMach software (available at www.gromada.com).

Skip USGS links group

Accessibility FOIA Privacy Policies and Notices

Take Pride in America logo USAGov logo U.S. Department of the Interior | U.S. Geological Survey
End of USGS links group