SEACOOS Data Management Code Repository

Table of contents

Intro

This is a place to post your OOS related code developments or links. Not sure the best way to organize this - currently I'd say by individual, institution or project/program.

Jeremy Cothran

Web graphics (dials/gauges)

Web graphs (time series plots) using GD::Graph

Easy twiki install

Online Data Dictionary utility

Developed for use by SeaCOOS^? , but the functionality is general enough that the application and the source code are being made available for general use by the community.

Buoy query and graphs using gnuplot

Going from Datalogger files to 'Point' form on a relational database

DODSFilterColumn

The code at the above link is designed to be used stand-alone and/or as a web service to help tranform the DODS client http ascii output(the 'Get ASCII' button) to something more column oriented (CSV Comma Separated Value). This can help the data be more easily viewed and understood as well as fed into other analysis tools, web services and databases. General and technical description available.

Basic php registration page

A basic html form processed by php with mail notification and appending of corresponding html and csv documents. Zip file should contain all files - change form fields, personnel notified by mail and site references as needed. Article reference http://www.thesitewizard.com/archive/feedbackphp.shtml

PlatformToDatabase

Some basic scripts showing the processing of a platform datastreams(directly, via telnet) to population of the relational database and some other side products.

ScreenScraping

Some basic scripts showing the process of screen/web scraping to create sql statements for populating a relational database.

Generating netCDF files for Seacoos aggregation

This perl script takes a platform_id and measurement_date which guides the query selection against the previous population of the database for that measurement timestamp. The returned query values are substituted into a text netCDF file with substitutible macros. ncgen is then run to produce a timestamp prefixed 'latest.nc' file.

Screen Scraping (NDBC) for Seacoos aggregation

Similar process to the previous example but with Screen scraping from NDBC of the input data.

ServerStats

Redeveloped some initial scripts authored by Peter Bense with the following bits of shell and perl code to get a better idea of the load on our server and when things might need more immediate attention.

NDBC Message Encoding for FM64, FM13 type messages

I use the attached perl scripts(ndbc.pl, ndbc_fm13.pl) which take the data we get from the platforms and convert them to the ascii message format required which calls the NDBC supplied encoder function to encode the message to a binary format which is then ftp'd to a Carocoops directory.

I'm also attaching the two sample message formats which we use with NDBC. FM13 is the message format for weather observations and water temperature at the surface, while FM64 is a newer message format for ADCP current type information at different bin depths and another 3 variables for depth,temperature and salinity.

Rana wrote the original 'ndbc.pl' script which runs a query against our database to populate a FM64 type message. This was back when we didn't have any weather data coming in from our buoys and I had tables set up by platform instead of by variable. This script could be adapted with the necessary data arguments supplied from the command line or in a file instead of from a database query.

The script ndbc_fm13.pl is a second script I wrote when the weather instrumentation was added to our buoys. It uses the supplied data arguments when called to populate the FM13 type message.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

The Cast-Net metadata tool was developed to provide a convenient and user-friendly mechanism for generating FGDC-compliant metadata. Increased and broad-based use of Cast-Net demonstrated the need for enhanced capabilities and more comprehensive capacities for data and system documentation. Meta-Door is a complete regeneration of the Cast-Net metadata tool designed to handle the needs of a broad range of contemporary users. Meta-Door resembles Cast-Net in appearance and functionality while providing an enhanced user driven file management system and an easier, more reliable application interface. It has been completely re-programmed to work with a relational database that will eventually allow for more sophisticated indexing services. Future versions of Meta-Door will allow users to incorporate additional metadata standards as new standards emerge and become widely used in the ocean coastal observing communities.

Meta-Door has been rebuilt from the ground up with Java Struts to be database- and OS-independent. New features include:

• User administration with simplified registration and profile management
• Additional metadata formatting options for review
• Option to remove published metadata from ISite
• Lightweight GIS for input of Geo-location information
• User interface improvements, including collapsible form layout

Users may register and use the Meta-Door system for entering their metadata records at the following link.
http://www.carocoops.org/metadoor

Meta-Door demos and sourcecode download
http://www.carocoops.org/resources/MetaDoor_CD

Sara Haines

• MATLAB Tools for NetCDF Files

• NCCOOS to SEACOOS NetCDF Standard

• NCCOOS data push to NDBC

• SABSOON to SEACOOS NetCDF Standard

• SABSOON data push to NDBC

• USGS Station Info

• NOS Station List and Info

• NOS Raw Data Grab

• NOS to SEACOOS NetCDF Standard

• NWS Station List and Info

• NWS Raw Data Grab

• NWS to SEACOOS NetCDF Standard

to top