SEACOOS Data Management Code Repository

Table of contents

Buoy query and graphs using gnuplot

Going from Datalogger files to 'Point' form on a relational database

DODSFilterColumn

SEACOOS Data Management Code Repository

Table of contents

Recently got into a discussion with at data provider about DODS servers for ASCII data. Researched the DODS FreeForm^?? and JGOFS server a bit and ended up trying the FreeForm^?? server install for Windows. Posting some notes below on the install process. Seems the only sticking point might be that I believe both services prefer data to have the ASCII data columns rigidly fixed in their layout, so CSV or other similarly delimited files are problematic. My suggestion in the absense of a DODS/OPeNDAP developed fix, is using or developing tools/services that filter from delimited to fixed column format.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

Recently got into a discussion with at data provider about DODS servers for ASCII data. Researched the DODS FreeForm^?? and JGOFS server a bit and ended up trying the FreeForm^?? server install for Windows. Posting some notes below on the install process. Seems the only sticking point might be that I believe both services prefer data to have the ASCII data columns rigidly fixed in their layout, so CSV or other similarly delimited files are problematic. My suggestion in the absense of a DODS/OPeNDAP developed fix, is using or developing tools/services that filter from delimited to fixed column format.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

Recently got into a discussion with at data provider about DODS servers for ASCII data. Researched the DODS FreeForm^?? and JGOFS server a bit and ended up trying the FreeForm^?? server install for Windows. Posting some notes below on the install process. Seems the only sticking point might be that I believe both services prefer data to have the ASCII data columns rigidly fixed in their layout, so CSV or other similarly delimited files are problematic. My suggestion in the absense of a DODS/OPeNDAP developed fix, is using or developing tools/services that filter from delimited to fixed column format.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

The code at the above link is designed to be used stand-alone and/or as a web service to allow data providers with column oriented ascii data to create a descriptive Seacoos convention XML document which can then be used to transform those datasets into Seacoos compatible netCDF files for aggregation or archival needs. The XML document can also be used by other groups or software in other aggregation or processing efforts.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

Some sample code demonstrating how data returned from the WFS URL request for the latest station data is processed and merged into an html webpage.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

The script ndbc_fm13.pl is a second script I wrote when the weather instrumentation was added to our buoys. It uses the supplied data arguments when called to populate the FM13 type message.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

SEACOOS Data Management Code Repository

Table of contents

Redeveloped some initial scripts authored by Peter Bense with the following bits of shell and perl code to get a better idea of the load on our server and when things might need more immediate attention.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

• NWS to SEACOOS NetCDF Standard

SEACOOS Data Management Code Repository

Table of contents

• NCCOOS to SEACOOS NetCDF Standard

• SABSOON to SEACOOS NetCDF Standard

• USGS Station Info

SEACOOS Data Management Code Repository

Table of contents

Sara Haines

• NCCOOS to NDBC (FM64)

• SABSOON to NDBC (FM64)

• NOS Raw Data Grab

• NWS Station List and Info

• NWS Raw Data Grab

This is a place to post your OOS related code developments or links. Not sure the best way to organize this - currently I'd say by individual, institution or project/program.

Developed for use by SeaCOOS^? , but the functionality is general enough that the application and the source code are being made available for general use by the community.

The code at the above link is designed to be used stand-alone and/or as a web service to help tranform the DODS client http ascii output(the 'Get ASCII' button) to something more column oriented (CSV Comma Separated Value). This can help the data be more easily viewed and understood as well as fed into other analysis tools, web services and databases. General and technical description available.

A basic html form processed by php with mail notification and appending of corresponding html and csv documents. Zip file should contain all files - change form fields, personnel notified by mail and site references as needed. Article reference http://www.thesitewizard.com/archive/feedbackphp.shtml

Some basic scripts showing the processing of a platform datastreams(directly, via telnet) to population of the relational database and some other side products.

Some basic scripts showing the process of screen/web scraping to create sql statements for populating a relational database.

This perl script takes a platform_id and measurement_date which guides the query selection against the previous population of the database for that measurement timestamp. The returned query values are substituted into a text netCDF file with substitutible macros. ncgen is then run to produce a timestamp prefixed 'latest.nc' file.

Similar process to the previous example but with Screen scraping from NDBC of the input data.

Redeveloped some initial scripts authored by Peter Bense with the following bits of shell and perl code to get a better idea of the load on our server and when things might need more immediate attention.

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

CodeRepository

Redeveloped some initial scripts authored by Peter Bense with the following bits of shell and perl code to get a better idea of the load on our server and when things might need more immediate attention.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

CodeRepository

This perl script takes a platform_id and measurement_date which guides the query selection against the previous population of the database for that measurement timestamp. The returned query values are substituted into a text netCDF file with substitutible macros. ncgen is then run to produce a timestamp prefixed 'latest.nc' file.

ServerStats

Redeveloped some initial scripts authored by Peter Bense with the following bits of shell and perl code to get a better idea of the load on our server and when things might need more immediate attention.

CodeRepository

This perl script takes a platform_id and measurement_date which guides the query selection against the previous population of the database for that measurement timestamp. The returned query values are substituted into a text netCDF file with substitutible macros. ncgen is then run to produce a timestamp prefixed 'latest.nc' file.

Meta-Door

CodeRepository

Some basic scripts showing the process of screen/web scraping to create sql statements for populating a relational database.

Meta-Door

Caro-COOPS is supporting the development of unique data management solutions for coastal ocean observing systems. For example, the "Meta-Door" second-generation online metadata entry tool is now available via the link below. This tool evolved from the "Cast-Net" initiative of the Southern Association of Marine Laboratories (SAML) and the Southeastern Universities Research Association (SURA) with funding from NSF EPSCoR^?. Development of the Cast-Net tool resulted from a cooperative effort among five regional sites: the Baruch Institute for Marine and Coastal Sciences at the University of South Carolina, Dauphin Island Sea Lab Alabama, the Louisiana Universities Marine Consortium, the Skidaway Institute of Oceanography of the University System of Georgia and the University of Southern Mississippi.

CodeRepository

This is a place to post your OOS related code developments or links. Not sure the best way to organize this - currently I'd say by individual, institution or project/program.

JeremyCothran

Web graphics (dials/gauges)

Easy twiki install

Buoy query and graphs using gnuplot

Going from Datalogger files to 'Point' form on a relational database

Basic php registration page

Meta-Door

CodeRepository

Basic php registration page

Meta-Door

CodeRepository

Intro

It was also suggested that your code license info(GNU, etc) should be placed at the top of your code to prevent companies from placing their own licenses on the code which might work against you.

Web graphics (dials/gauges)

-- JeremyCothran - 21 May 2025

CodeRepository

This is just a place to post your OOS related code developments or links. Not sure the best way to organize this - currently I'd say by individual, institution or project/program.

JeremyCothran

Web graphics (dials/gauges)