Skip to topic | Skip to bottom
Home
Main
Main.ObsKMLr1.66 - 13 Apr 2009 - 15:58 - JeremyCothrantopic end

Start of topic | Skip to actions

ObsKML

UPDATE: December 4, 2008 this schema references the 'Metadata' tag from the earlier version of KML, the latest version of KML 2.2 specifies using the 'ExtendedData' tag instead http://code.google.com/apis/kml/documentation/extendeddata.html which should be an easy translation to provide an example of. If anyone has an interest in such an encoding please contact me at jeremy.cothran[at]gmail.com

UPDATE: March 27, 2008 Xenia documentation is being migrated to http://code.google.com/p/xenia/wiki/XeniaHome

see also XeniaPackageSqlite and JCNotes

A discussion page for an observations oriented KML. See also this developer article .

This webpage can also be linked to with the shorter url http://tinyurl.com/4mkwfc

KML currently handles spatial and temporal tags, but not data content. The below is an attempt to devise a content schema and mapping which would allow not only the display of data content, but also some meaningful data sharing within the observations community using KML/KMZ as a data transport mechanism. The kml tag used below is the 'Metadata' tag which carries the content in a more formalized xml schema.

Anyone wishing to share their data observation similarly, please email jeremy.cothran[at]gmail.com . I will maintain a registry of data observations available in this format to promote the format. I can also act to collect and promote additional lookup entries(observation and unit of measure types, etc) as needed. Examples of perl scripts which using XML::LibXML to reprocess xml feeds into ObsKML are located here

ObsKML Design Goals

The current design goals of ObsKML are to provide:

  • a minimal common XML schema/vocab for sharing latest in-situ data on a report basis
  • a low entry cost facilitator/catalyst to other aggregations/products/services
  • some default analysis/visualization through some simple XML reprocessing/restyling http://carocoops.org/gearth/latest_placemarks.kmz for KML based products like Google Earth/Maps

or conversely to avoid the opposite of the above - stalled out technical specifications which never cross the field from expert debate and testbeds to actual utility in part due to an overly complex services/query only focus rather than a simpler data/document one. Popular data and data formats are primary critical issues - software and services are dependent and secondary considerations.

Another way of looking at this is towards a REST or Resource Oriented Architecture(ROA) approach(a file/format based approach) to sharing data as contrasted with a Service Oriented Architecture(SOA) web services based approach.

See also Benjamin Carlyle's web design thoughts at lessons of the web and the war between REST and WS-*

excerpted from from the Carlyle link below

Prefer low-semantic-precision document types over newly-invented document types
I think this is one of the most interesting lessons of the Web. The reason for the success of the Web is that a host of applications can be added to the network and add value to the network using a single basic content type. HTML is used for every purpose under the sun. If each industry or service on the Web defined its own content types for communicating with its clients we would have a much more fragmented and less valuable World-Wide-Web.

Schema simple

http://tinyurl.com/a9q8wf reference to this section

see also http://code.google.com/p/xenia/wiki/InstrumentationExamples

The below sample kml file shows an example of two observations listed at a platform. The xml schema is available here.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1"
     xmlns:obskml="http://carocoops.org/obskml/1.0.0/obskml_simple.xsd">
<Document>
  <name>Obskml sample</name>
  <open>1</open>
  <Placemark id="ndbc.41024.buoy">
    <name>ndbc.41024.buoy</name>
    <description>An html table derived from the obs kml metadata tags would be displayed here</description>
    <Point>
      <coordinates>-77.0,32.0,0</coordinates>
    </Point>
    <TimeStamp><when>2009-03-14T16:00Z</when></TimeStamp>

    <ExtendedData xmlns:obskml="http://carocoops.org/obskml/1.0.0/">
    <obskml:obsList>
    <obskml:obs>
      <obskml:obsType>air_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>21</obskml:value>
      <obskml:elev>3</obskml:elev>
    </obskml:obs>

    <obskml:obs>
      <obskml:obsType>water_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>16</obskml:value>
      <obskml:elev>-1</obskml:elev>
    </obskml:obs>
    </obskml:obsList>
    </ExtendedData>

  </Placemark>
</Document>
</kml>

The above schema represents two observations from a single platform.

The 'elev' tag in the above schema corresponds to elevation. Positive is meters above sea level and negative is meters below sea level. Missing or unknown elevation values can leave an empty elev tag like <elev />

A recommended but not required convention is to use the Placemark id attribute to help uniquely identify observations between data providers. The recommended convention is the concatenation of <operator>.<platform>.<package> like carocoops.CAP1.buoy or nws.KCAE.met

Observation listing order is preferred to be from highest elevation/altitude to lowest elevation/depth.

Redundant sensors/observations would be listed in their order of importance (primary, secondary, etc).

Schema simple JSON

Here's a crude initial attempt at describing ObsKML as JSON over several hourly reports.

{
    "schema": "urn:secoora.org:schema_obs_geojson#",
    "dictionary": "urn:secoora.org:dictionary_geojson#",
    "operatorURL": "http://www.ndbc.noaa.gov",
    "platform": [
        {
            "platformName": "ndbc.41024.buoy",
            "platformURL": "http://www.ndbc.noaa.gov/station_page.php?station=41024",
            "obs": [
                {
                    "type": "Point",
                    "coordinates": [
                        -77.0,
                        32.0 
                    ],
                    "timeStamp" : {
                        "when": "2009-03-14T16:00:00Z" 
                    },
                    "summary": "Air Temp:22 degrees Celsius, Water Temp: 17 degrees Celsius, ...",
                    "obsList": [
                        {
                            "obstype": "air_temperature",
                            "uomtype": "celsius",
                            "value": "22",
                            "elev": "3" 
                        },
                        {
                            "obstype": "water_temperature",
                            "uomtype": "celsius",
                            "value": "17",
                            "elev": "-1" 
                        } 
                    ] 
                },
                {
                    "type": "Point",
                    "coordinates": [
                        -77.0,
                        32.0 
                    ],
                    "timeStamp" : {
                        "when": "2009-03-14T15:00:00Z" 
                    },
                    "summary": "Air Temp:21 degrees Celsius, Water Temp: 16 degrees Celsius, ...",
                    "obsList": [
                        {
                            "obstype": "air_temperature",
                            "uomtype": "celsius",
                            "value": "21",
                            "elev": "3" 
                        },
                        {
                            "obstype": "water_temperature",
                            "uomtype": "celsius",
                            "value": "16",
                            "elev": "-1" 
                        } 
                    ] 
                } 
            ] 
        } 
    ]
}

Location follows GeoJSON? formatting and time follows KML time schema tags/conventions. Additional tag-specific related links/info could be listed at the following levels:

  • operator(data provider) top level (like operatorURL)
  • platform level (like platformURL)
  • obs level (like elev)

Above encoding could contain one or more platforms/packages reporting one or more time/locations. summary field is optional, just to support RSS style free-form text output to readers, etc.

Used the following links for examples, validation
http://www.ibm.com/developerworks/library/x-atom2json.html
http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html?page=1
http://www.jsonlint.com/

Simple schema JSON - Alternate 1 - ObsJSON?

Update April 13, 2009 The following documentation regarding ObsJSON? has been copied and being actively maintained/updated at http://code.google.com/p/xenia/wiki/ObsJSON

Below example is more flat/array oriented like netCDF,CSV and could support moving platforms(ships,gliders) as well as stationary ones.

elevRel is the 'relative elevation' in meters height(+) or depth(-) from the given GeoJSON? z coordinate(s) with zero z coordinate corresponding to mean sea level(MSL).

The ordering of effected list arguments is in time increasing order(oldest first, latest last) allowing picking off the latest value be grabbing the last associated set of time/values off the list.

Stationary platforms would use GeoJSON? 'Point' type and mobile platforms would use 'MultiPoint' type. Multipoint coordinates are paired with listed time values.

Observation listing order is preferred to be from highest elevation/altitude to lowest elevation/depth.

Redundant sensors/observations would be listed in their order of importance (primary, secondary, etc) or depth(highest to lowest elevation).

{
    "type": "Feature",
    "geometry": {
        "type": "MultiPoint",
        "coordinates": [[-80.55,30.04,0],[-79.00,31.00,0],[-78.00,32.00,0]] 
    },
    "properties": {
        "schemaRef": "ioos_blessed_schema_name_reference",
        "dictionaryRef": "ioos_blessed_obstype_uom_dictionary_reference",
        "dictionaryURL": "http://nautilus.baruch.sc.edu/obsjson/secoora_dictionary.json",
        "operatorName": "ndbc",
        "operatorURL": "http://www.ndbc.noaa.gov/",
        "platformName": "41012",
        "platformURL": "http://www.ndbc.noaa.gov/station_page.php?station=41012",
        "stationId": "urn:x-noaa:def:station:noaa.nws.ndbc::41012",
   
        "time": ["2009-03-31T10:50:00Z","2009-03-31T11:50:00Z","2009-03-31T12:50:00Z"],

        "obsList": [
            {
                "obsType": "air_temperature",
                "uomType": "celsius",
                "valueList": ["22.0","23.0","24.0"],
                "elevRel": "3" 
            },
            {
                "obsType": "water_temperature",
                "uomType": "celsius",
                "valueList": ["17.0","18.0","19.0"],
                "elevRel": "-1" 
            } 
        ] 
    } 
} 

Example GeoJSON? embedded in KML as atom:link

Could also use the KML TimeSpan? tag below as well (especially if only referencing TimeSpan? files only)

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1"
     xmlns:obskml="http://carocoops.org/obskml/1.0.0/obskml_simple.xsd">
<Document>
  <name>Obskml sample</name>
  <open>1</open>
  <Placemark id="ndbc.41024.buoy">
    <name>ndbc.41024.buoy</name>
    <description>An html table derived from the obs kml metadata tags would be displayed here</description>
    <Point>
      <coordinates>-77.0,32.0,0</coordinates>
    </Point>
    <TimeStamp><when>2009-03-14T16:00Z</when></TimeStamp>

    <ExtendedData>

   <!-- GeoJSON related link(latest obs) -->
   <atom:link type="application/json"
    href="http://myurl/feeds/ndbc/latest/ndbc.41024.buoy.json" />

   <!-- GeoJSON related link (latest obs - past 48 hours -->
   <atom:link type="application/json"
    href="http://myurl/feeds/ndbc/latest_hours_48/ndbc.41024.buoy.json" />

    </ExtendedData>

  </Placemark>
</Document>
</kml>

Schema simple AtomPub?

...
<entry>
   <!-- platform metadata/links at this schema level -->
   <id>latest</id>
   <title>ndbc.41024.buoy</title>
   <updated>2009-03-14T16:00:00Z</updated>
   <georss:where>
     <gml:Point><gml:pos>-77 32</gml:pos></gml:Point>
   </georss:where>

   <!-- GeoJSON related link -->
   <link rel="alternate"
    type="application/json"
    href="http://myurl/feeds/ndbc/ndbc.41024.buoy_latest.json" />

   <!-- Human-readable summary -->
   <summary>Air Temp:21 degrees Celsius, Water Temp: 16 degrees Celsius, ...</summary>

   <content type="text/xml">
      
     <!-- NDBC 41024 data here -->
    <obskml:obsList>

    <obskml:platformName>ndbc.41024.buoy</obskml:platformName>
    <obskml:platformURL>http://www.ndbc.noaa.gov/station_page.php?station=41024</obskml:platformURL>
    <obskml:operatorURL>http://www.ndbc.noaa.gov<obskml:operatorURL>

    <obskml:obs>
      <obs/sensor specific metadata/links at this schema level -->
      <obskml:obsType>air_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>21</obskml:value>
      <obskml:elev>3</obskml:elev>
    </obskml:obs>

    <obskml:obs>
      <obskml:obsType>water_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>16</obskml:value>
      <obskml:elev>-1</obskml:elev>
    </obskml:obs>

    </obskml:obsList>   
   </content>
</entry>

<entry>
   <!-- platform metadata/links at this schema level -->
   <id>latest_minus_hour_1</id>
   <title>ndbc.41024.buoy</title>
   <updated>2009-03-14T15:00:00Z</updated>
   <georss:where>
     <gml:Point><gml:pos>-77 32</gml:pos></gml:Point>
   </georss:where>

   <!-- GeoJSON related link -->
   <link rel="alternate"
    type="application/json"
    href="http://myurl/feeds/ndbc/ndbc.41024.buoy_latest_minus_hour_1.json" />

   <!-- Human-readable summary -->
   <summary>Air Temp:22 degrees Celsius, Water Temp: 17 degrees Celsius, ...</summary>

   <content type="text/xml">
      
     <!-- NDBC 41024 data here -->
    <obskml:obsList>

    <obskml:platformName>ndbc.41024.buoy</obskml:platformName>
    <obskml:platformURL>http://www.ndbc.noaa.gov/station_page.php?station=41024</obskml:platformURL>
    <obskml:operatorURL>http://www.ndbc.noaa.gov<obskml:operatorURL>

    <obskml:obs>
      <obs/sensor specific metadata/links at this schema level -->
      <obskml:obsType>air_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>22</obskml:value>
      <obskml:elev>3</obskml:elev>
    </obskml:obs>

    <obskml:obs>
      <obskml:obsType>water_temperature</obskml:obsType>
      <obskml:uomType>celsius</obskml:uomType>
      <obskml:value>17</obskml:value>
      <obskml:elev>-1</obskml:elev>
    </obskml:obs>

    </obskml:obsList>   
   </content>
</entry>
...

The above example shows the last two hourly reports as entries. The id/file name reference convention is 'latest','latest_minus_hour_1','latest_minus_hour_2',...

The example also shows entry alternate link references to the GeoJSON? oriented feeds.

Used the following links for examples
http://sgillies.net/blog/883/sensible-observation-services
http://www.youtube.com/watch?v=T04fKsD56LU

Schema simple AtomPub? version 2

A simpler AtomPub? using just JSON links, in this scenario each entry could reference the available platforms from a given provider(instead of entries corresponding to hourly references from a single platform).

...
<entry>
   <id>latest/ndbc.41024.buoy.json</id>
   <title>ndbc.41024.buoy</title>
   <updated>2009-03-14T16:00:00Z</updated>
   <georss:where>
     <gml:Point><gml:pos>-77 32</gml:pos></gml:Point>
   </georss:where>

   <!-- GeoJSON related link -->
   <link type="application/json"
    href="http://myurl/feeds/ndbc/latest/ndbc.41024.buoy.json" />
</entry>

<entry>
   <id>latest_hours_48/ndbc.41024.buoy.json</id>
   <title>ndbc.41024.buoy</title>
   <updated>2009-03-14T16:00:00Z</updated>
   <georss:where>
     <gml:Point><gml:pos>-77 32</gml:pos></gml:Point>
   </georss:where>

   <!-- GeoJSON related link -->
   <link type="application/json"
    href="http://myurl/feeds/ndbc/latest_hours_48/ndbc.41024.buoy.json" />
</entry>
...

Schema complex

Here is a link to a more complex xml schema for the kml metadata tag, incorporating

  • value descriptive measurement types like for weather - overcast, cloudy, etc

  • organization name
  • organization url
  • platform name
  • platform url
  • data url(s) - access to data in various formats and via various services
  • sensor_order(s_order) - a way to distinguish between redundant sensors primary,secondary,etc
  • quality control flag(s) - is data good, bad, suspect, etc

This additional metadata may or may not be available depending on the dataset and I want to focus on providing at least the minimal simple observation and unit of measure before branching into something more complex.

Generating and Styling ObsKML

Some sample/example cases showing how to generate or style ObsKML from other sources or convert from ObsKML to other formats.

Additional links

The Xenia relational database schema is the working schema we are using to aggregate various in-situ observations to before providing a variety of default output products such as maps, graphs and other csv and xml files.

JCNotes is a running developer log of points of interest, mainly in regards to tools/techniques for sharing observation data ( shorter summary of tools/approaches used ).

Diagram

Link to below image with further active image links
http://www.gliffy.com/publish/1329032/

Related Schemas

Extended Environment ML(simple schema similar to ObsKML, use of folksonomy tags for observation types) - urban application focus - http://www.eeml.org http://pachube.com

Joint METOC Broker Language (JBML) https://wiki.ucar.edu/display/NNEWD/JMBL

Additional schemas

WaterML?

http://river.sdsc.edu/wiki/WaterML.ashx

XBRL

Free the Data: eGov and Open Standards http://web2.sys-con.com/node/875680

XBRL (Extensible Business Reporting Language) http://www.xbrl.org/Specification/XBRL-RECOMMENDATION-2003-12-31+Corrected-Errata-2008-07-02.htm#_list_of_examples

example RESTful implementations

#ERDDAP http://coastwatch.pfeg.noaa.gov/erddap/rest.html

http://www.jasonbirch.com/nodes/2009/01/31/269/mapguide-rest-extension-feedback-wanted/

Related Links

Sean Gilies blog
geo-web-rest listserv

Charlie Savage blog
http://cfis.savagexi.com/articles/2007/08/13/why-is-rest-so-hard-to-understand
http://cfis.savagexi.com/articles/2007/08/14/resources-and-representations-redux
http://cfis.savagexi.com/articles/2007/11/09/its-the-links-stupid

http://www.infoq.com/articles/rest-introduction
http://en.wikipedia.org/wiki/Resource_oriented_architecture

A Simpler Approach To SOA pdf
REST Questions

Government and the Invisible Hand
REST-Web-Services

http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/JCNotes#REST_vs_SOAP_document_vs_protoco

http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

link from John Graybeal at MMI http://www.w3.org/2007/04/wsec_report

#WS-* vs the REST (April 2006)
http://www.theregister.co.uk/2006/04/29/oreilly_amazon/
I think there are also some political aspects. Early in the web services discussion, I remember talking with a SOAP architects at a large company. He let slip that it was actually the company objective to make the standard sufficiently complex that only the tools would read and write this stuff, and not humans. So that was a strategy tax that was imposed by the big companies on some of this technology, where they made it more complicated than it needed to be so they could sell tools. Itís not necessarily just Machiavellian scheming. I think companies really believe that you can create better user experiences with tools that give people so much more power. But it is also a business strategy, whereas a lot of innovation comes when things are simple enough for people just to try them out and jam against them, if you like, to use a jazz term. If you look at a lot of innovations in the computer industry, they come when something is simple and the barriers to experimentation are low, and not from big company standards-driven top-down initiatives. My point is that it is really too early to build some of the things that people have been trying to push out as web services, and right now simpler is better, because people are just figuring stuff out from the bottom up. ==
Barr on the debate
The trend is towards simplicity and getting applications running quickly. That implies away from SOAP and more toward REST, and even toward more lightweight protocols. There's a new protocol we're looking at called JSON, the Javascript Object Notation. Developers see the standards as helping them a little bit, but they're not all that put off if it's something custom. They want efficient development, efficient processing. It's approximately 20 per cent SOAP, 80 per cent REST.

#soap vs rest developer discussion thread (January 2007)
http://developer.amazonwebservices.com/connect/message.jspa?messageID=50707

#.NET WCF (supporting both SOAP and REST/JSON/RSS) (November 2007)
http://blogs.msdn.com/dotnetinterop/archive/2007/11/13/soap-rest-whichever-you-choose-wcf-is-the-right-framework.aspx

http://www.infoq.com/interviews/tim-bray-future-of-web (November 2008)
to top


You are here: Main > ObsKML

to top

Copyright © 1999-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding DMCC? Send feedback