wiki:NERCPortals

Version 11 (modified by mkochan, 11 years ago) (diff)

--

The NERC Portals project

Since it's inception the NERC Portals project has started to be called the Data Portals Prototype Project (DPPP) across the participating institutions.

See

Conversion from CSML to KML

Since CSML currently lacks certain aspects of describing the context data it contains in the features (i.e. the metadata layer), the actual mechanisms used to convert data from CSML to KML do use only limited amount of CSML as a data source. The rest comes from different languages derived from GML. Hence this project became rather a demonstration of how OGC web services can be integrated into a common framework, which has a potential of working using purely CSML data, once CSML matures enough to contain all information in it.

Currently only CSML documents containing solely < csml:GridSeriesFeature> and < csml:PointSeriesFeature> elements are supported.

The functionality is split into two eggs:

  • csml2kml - provides static functionality, with the output being KML stored in files (*.kml, *.kmz),
  • csml2kmlpylon - is a web server (to be run by the BADC) providing web services which bring dynamic content to the viewing experience in Google Earth via HTTP.

The module KMLDocument.py

Encapsulation of a KML document is achieved via the module KMLDocument.py. The whole KML document can be contained in an KMLDocument.KMLDocument object; this object in turn contains objects derived from class KMLDocument.KMLElement. Once a KMLDocument object has been generated, it can be saved into a file using the method KMLDocument.save() -- actual KML is generated only during saving.

The class GridSeriesConvertor

Example test is contained in kml/csml2kml/python/csml2kml/tests/testGridSeriesConvertor.py. This test reads in the climate modelling file clim_10.csml with month/decade values for various  csml:GridSeriesFeature's. The CSML file does not contain the grid data themselves -- these are read in from WMS. Since this information is not part of the CSML file, there must be a config file supplied as well, which contains the context for the conversion. Its format is as follows:

<CSML2KMLConfig>
  <CSMLGridSeriesFeatureWMSRequest>
    <URL>http://www-devel.ipcc-data.org/maps/wms/obs</URL>
    <ServiceVersion>1.1.1</ServiceVersion>
    <ImageFormat>image/png</ImageFormat>
    <ImageWidth>960</ImageWidth>
    <ImageHeight>480</ImageHeight>
    <CRS>EPSG:4326</CRS>
    <LayerName>#FILENAME_EXCL_SUFFIX#/#FEATURE_NAME#</LayerName>
    <LongitudeBounds>UNCONSTRAINED</LongitudeBounds>
    <LatitudeBounds>UNCONSTRAINED</LatitudeBounds>
  </CSMLGridSeriesFeatureWMSRequest>
  <View name="Whole century">
    <SplitTimeStepsBy></SplitTimeStepsBy>
    <LogicalDateTimeTransform>20_CENTURY_DECADE</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>MONTH_HENCE</LogicalDateTimeDelta>
  </View>
  <View name="Compare decades">
    <SplitTimeStepsBy>year</SplitTimeStepsBy>
    <CategoryNamingPattern>Decade of #VERBATIM#</CategoryNamingPattern>
    <LogicalDateTimeTransform>FIRST_DAY_OF_MONTH</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>MONTH_HENCE</LogicalDateTimeDelta>
  </View>
  <View name="Compare months">
    <SplitTimeStepsBy>month</SplitTimeStepsBy>
    <CategoryNamingPattern>#MONTH#</CategoryNamingPattern>
    <LogicalDateTimeTransform>FIRST_DAY_OF_MONTH</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>DECADE_HENCE</LogicalDateTimeDelta>
  </View>
</CSML2KMLConfig>

The CSMLGridSeriesFeatureWMSRequest element contains information about the WMS to be used from retrieval. Each View element describes a different way of viewing the data, i.e. how they should be organized in Google Earth's left panel. This also determines how they can be animated.

The code for  csml:GridSeriesFeature's conversion is organised in the following modules:

  • GridSeriesConvertor.py -- contains class GridSeriesConvertor which performs the conversion
  • csmlwrappers.py -- contains wrapper classes which wrap around various CSML features. Currently there is only class GridSeriesFeatureWrapper.
  • kmlfeatures.py -- encapsulation of CSML features as KML elements. Does not use KMLDocument and is only used by GridSeriesConvertor.

The module StationConvertor.py

Example run is contained in kml/csml2kml/python/csml2kml/tests/testStationConvertor.py. This test reads in a GML document from an arbitrary URL (which normally is a GeoServer? call). It also requires a configuration object, which sets context for the conversion. The format of input is not CSML, but rather XML derived from GML that specifies locations of meteorological stations; each station can have several  csml:PointSeriesFeature's associated with it:

<?xml version="1.0" encoding="UTF-8"?>
<gml:FeatureCollection xmlns="http://ndg.nerc.ac.uk/np" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="..." gml:id="MIDAS_Stations">
	<gml:featureMember>
		<Station gml:id="midas.station.1600">
			<stationName>GAN, MALDIVE IS</stationName>
			<stationID>1600</stationID>
			<location>
				<gml:Point gml:id="midas.station.1600.point">
					<gml:pos>73.15 0.683</gml:pos>

				</gml:Point>
			</location>
		</Station>
	</gml:featureMember>
	<gml:featureMember>
		<Station gml:id="midas.station.17708">
                  ...
		</Station>
	</gml:featureMember>
	<gml:featureMember>
		<Station gml:id="midas.station.18008">
                  ...
		</Station>
	</gml:featureMember>
</gml:FeatureCollection>

Note: The same server should return CSML of individual CSML features for any station. At the moment, how exactly this will be done is a matter of debate.

The configuration is obtained from a config file which has format similar to this:

<CSML2KMLConfig>
  <CSMLStationPointSeries>
    <name>MIDAS stations (example)</name>
    <UseRegions>yes</UseRegions>
    <GeoServerRequest>
      <URL>http://bond.badc.rl.ac.uk:8089/dummyGeoServer/GetStationCSMLFeatures?gml_id=MIDAS_Stations</URL>
      <BalloonTemplate>&lt;img src="http://bond.badc.rl.ac.uk:8089/csmlGrapher/plot?feature_id=$[feature_id]"&gt;</BalloonTemplate>
      <StationData>
        <Datum name="feature_id">#ID#</Datum>
      </StationData>
    </GeoServerRequest>
  </CSMLStationPointSeries>
</CSML2KMLConfig>

The element GeoServerRequest determines how stations list should be obtained (from which URL) and how information on each station should be shown in a balloon that pops out in Google Earth when a station is clicked. Each station can how certain data associated with it, which are substituted into the template. Using the #ID# value will in turn be substituted by the individual feature's ID.

Support for KML regions

If all stations were visible in Google Earth at the same time, the rendering would be very slow. It is therefore possible in Google Earth to organise stations (represented by kml:Placemark elements) into a hierarchical structure of regions using the kml:Folder and kml:Region tags that causes stations to be rendered only for closer zooms onto the ground ( http://code.google.com/apis/kml/documentation/regions.html#nestingregions).

Therefore, when the config-file tag UseRegions is set to "yes", the convertor will generate a KML document which will contain all stations contained in a single folder, with hierarchy of regions inside.

Codewise, the process performed two steps:

  • First, 2D globe space gets split into 4 smaller regions, and this is repeated recursively, until a small number of stations (10) are present in the current (low-level) region. Thus a so-called quad-tree is constructed (quad = each node has 4 children) which contains the hierarchy, with actual stations as leaves. The quad-tree functionality is contained in the module QuadTree.py. Thus, a QuadTree.QuadTree object is constructed.
  • Second, the QuadTree object is "translated" into a KMLDocument object.