wiki:NERCPortals

Version 8 (modified by mkochan, 12 years ago) (diff)

--

The NERC Portals project

Since it's inception the NERC Portals project has started to be called the Data Portals Prototype Project (DPPP) across the participating institutions.

See

Conversion from CSML to KML (csml2kml)

SVN repository currently contains 2 directories which are supposed to serve as root directories of future Python eggs, usable for easy re-distribution across BADC machines. These are:

  • Located at kml/csml2kml/python/csml2kml - contains convertors from CSML to KML (although CSML is required to be accompanied by context/configuration files or additional GML documents providing context for the conversion)
  • Located at kml/csml2kml/python/pylonsstack - contains code for generating dynamic web content, implemented using the Pylons web framework. This has to be used whenever some code has to be run at view time (from user.s perspective). This is because Google Earth does not allow running of any code on the client side and therefore this must be provided via dynamic web content, served on BADC side

In addition to these, there are two directories:

  • kml/csml2kml/testdata contains all testing data (inputs)
  • kml/csml2kml/outputs contains all outputs from tests

Currently only CSML documents containing solely  csml:GridSeriesFeature and  csml:PointSeriesFeature elements are supported.

The module KMLDocument.py

Encapsulation of a KML document is achieved via the module KMLDocument.py. The whole KML document can be contained in an KMLDocument.KMLDocument object; this object in turn contains objects derived from class KMLDocument.KMLElement. Once a KMLDocument object has been generated, it can be saved into a file using the method KMLDocument.save() -- actual KML is generated only during saving.

The class GridSeriesConvertor

Example test is contained in kml/csml2kml/python/csml2kml/tests/testGridSeriesConvertor.py. This test reads in the climate modelling file clim_10.csml with month/decade values for various  csml:GridSeriesFeature's. The CSML file does not contain the grid data themselves -- these are read in from WMS. Since this information is not part of the CSML file, there must be a config file supplied as well, which contains the context for the conversion. Its format is as follows:

<CSML2KMLConfig>
  <CSMLGridSeriesFeatureWMSRequest>
    <URL>http://www-devel.ipcc-data.org/maps/wms/obs</URL>
    <ServiceVersion>1.1.1</ServiceVersion>
    <ImageFormat>image/png</ImageFormat>
    <ImageWidth>960</ImageWidth>
    <ImageHeight>480</ImageHeight>
    <CRS>EPSG:4326</CRS>
    <LayerName>#FILENAME_EXCL_SUFFIX#/#FEATURE_NAME#</LayerName>
    <LongitudeBounds>UNCONSTRAINED</LongitudeBounds>
    <LatitudeBounds>UNCONSTRAINED</LatitudeBounds>
  </CSMLGridSeriesFeatureWMSRequest>
  <View name="Whole century">
    <SplitTimeStepsBy></SplitTimeStepsBy>
    <LogicalDateTimeTransform>20_CENTURY_DECADE</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>MONTH_HENCE</LogicalDateTimeDelta>
  </View>
  <View name="Compare decades">
    <SplitTimeStepsBy>year</SplitTimeStepsBy>
    <CategoryNamingPattern>Decade of #VERBATIM#</CategoryNamingPattern>
    <LogicalDateTimeTransform>FIRST_DAY_OF_MONTH</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>MONTH_HENCE</LogicalDateTimeDelta>
  </View>
  <View name="Compare months">
    <SplitTimeStepsBy>month</SplitTimeStepsBy>
    <CategoryNamingPattern>#MONTH#</CategoryNamingPattern>
    <LogicalDateTimeTransform>FIRST_DAY_OF_MONTH</LogicalDateTimeTransform>
    <LogicalDateTimeDelta>DECADE_HENCE</LogicalDateTimeDelta>
  </View>
</CSML2KMLConfig>

The CSMLGridSeriesFeatureWMSRequest element contains information about the WMS to be used from retrieval. Each View element describes a different way of viewing the data, i.e. how they should be organized in Google Earth's left panel. This also determines how they can be animated.

The code for  csml:GridSeriesFeature's conversion is organised in the following modules:

  • GridSeriesConvertor.py -- contains class GridSeriesConvertor which performs the conversion
  • csmlwrappers.py -- contains wrapper classes which wrap around various CSML features. Currently there is only class GridSeriesFeatureWrapper.
  • kmlfeatures.py -- encapsulation of CSML features as KML elements. Does not use KMLDocument and is only used by GridSeriesConvertor.

The module StationConvertor.py

Example run is contained in kml/csml2kml/python/csml2kml/tests/testStationConvertor.py. This test reads in a GML document from an arbitrary URL (which normally is a GeoServer? call). It also requires a configuration object, which sets context for the conversion. The format of input is <i>not</i> CSML, but rather XML derived from GML that specifies locations of meteorological stations; each station can have several  csml:PointSeriesFeature's associated with it:

...

Note: The same server should return CSML of individual CSML features for any station. At the moment, how exactly this will be done is a matter of debate.

The configuration is obtained from a config file which has format similar to this:

<CSML2KMLConfig>
  <CSMLStationPointSeries>
    <name>MIDAS stations (example)</name>
    <UseRegions>yes</UseRegions>
    <GeoServerRequest>
      <URL>http://bond.badc.rl.ac.uk:8089/dummyGeoServer/GetStationCSMLFeatures?gml_id=MIDAS_Stations</URL>
      <BalloonTemplate>&lt;img src="http://bond.badc.rl.ac.uk:8089/csmlGrapher/plot?feature_id=$[feature_id]"&gt;</BalloonTemplate>
      <StationData>
        <Datum name="feature_id">#ID#</Datum>
      </StationData>
    </GeoServerRequest>
  </CSMLStationPointSeries>
</CSML2KMLConfig>

The element GeoServerRequest determines how stations list should be obtained (from which URL) and how information on each station should be shown in a balloon that pops out in Google Earth when a station is clicked. Each station can how certain data associated with it, which are substituted into the template. Using the #ID# value will in turn be substituted by the individual feature's ID.

Support for KML regions

If all stations were visible in Google Earth at the same time, the rendering would be very slow. It is therefore possible in Google Earth to organise stations (represented by kml:Placemark elements) into a hierarchical structure of regions using the kml:Folder and kml:Region tags that causes stations to be rendered only for closer zooms onto the ground (link).

Therefore, when the config-file tag UseRegions is set to "yes", the convertor will generate a KML document which will contain all stations contained in a single folder, with hierarchy of regions inside.

Codewise, the process performed two steps:

  • First, 2D globe space gets split into 4 smaller regions, and this is repeated recursively, until a small number of stations (10) are present in the current (low-level) region. Thus a so-called quad-tree is constructed (quad = each node has 4 children) which contains the hierarchy, with actual stations as leaves. The quad-tree functionality is contained in the module QuadTree.py. Thus, a QuadTree.QuadTree object is constructed.
  • Second, the QuadTree object is "translated" into a KMLDocument object.