wiki:T02_CSML

T02 CSML

The Climate Science Modelling Language provides the data virtualisation mechanism in NDG. It enables data to be described, accessed and manipulated within an ISO standards compliant framework, enabling interoperability within NDG partners and potentially more widely.

Quick links to documentation:

The key subcomponents of CSML are:

  • Feature Type Definitions: A set of UML conceptual models ('feature types') for a range of atmospheric and oceanographic data types relevant to key NDG Data Providers, based on the framework and components provided by ISO standards for geospatial information modelling. See CSML_FeatureTypes
  • CSML itself (an application schema of GML), built around these feature types providing a standards-based reference encoding for NDG datasets.
  • CSML Tools:
    • A CSML 'scanner' to facilitate the production of CSML markup for some existing data file formats (CSML_Scanner)
    • A CSML 'parser' to demarshall a CSML document into corresponding object instances (CSML_Parser)
  • CSML Services including:
    • A wrapper mechanism for exposing NDG data as CSML datasets.
    • A wrapper mechanism for exploiting internal and external vocabularies.
  • CSML reference guide providing description and use guidelines for CSML (CSML_RefGuide).
  • CSML Development in an international context: collaborative work with MOTIIVE, INSPIRE, the IOC, and the WMO on application schema for MetOcean data.
  • CSML Web Services: see CdataServices (new services being developed in 2007)

See the CSML Population Schedule (CSML_Population).

Key Integration Milestones

Note, some dates may change - need to discuss with Andrew re: his other commitments.

Note that CSML V1.0 exists prior to NDG2.

  • 1.CSML-Parser 1.0 (April 7, 2006) 99
  • 1b. CSML i/o layer to parser (May 30 2006) 127
  • 2.CSML-Scanner 1.0 Scanner (April 21, 2006) 100
  • 3.CSML-FT 1.1 Modified Feature Types (May 19, 2006)
  • 4.CSML-File 1.1 CSML CF-netCDF file instantiation service for 1.1 Feature Types (June 16, 2006)
  • 5.CSML-FT 2.0 UML Feature Type Definitions (June 16, 2006)
  • 6.CSML-Op 2.0 Document on framework for handling FT ‘operations’ (e.g. subsetting, rendering) (June 1, 2006)
  • 7.CSML-AS 2.0 Revised Application Schema (June 16, 2006)
  • 8.CSML-RG 2.0 ReferenceGuide? (June 16, 2006)
  • 9.CSML-Scanner 2.0 Scanner updated to conform (June 23,2006)
  • 10.CSML-AFT 2.0 Abstract File Type Wrapper Mechanism to inherit data (June 23, 2006)
  • 11.CSML 2.0 Release (June 23, 2006)
  • 12.NDG Alpha (July 2006): Functionality expected: CSML 2.1 (Bug fixes on CSML 2.0 plus any minor enhancements noted and requested).
  • 13.Wrapper Mechanism to exploit external Vocabularies (September 1, 2006)
  • 14.Expanded CSML delivery services – OPeNDAP, W*S (December 1, 2006)
  • 15.CSML 3.0 Release (December 1, 2006).
  • 16.CSML schema roundtripping (December 1, 2006)
  • 17.NDG Beta (January 2007): Functionality expected: CSML 3.1
  • 18.NDG Final. Functionality expected. CSML 3.1

Integration Dependencies

  • 1.Delivery Service (Delivery)
  • 2.Security Metadata in CSML decision (with MOLES, Security etc, February 2006)
  • 3.Vocab server

Internal Development Stages

(Note these are all milestone activities, probably need further breakdown, especially for the long things.)

  • 1.CSML-Parser [V1.0, 1 week, Dominic, April 7 2006]
    • a.[Definite] Complete CSML 1.0 feature types
    • b.[Definite] Code on CVS
    • c.[Definite] Tested on Instance Documents
  • 2.CSML-Scanner [V1.0, 3 weeks, Dominic, April 21 2006]
    • a.[Definite] Remove current xml writing code and replace with Parser module.
    • b.[Definite] Nasa Ames Input for points, profiles and profile series (for selected data)
    • c.[Definite] CF-Netcdf for grids and grid series
    • d.[Definite] PP for grids and grid series
    • e.[Maybe] Grid for grids and grid seriesWiki_page?
  • 3.CSML-FT [V1.1, 5 days, Andrew, May 19, 2006]
    • a.[Discussion] UML Resolution of problems with species and METARS for CSML1.0 (How far do we go to support EcoGrid? data with CSML?) Set of CTD profiles issue.
    • b.[Definite] Finalised 1.0 Application Schema
    • c.[Definite] Finalised 1.0 Reference Guide
  • 4.CSML-File [V1.1, 15 days, Andrew/Dominic?, June 16, 2006]
    • a.[Definite] CF-netCDF file instantiation service for 1.1 Feature Types
  • 5.CSML-FT [V2.0, 15 days, Andrew, June 16, 2006]
    • a.[Probable] Revised Obs and Measurement-compliant feature-type definitions
    • b.[Definite] Support for swathes
    • c.[Definite] Support for multiple measurands
  • 6.CSML-Op [V2.0, 5 days, Andrew, June 1, 2006]
    • a.[Definite] Short document on framework and suggested roadmap for handling FT ‘operations’ (e.g. subsetting, rendering); cf. Atkinson et. al. ‘operations’ work and OGC ‘Styled Layer Descriptor’ approach for associating rendering styles with feature types NB: DX/Geosplat potential impact
  • 7.CSML-AS [V2.0, 30 days, Andrew, June 16, 2006]
    • a.[Definite] Separation out of non-GML compliant components
    • b.[Definite] Final resolution of GML-modifications required
    • c.[Definite] Support for new UML types
    • d.[Definite] Short report on best practice legacy data integration (cf. NDG AbstractArray? Descriptor vs SEEGrid XPath/geoserver/SQL mechanism) NB: This is an AUKEGGS deliverable
    • e.[Definite] Modify CSML to separate out wrapper mechanism, incorporating outcomes of (d).
  • 8.CSML-RG [V2.0 4 days, Andrew, June 23, 2006]
    • a.[Definite] Revisions to support changes to AS and FT.
    • b.[Probable] Revisions to outline future of vocabularies and wrappers
  • 9.CSML-Scanner [V2.0, 20 days, Dominic, June 23, 2006]
    • a.[Definite] Support for all CSML1.0 feature types which remain in CSML2.0
    • b.[Probable] Support for all CSML2.0 feature types.
  • 10.CSML-AFT [V2.0, 20 days, Dominic, June 23, 2006]
    • a.[Definite] Support for libcdunif
    • b.[Definite] Support for NasaAmes?
  • 11.CSML [2.0, 2 days total, Andrew/Dominic?, June 26, 2006]
    • a.[Definite] Website Modifications.
  • 12.CSML [2.1, 10 days each, Andrew/Dominic?, July 1, 2006]
    • a.[Probable] Bug fixes

NDG Alpha Milestone Complete

  • 13.CSML [2.X, n days total, Andrew, September 1 2006]
    • a.[Definite] Modify internal dictionary mechanism to exploit BODC vocabulary server
  • 14.CSML-Services [3.0, n days total, Andrew/Dominic/Ag?, December 1, 2006]
    • a.OPeNDAP
    • b.OGC W*S
  • 15.CSML [3.0, n days total, Andrew, December 1, 2006].

NDG Beta Milestone Complete

  • 16.CSML-Schema [3.0, Andrew, December 1, 2006]
    • a.Automated UMLGML App Schema (UGAS): comes from OGC/Clemens Portele/MOTIIVE/Rob Atkinson work
    • b.Automated UMLparser: comes from techniques used in UGAS, ‘hardened’ GML (ISO 19136), and CSML parser experience
    • c.Integration: the requirement is a ‘turnkey’ system for UML schema design auto-generating both GML app schema and corresponding parsing code
  • 17.NDG Final. Functionality expected. CSML 3.1

Issues

  • BODC profile collections requirements into CSML needed.
  • BODC to produce CSML docs for things that are supported by CSML1.
  • CSML2 not to break existing feature types …
  • Parser version 2 needs to be in there.
  • Vocab responsibilities.
  • Dates for 8-11 need to respect date for 7.
  • QXF enabling the wrapper etc.
  • PML and NOCS contributions for IO subsystems for limited “non-standard” cases. Cookbook guide on how to do it.
  • CSML docs in a database? Advantages disadvantages? Implications on library and data provider package (preconfiguring exist).
  • csmlAPIQuestions
  • Importance of  ID issues