wiki:UsingTheParserToCreateCSMLV2

Version 4 (modified by domlowe, 12 years ago) (diff)

changing TimeSeriesDomain? to TimeSeries?

How to create CSML V2 using the CSML Parser classes

As you will know if you read the other parser how to? (which hasn't been written yet, but the old one is here, the CSML parser creates a python representation of a CSML document, using python classes to represent XML data types.

Each parser class has a fromXML() and a toXML() method. The fromXML method takes an XML document (or fragment) and populates the attributes of the python object, the toXML method takes the attributes and structure of the python object and creates an XML ElementTree fragment.

So to create CSML from scratch using python, all you need to do is:

  • Create some parser objects - a Dataset, a FeatureCollection, some Features etc.
  • Set their attributes to the right values for your data*.
  • Call the toXML method of the root level Dataset() object.
  • Sit back and enjoy your CSML document.

*In this example I have hard-coded data.

The following code snippets go through this process for a sample feature. To find out more you will have to explore the data structures of the parser a bit more using PyDoc or dir(). But the general pattern remains the same for all feature types. (Or ask me!)

The Code:

Here is the full code listing which we will look at in more detail:

import csml.parser
import csml.parser_extra


  #########################################
    ##  construction of CSML objects ##
  #########################################


# create an empty list to hold featureMembers
fms =[]


#### create a PointSeriesFeature: #####
ptsf=csml.parser.PointSeriesFeature()

#set the id attribute
ptsf.id='testbed270401'

#set the description
#note, as description is an xml element not an attribute it is of type csString:
ptsf.description=csml.parser.csString('Station BLUEBIRD')

#create a PointSeriesCoverage
ptscvg=csml.parser.PointSeriesCoverage()

#create a TimeSeriesDomain (appropriate domain for a PointSeriesFeature
ptsd=csml.parser.TimeSeries()
#create (and populate) a TimePositionList. Using keyword arguements for conciseness.
ptsd.time=csml.parser.TimePositionList(frame='#pred20060427001',CONTENT='-18 -17 -16 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60')
    
#create a RangeSet
rs=csml.parser.RangeSet()
#set the quantityList attribute of the RangeSet to be a MeasureOrNullList containing the values for the rangeSet
rs.quantityList=csml.parser.MeasureOrNullList(uom='MLUnits.xml#m',CONTENT='0.27 0.25 0.25 0.23 0.22 0.22 0.21 0.17 0.17 0.14 0.14 0.12 0.10 0.08 0.08 0.08 0.11 0.13 0.11 0.11 0.13 0.10 0.06 0.06 0.06 0.01 -0.03 -0.06 -0.09 -0.11 -0.11 -0.13 -0.16 -0.18 -0.17 -0.16 -0.18 -0.16 -0.12 -0.09 -0.08 -0.07 -0.06 -0.05 -0.04 -0.03 -0.03 -0.04 -0.02 -0.01 -0.02 -0.02 -0.02 -0.03 -0.03 -0.02 -0.01 -0.01 -0.02 -0.02 -0.03 -0.04 -0.04 -0.05 -0.04 -0.05 -0.08 -0.09 -0.11 -0.12 -0.12 -0.12 -0.12 -0.12 -0.13 -0.14 -0.15')


#Add the domain and rangeSet as attributes of the coverage
ptscvg.pointSeriesDomain=ptsd
ptscvg.rangeSet=rs

#set the coverage to be the 'value' of the feature
ptsf.value=ptscvg


#the parameter of the feature is of type Phenomenon, here href creates "xlink:href=..."
param=csml.parser.Phenomenon(href='CFStandardNames.xml#temperature')

#set the parameter of the feature to be the parameter just defined
ptsf.parameter=param

#append the feature to the list of featureMembers
fms.append(ptsf)


#instantiate a FeatureCollection object (and add all featureMembers)
fc=csml.parser.CSMLFeatureCollection(featureMembers=fms)


########### The Dataset  ##############
#Create an Empty Dataset
ds = csml.parser.Dataset()
#set the xml attribute 'id'
ds.id='Test001'

#set the description element 
ds.description=csml.parser.csString('This is a test Water level.Period 26/04/06 06h -> 29/04/06 12h')

#set the metaDataProperty(s)
#if there is only one metaDataProperty you can just do:
#ds.metaDataProperty=csml.parser.csString('Data (c) 2007 CCLRC // www.cclrc.ac.uk')

#but if there are several you need to put them in a list:
mdplist=[]
mdp1=csml.parser.csString('Data (c) 2007 CCLRC // www.cclrc.ac.uk')
mdp2=csml.parser.csString('Another CCLRC MetaDataProperty')
mdplist.append(mdp1)
mdplist.append(mdp2)
ds.metaDataProperty=mdplist

#now add the featureCollection to the dataset
ds.featureCollection=fc

#finished creating all the python objects!
########################################

####### Generating XML #################
            
#call the toXML method of the Dataset object:
csmldoc = ds.toXML()
#parse and pretty print the result
strCSML=csml.parser_extra.PrettyPrint(csmldoc)
strCSML=csml.parser_extra.removeInlineNS(strCSML)
print strCSML

#(you could save strCSML to a file at this point)

You can download this code from here.

Now to look at this in more detail:

First we import the parser and parser_extra (which contains some essential extra functions):

import csml.parser
import csml.parser_extra

Now we can start the construction of CSML objects. However it probably helps to look at the end of the code first. Here we create a Dataset object (ds) and set other objects as attributes of the Dataset object.

########### The Dataset  ##############
#Create an Empty Dataset
ds = csml.parser.Dataset()
#set the xml attribute 'id'
ds.id='Test001'
#set the description element 
ds.description=csml.parser.csString('This is a test Water level.Period 26/04/06 06h -> 29/04/06 12h')
#(snipped a bit here for compactness)
#now add the featureCollection to the dataset
ds.featureCollection=fc
########################################

So the feature collection is as an attribute of the Dataset. (Remember when we call the toXML() method on the Dataset, this also calls the toXML() methods of the !CSMLFeatureCollection, ReferenceSystemDefinitions etc.)

If the aim is to get a fully populated Dataset object, we need to create some feature objects to put in the !CSMLFeatureCollection. Below we are creating a PointSeriesFeature, and building up the constituent parts.

# #first we need an empty list to hold the featureMembers
fms =[]
#### create a PointSeriesFeature: #####
ptsf=csml.parser.PointSeriesFeature()

#set the id attribute
ptsf.id='testbed270401'
#set the description
#note, as description is an xml element not an attribute it is of type csString:
ptsf.description=csml.parser.csString('Station BLUEBIRD')

#create a PointSeriesCoverage
ptscvg=csml.parser.PointSeriesCoverage()

#create a TimeSeriesDomain (appropriate domain for a PointSeriesFeature
ptsd=csml.parser.TimeSeries()
#create (and populate) a TimePositionList. Using keyword arguements for conciseness.
ptsd.time=csml.parser.TimePositionList(frame='#pred20060427001',CONTENT='-18 -17 -16 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60')
    
#create a RangeSet
rs=csml.parser.RangeSet()
#set the quantityList attribute of the RangeSet to be a MeasureOrNullList containing the values for the rangeSet
rs.quantityList=csml.parser.MeasureOrNullList(uom='MLUnits.xml#m',CONTENT='0.27 0.25 0.25 0.23 0.22 0.22 0.21 0.17 0.17 0.14 0.14 0.12 0.10 0.08 0.08 0.08 0.11 0.13 0.11 0.11 0.13 0.10 0.06 0.06 0.06 0.01 -0.03 -0.06 -0.09 -0.11 -0.11 -0.13 -0.16 -0.18 -0.17 -0.16 -0.18 -0.16 -0.12 -0.09 -0.08 -0.07 -0.06 -0.05 -0.04 -0.03 -0.03 -0.04 -0.02 -0.01 -0.02 -0.02 -0.02 -0.03 -0.03 -0.02 -0.01 -0.01 -0.02 -0.02 -0.03 -0.04 -0.04 -0.05 -0.04 -0.05 -0.08 -0.09 -0.11 -0.12 -0.12 -0.12 -0.12 -0.12 -0.13 -0.14 -0.15')

#Add the domain and rangeSet as attributes of the coverage
ptscvg.pointSeriesDomain=ptsd
ptscvg.rangeSet=rs

#set the coverage to be the 'value' of the feature
ptsf.value=ptscvg

#the parameter of the feature is of type Phenomenon, here href creates "xlink:href=..."
param=csml.parser.Phenomenon(href='CFStandardNames.xml#temperature')

#set the parameter of the feature to be the parameter just defined
ptsf.parameter=param

Now add this feature to the list of features (which will later become the contents of the !CSMLFeatureCollection).

#append the feature to the list of featureMembers
fms.append(ptsf)

Okay, so you can create as many features as you like and add them to the fms[] (feature members) list.

The process for creating file extracts e.g. NetCDFExtract() is very similar. Just instantiate a NetCDFExtract class, and set its attributes (id, variableName, fileName) to the values you want. The file extracts are kept inside a !CSMLStorageDescriptor, which is like a FeatureCollection.

Anyway that was an aside. Now we have a feature or features we need to create a !CSMLFeatureCollection object to contain them.

#instantiate a FeatureCollection object (and add all featureMembers)
fc=csml.parser.CSMLFeatureCollection(featureMembers=fms)

Now we can create the Dataset object:

#Create an Empty Dataset
ds = csml.parser.Dataset()
#set the xml attribute 'id'
ds.id='Test001'

#set the description element 
ds.description=csml.parser.csString('This is a test Water level.Period 26/04/06 06h -> 29/04/06 12h')

#set the metaDataProperty(s)
#if there is only one metaDataProperty you can just do:
#ds.metaDataProperty=csml.parser.csString('Data (c) 2007 CCLRC // www.cclrc.ac.uk')

#but if there are several you need to put them in a list:
mdplist=[]
mdp1=csml.parser.csString('Data (c) 2007 CCLRC // www.cclrc.ac.uk')
mdp2=csml.parser.csString('Another CCLRC MetaDataProperty')
mdplist.append(mdp1)
mdplist.append(mdp2)
ds.metaDataProperty=mdplist

Note the difference between setting an stringlike XML attribute (id) and a stringlike child element (description). The child element must be a parser object not just a string. As there is no Description() type in the parser it uses the generic csml.parser.csString() which is used for simple elements like gml:description.

And also note that the metaDataProperty behaviour requires that you create a list if you want multiple metaDataProperties.

And now as shown earlier, we tie everything together by assigning your feature collection to be an attribute of the Dataset object:

#now add the featureCollection to the dataset
ds.featureCollection=fc
#finished creating all the python objects!

To generate your XML document you now need to call the toXML() method of the dataset, and then also run a couple of functions that pretty print the result and fix up ElementTree namespaces.

####### Generating XML #################
            
#call the toXML method of the Dataset object:
csmldoc = ds.toXML()
#parse and pretty print the result
strCSML=csml.parser_extra.PrettyPrint(csmldoc)
strCSML=csml.parser_extra.removeInlineNS(strCSML)
print strCSML

#(you could save strCSML to a file at this point)

That's it. So to recap, all we have done is:

  • Created some parser objects - a Dataset, a !CSMLFeatureCollection, some Features (well, one feature) etc.
  • Set the values of their attributes (and the values of the attributes attributes if the attribute is itself a parser object).
  • Set these objects to be attributes of the root level Dataset object.
  • Called the toXML method of the Dataset object.
  • Called a couple of utilty functions to tidy up the XML.
  • Sat back and enjoyed our CSML document.