Changeset 5104

13/03/09 14:34:50 (12 years ago)

Adding CSML API and OWSLib to tutorial

1 edited


  • TI09-UKCollaboration/trunk/DS_Workshop/doc/walkthrough.rst

    r5103 r5104  
    7 This tutorial will walk you through creating a combined WMS/WCS server 
    8 for NetCDF data conforming to the CF conventions using the CEDA OWS 
    9 Framework (COWS). 
     7This tutorial will show you how to use CSML (Climate Science Modelling Language) tools and then walk you through creating a combined WMS/WCS server for NetCDF data conforming to the CF conventions using the CEDA OWS Framework (COWS).  
    119Setting Up your environment 
    4543NOTE:: Experienced UNIX users might want to put this in your ~/.profile file. 
     46Installing CSML 
    4749Most COWS components can be automatically downloaded and installed 
    5961  find-links = /home/spascoe/ds_workshop/eggs 
    61 You can now install COWS along with the improved Python shell 
    62 ``ipython``.  All COWS dependencies will be installed 
     64You can now install CSML along with the improved Python shell 
     65``ipython``.  All CSML dependencies will be installed 
    65   $ easy_install cows ipython 
    69 Generating CSML for the test dataset 
     68  $ easy_install csml ipython 
     71Using the CSML Scanner to create a CSML document 
     73In the directory /home/dlowe/data/WindFeb09 there is some test data. This data contains a month long timeseries of wind measurements from a stationary observing platform. The data is stored in an ASCII file format called NASAAmes, and it is stored in multiple files (one file per day).  
     75Open one of the files and have a look at it:: 
     77  $ less /home/dlowe/data/WindFeb09/ 
     79You can see the data contains 4 variables (eastward_wind, northward_wind etc.) measured at a single location.  
     81Each of these variables can be represented as a CSML PointSeries feature. So for example one CSML feature can contain 'eastward_wind' across all the NASAAmes files, creating a timeseries of the entire month's data. 
     83We will use the 'csmlscan' tool to create a CSML document to describe these files, but first 
     84you need to install a library that can read NASAAmes files. This library is called NAppy (NASA Ames Processing in Python):: 
     86  $ easy_install nappy 
     88Now the 'csmlscan' tool can be used to create a CSML document to describe these files:: 
     90  $ csmlscan -f PointSeries -L '52.42 -4.05' -d /home/dlowe/data/WindFeb09 -o mycsml.xml  
     93What do the command line arguments to csmlscan mean? 
     95  -f PointSeries 
     96   This flag tells the scanner to create PointSeries features. 
     98  -L '52.42 -4.05'  
     100   This flag is used to specify a latitude longitude location for the PointSeries. Unfortunately in this case we have to manually supply it as NASAAmes metadata is not consistent between files, so a computer can't read it easily. Look in the NASAAmes file again and you can see:: 
     102    Location name: Frongoch farm (near Aberystwyth, UK) 
     103    Location: 52.42 degrees N, -4.05 degrees E 
     104              base of tower 140 m above mean sea level 
     106  -d /home/dlowe/data/WindFeb09 
     108   d is for 'directory' where the dataset resides 
     110  -o mycsml.xml 
     112   o is for 'outputfile' where your CSML document will be created 
     115So you should now have a CSML file, 'mycsml.xml'. Open it in a text editor and have a look over it. The XML should contain 4 PointSeries features for the entire month's data. 
     118Experimenting with the CSML API 
     120CSML has a python based Application Programming Interface (API) to interact with CSML documents. We will explore this interactively in ipython. 
     123Start ipython:: 
     125  $ipython 
     127and import csml:: 
     129  >>> import csml 
     131Now you can parse your csml file into a Dataset object:: 
     133  >>> ds=csml.parser.Dataset('mycsml.xml') 
     135and you can list the features in that file:: 
     137  >>> ds.listFeatureIDs() 
     140select a feature by id and assign it to a varible 'f':: 
     142  >>> f=ds['Mean_eastward_wind_in_1_minute_sample_period'] 
     144you can now look at the data in that feature:: 
     146  >>> f.getLatLon() #get the lat lon of the station 
     148  >>> len(f.getTimes()) #see how many times there are  (len="length") 
     149  >>> len(f.getDataValues()) # see how many data values 
     151  >>> t=f.getTimes()[0:10]   #Assign the first 10 time values to a variable, t 
     152  >>> v=f.getDataValues()[0:5] #Assign the first 10 time values to a variable, v 
     153  >>> zip(t,v) #display them together 
     155There is also a built in method getSubsetOfData() which allows you request data within a time range. 
     157e.g. Get all the data for February 5th 2009:: 
     159  >>> f.getSubsetOfData(('2009-02-05T00:00:00.0', '2009-02-05T23:59:59.59')) 
     163Installing COWS ready to setup OGC services 
     166You can install COWS using easy_install as before:: 
     168  $ easy_install cows  
     170Again this command will install the egg (cows) and all it's dependencies - it will check 
     171you have CSML installed - you should already have from earlier, but if not it will install it  
     174So we could equally have got to this stage and installed cows, csml & ipython with a single command:: 
     176  $easy_install cows ipython 
     179Generating CSML for the test gridded dataset 
    72182Some CF-NetCDF has been placed in /home/spascoe/ds_walkthrough/data. 
    73183!TODO: describe datasets.  To generate CSML for this data use the 
    74 ``csmlscan`` tool.  First create a configuration file: 
     184``csmlscan`` tool again. This time we will use a configuration file 
     185rather than command line arguments. First create a configuration file: 
    107218  ******************************************************************** 
    109 Interacting with the CSML API 
    110 ----------------------------- 
    112 !TODO 
     220In this case, if you look at the CSML file you will notice that all the data is not  
     221stored inline in the CSML, but is kept in the original NetCDF files and referenced via 
     222'NetCDFExtract' elements in the CSML document. 
     224Interacting with GridSeries data via the CSML API 
     227As before you can parse the csml file, list the features and select one: 
     229  >>> import csml 
     230  >>> ds=csml.parser.Dataset('csml/hadcm3.xml') 
     231  >>> ds.listFeatureIDs() 
     232  >>> f=ds['air_temperature'] 
     234This time you can look at the 4D spatio-temporal domain of the grid:: 
     236  >>> f.getDomain() 
     238And you can subset the GridSeries to a smaller GridSeries feature:: 
     240  >>> outputdir='./' 
     241  >>> outputfile='' 
     242  >>> subsetDictionary = {'latitude': (-10, 10), 'longitude': (55, 65), 'time':('2400-01-00T00:00:00.0', '2410-01-00T00:00:00.0')} 
     243  >>> f.subsetToGridSeries(outputdir, outputfile, **subsetDictionary) 
     245As you can see this method writes the new grid out as both a NetCDF file and as a new CSML feature. Come out of ipython and run 'ncdump' to view the headers of your new grid:: 
     247  ncdump -c 
     249note, there are other methods for subsetting a GridSeries feature to other feature types such as PointSeries or Profiles. 
    188325Try visiting ``<username>/``.  Note the 
    189326trailing slash which is important.  You should see the COWS server 
    190 catalogue. 
     327catalogue and can try out the WMS demo. 
     329Accessing the Web Coverage Service (WCS) with OWSLib 
     331The WCS can be accessed using any standard WCS client. Unfortunately WCS is not as mature a standard as WMS so not many of these have been built yet! 
     333There is however an Open Souce python OGC client library called OWSLib which we will use to communicate with the WCS. Again we need to install this with easy_install:: 
     335  $easy_install owslib 
     337Start ipython again and make a connection to the WCS: 
     339  >>> from owslib.wcs import WebCoverageService 
     340  >>> wcs=WebCoverageService('',version='1.0.0') 
     341  >>> print 'Accessing WCS version %s at %s'%(wcs.version, wcs.url) 
     343The OWSLib wcs 'identification' object contains general information about the service (some of which is missing in this service!): 
     345  >>> wcs.identification.service 
     346  >>> wcs.identification.title 
     347  >>> wcs.identification.abstract 
     348  >>> wcs.identification.keywords 
     349  >>> wcs.identification.fees 
     350  >>> wcs.identification.accessConstraints 
     352The OWSLib wcs 'provider' oject contains general information about the service provider:: 
     354  >>> 
     355  >>> wcs.provider.url 
     356  >>> 
     357  >>> 
     358  >>> 
     359  >>> 
     360  >>> 
     361  >>> 
     362  >>> 
     363  >>> 
     366Now take a look at the available coverages:: 
     368  >>> wcs.wcs.contents 
     370And select a coverage from the list:: 
     372  >>> cvg=wcs['air_temperature'] 
     374Investigate it:: 
     376  >>> cvg.boundingBoxWGS84 
     377  >>> cvg.timepositions 
     378  >>> cvg.timelimits 
     379  >>> cvg.supportedFormats 
     380  >>> cvg.supportedCRS 
     382Now we know a bit about the coverage we can make a request to get a subset of it from the WCS:: 
     384  >>> response=wcs.getCoverage(identifier='air_temperature',time=['2992-11-16T00:00:00.0'],bbox=(-80,30,50,60), crs='WGS84', format='cf-netcdf') 
     386You can write this response to a file: 
     387  >>> f=open('', 'wb') 
     388  >>> f.write( 
     389  >>> f.close() 
     391Come out of ipython  and use ncdump again to investigate your WCS response:: 
     393  ncdump -c 
Note: See TracChangeset for help on using the changeset viewer.