Changeset 5104 for TI09-UKCollaboration


Ignore:
Timestamp:
13/03/09 14:34:50 (11 years ago)
Author:
domlowe
Message:

Adding CSML API and OWSLib to tutorial

File:
1 edited

Legend:

Unmodified
Added
Removed
  • TI09-UKCollaboration/trunk/DS_Workshop/doc/walkthrough.rst

    r5103 r5104  
    55======================================== 
    66 
    7 This tutorial will walk you through creating a combined WMS/WCS server 
    8 for NetCDF data conforming to the CF conventions using the CEDA OWS 
    9 Framework (COWS). 
     7This tutorial will show you how to use CSML (Climate Science Modelling Language) tools and then walk you through creating a combined WMS/WCS server for NetCDF data conforming to the CF conventions using the CEDA OWS Framework (COWS).  
    108 
    119Setting Up your environment 
     
    4442 
    4543NOTE:: Experienced UNIX users might want to put this in your ~/.profile file. 
     44 
     45 
     46Installing CSML 
     47------------------ 
    4648 
    4749Most COWS components can be automatically downloaded and installed 
     
    5961  find-links = /home/spascoe/ds_workshop/eggs 
    6062 
    61 You can now install COWS along with the improved Python shell 
    62 ``ipython``.  All COWS dependencies will be installed 
     63 
     64You can now install CSML along with the improved Python shell 
     65``ipython``.  All CSML dependencies will be installed 
    6366automatically:: 
    6467 
    65   $ easy_install cows ipython 
    66  
    67  
    68  
    69 Generating CSML for the test dataset 
     68  $ easy_install csml ipython 
     69 
     70 
     71Using the CSML Scanner to create a CSML document 
     72-------------------------------------------------- 
     73In the directory /home/dlowe/data/WindFeb09 there is some test data. This data contains a month long timeseries of wind measurements from a stationary observing platform. The data is stored in an ASCII file format called NASAAmes, and it is stored in multiple files (one file per day).  
     74 
     75Open one of the files and have a look at it:: 
     76 
     77  $ less /home/dlowe/data/WindFeb09/wind-sensors_frongoch_20090208.na 
     78 
     79You can see the data contains 4 variables (eastward_wind, northward_wind etc.) measured at a single location.  
     80 
     81Each of these variables can be represented as a CSML PointSeries feature. So for example one CSML feature can contain 'eastward_wind' across all the NASAAmes files, creating a timeseries of the entire month's data. 
     82 
     83We will use the 'csmlscan' tool to create a CSML document to describe these files, but first 
     84you need to install a library that can read NASAAmes files. This library is called NAppy (NASA Ames Processing in Python):: 
     85 
     86  $ easy_install nappy 
     87 
     88Now the 'csmlscan' tool can be used to create a CSML document to describe these files:: 
     89 
     90  $ csmlscan -f PointSeries -L '52.42 -4.05' -d /home/dlowe/data/WindFeb09 -o mycsml.xml  
     91 
     92 
     93What do the command line arguments to csmlscan mean? 
     94 
     95  -f PointSeries 
     96   This flag tells the scanner to create PointSeries features. 
     97 
     98  -L '52.42 -4.05'  
     99 
     100   This flag is used to specify a latitude longitude location for the PointSeries. Unfortunately in this case we have to manually supply it as NASAAmes metadata is not consistent between files, so a computer can't read it easily. Look in the NASAAmes file again and you can see:: 
     101 
     102    Location name: Frongoch farm (near Aberystwyth, UK) 
     103    Location: 52.42 degrees N, -4.05 degrees E 
     104              base of tower 140 m above mean sea level 
     105 
     106  -d /home/dlowe/data/WindFeb09 
     107 
     108   d is for 'directory' where the dataset resides 
     109 
     110  -o mycsml.xml 
     111 
     112   o is for 'outputfile' where your CSML document will be created 
     113 
     114 
     115So you should now have a CSML file, 'mycsml.xml'. Open it in a text editor and have a look over it. The XML should contain 4 PointSeries features for the entire month's data. 
     116 
     117 
     118Experimenting with the CSML API 
    70119------------------------------------ 
     120CSML has a python based Application Programming Interface (API) to interact with CSML documents. We will explore this interactively in ipython. 
     121 
     122 
     123Start ipython:: 
     124 
     125  $ipython 
     126 
     127and import csml:: 
     128 
     129  >>> import csml 
     130 
     131Now you can parse your csml file into a Dataset object:: 
     132 
     133  >>> ds=csml.parser.Dataset('mycsml.xml') 
     134 
     135and you can list the features in that file:: 
     136 
     137  >>> ds.listFeatureIDs() 
     138 
     139 
     140select a feature by id and assign it to a varible 'f':: 
     141 
     142  >>> f=ds['Mean_eastward_wind_in_1_minute_sample_period'] 
     143 
     144you can now look at the data in that feature:: 
     145 
     146  >>> f.getLatLon() #get the lat lon of the station 
     147 
     148  >>> len(f.getTimes()) #see how many times there are  (len="length") 
     149  >>> len(f.getDataValues()) # see how many data values 
     150 
     151  >>> t=f.getTimes()[0:10]   #Assign the first 10 time values to a variable, t 
     152  >>> v=f.getDataValues()[0:5] #Assign the first 10 time values to a variable, v 
     153  >>> zip(t,v) #display them together 
     154 
     155There is also a built in method getSubsetOfData() which allows you request data within a time range. 
     156 
     157e.g. Get all the data for February 5th 2009:: 
     158 
     159  >>> f.getSubsetOfData(('2009-02-05T00:00:00.0', '2009-02-05T23:59:59.59')) 
     160 
     161 
     162 
     163Installing COWS ready to setup OGC services 
     164-------------------------------------------------- 
     165 
     166You can install COWS using easy_install as before:: 
     167 
     168  $ easy_install cows  
     169 
     170Again this command will install the egg (cows) and all it's dependencies - it will check 
     171you have CSML installed - you should already have from earlier, but if not it will install it  
     172too.  
     173 
     174So we could equally have got to this stage and installed cows, csml & ipython with a single command:: 
     175 
     176  $easy_install cows ipython 
     177 
     178 
     179Generating CSML for the test gridded dataset 
     180---------------------------------------------- 
    71181 
    72182Some CF-NetCDF has been placed in /home/spascoe/ds_walkthrough/data. 
    73183!TODO: describe datasets.  To generate CSML for this data use the 
    74 ``csmlscan`` tool.  First create a configuration file: 
     184``csmlscan`` tool again. This time we will use a configuration file 
     185rather than command line arguments. First create a configuration file: 
    75186 
    76187**hadcm3.cfg**:: 
     
    107218  ******************************************************************** 
    108219 
    109 Interacting with the CSML API 
    110 ----------------------------- 
    111  
    112 !TODO 
     220In this case, if you look at the CSML file you will notice that all the data is not  
     221stored inline in the CSML, but is kept in the original NetCDF files and referenced via 
     222'NetCDFExtract' elements in the CSML document. 
     223 
     224Interacting with GridSeries data via the CSML API 
     225---------------------------------------------- 
     226 
     227As before you can parse the csml file, list the features and select one: 
     228 
     229  >>> import csml 
     230  >>> ds=csml.parser.Dataset('csml/hadcm3.xml') 
     231  >>> ds.listFeatureIDs() 
     232  >>> f=ds['air_temperature'] 
     233 
     234This time you can look at the 4D spatio-temporal domain of the grid:: 
     235 
     236  >>> f.getDomain() 
     237 
     238And you can subset the GridSeries to a smaller GridSeries feature:: 
     239 
     240  >>> outputdir='./' 
     241  >>> outputfile='mygrid.nc' 
     242  >>> subsetDictionary = {'latitude': (-10, 10), 'longitude': (55, 65), 'time':('2400-01-00T00:00:00.0', '2410-01-00T00:00:00.0')} 
     243  >>> f.subsetToGridSeries(outputdir, outputfile, **subsetDictionary) 
     244 
     245As you can see this method writes the new grid out as both a NetCDF file and as a new CSML feature. Come out of ipython and run 'ncdump' to view the headers of your new grid:: 
     246 
     247  ncdump -c mygrid.nc 
     248 
     249note, there are other methods for subsetting a GridSeries feature to other feature types such as PointSeries or Profiles. 
    113250 
    114251 
     
    188325Try visiting ``http://cirrus.badc.rl.ac.uk/<username>/``.  Note the 
    189326trailing slash which is important.  You should see the COWS server 
    190 catalogue. 
     327catalogue and can try out the WMS demo. 
     328 
     329Accessing the Web Coverage Service (WCS) with OWSLib 
     330----------------------------------------------------- 
     331The WCS can be accessed using any standard WCS client. Unfortunately WCS is not as mature a standard as WMS so not many of these have been built yet! 
     332 
     333There is however an Open Souce python OGC client library called OWSLib which we will use to communicate with the WCS. Again we need to install this with easy_install:: 
     334 
     335  $easy_install owslib 
     336 
     337Start ipython again and make a connection to the WCS: 
     338 
     339  >>> from owslib.wcs import WebCoverageService 
     340  >>> wcs=WebCoverageService('http://cirrus.badc.rl.ac.uk/dlowe/hadcm3/wcs',version='1.0.0') 
     341  >>> print 'Accessing WCS version %s at %s'%(wcs.version, wcs.url) 
     342 
     343The OWSLib wcs 'identification' object contains general information about the service (some of which is missing in this service!): 
     344 
     345  >>> wcs.identification.service 
     346  >>> wcs.identification.title 
     347  >>> wcs.identification.abstract 
     348  >>> wcs.identification.keywords 
     349  >>> wcs.identification.fees 
     350  >>> wcs.identification.accessConstraints 
     351 
     352The OWSLib wcs 'provider' oject contains general information about the service provider:: 
     353 
     354  >>> wcs.provider.name 
     355  >>> wcs.provider.url 
     356  >>> wcs.provider.contact.name 
     357  >>> wcs.provider.contact.email 
     358  >>> wcs.provider.contact.organization 
     359  >>> wcs.provider.contact.address 
     360  >>> wcs.provider.contact.city 
     361  >>> wcs.provider.contact.region 
     362  >>> wcs.provider.contact.postcode 
     363  >>> wcs.provider.contact.country 
     364 
     365 
     366Now take a look at the available coverages:: 
     367   
     368  >>> wcs.wcs.contents 
     369 
     370And select a coverage from the list:: 
     371 
     372  >>> cvg=wcs['air_temperature'] 
     373 
     374Investigate it:: 
     375 
     376  >>> cvg.boundingBoxWGS84 
     377  >>> cvg.timepositions 
     378  >>> cvg.timelimits 
     379  >>> cvg.supportedFormats 
     380  >>> cvg.supportedCRS 
     381 
     382Now we know a bit about the coverage we can make a request to get a subset of it from the WCS:: 
     383 
     384  >>> response=wcs.getCoverage(identifier='air_temperature',time=['2992-11-16T00:00:00.0'],bbox=(-80,30,50,60), crs='WGS84', format='cf-netcdf') 
     385 
     386You can write this response to a file: 
     387  >>> f=open('mywcsoutput.nc', 'wb') 
     388  >>> f.write(response.read()) 
     389  >>> f.close() 
     390 
     391Come out of ipython  and use ncdump again to investigate your WCS response:: 
     392   
     393  ncdump -c mywcsoutput.nc 
Note: See TracChangeset for help on using the changeset viewer.