source: TI09-UKCollaboration/trunk/DS_Workshop/doc/walkthrough.rst @ 5100

Subversion URL:
Revision 5100, 5.3 KB checked in by spascoe, 11 years ago (diff)

Material for the NDG3 Data Services Workshop to be held on March 18th

Scientific Data Services Workshop: Day 2

This tutorial will walk you through creating a combined WMS/WCS server for NetCDF data conforming to the CF conventions using the CEDA OWS Framework (COWS).

Setting Up your environment

COWS is written in Python and requires a UNIX-like environment. In order to run the examples in this workshop we have configured a server at RAL that everyone can log onto and run a COWS web server. The server is Your username and password have been given to you on a separate sheet.

To log into cirrus use the putty application. !TODO: more details

NOTE:: Although not required you might find it useful to enable the X Windowing system for your putty session. To do this goto the tunneling section of the putty configuration and enable X11 forwarding.

The workshop will guide you through how COWS is installed. The first time you login you should create an isolated Python environment into which you will install the required components. We will use the virtualenv tool for this:

cirrus$ virtualenv .
New python executable in ./bin/python2.5
Also creating executable in ./bin/python
Installing setuptools............done.

Each time you login you need to import the virtualenv settings with the source command:

cirrus$ source ./bin/activate

NOTE:: Experienced UNIX users might want to put this in your ~/.profile file.

Most COWS components can be automatically downloaded and installed from the internet with the easy_install command. Components installed with easy_install are called eggs. There are a few eggs that need manual compilation. Therefore, for the purposes of this workshop all eggs have been gathered together in the directory /home/spascoe/ds_workshop/eggs. To tell easy_install to look in this directory create the file .pydistutils.cfg in your home directory:


find-links = /home/spascoe/ds_workshop/eggs

You can now install COWS along with the improved Python shell ipython. All COWS dependencies will be installed automatically:

$ easy_install cows ipython

Generating CSML for the test dataset

Some CF-NetCDF has been placed in /home/spascoe/ds_walkthrough/data. !TODO: describe datasets. To generate CSML for this data use the csmlscan tool. First create a configuration file:


type: GridSeries
number: many
root: /home/spascoe/ds_workshop/data/hadcm3
mapping: onetoone
output: ./csml/hadcm3.xml
timedimension: time

Create the destination directory csml and run csmlscan:

$ mkdir csml
$ csmlscan -c hadcm3.cfg
CSML file is at: ./csml/hadcm3.xml

Interacting with the CSML API


Create CowsServer

COWS is built on the Pylons web framework. Most command-line tasks in Pylons are done through the paster command. When COWS was installed it automatically makes available the cows_server template to paster:

$ paster create --list-templates
Available templates:
  basic_package:   A basic setuptools-enabled package
  cows_server:     A Pylons template to create CSML-enabled COWS server
  paste_deploy:    A web application deployed through paste.deploy
  pylons:          Pylons application template
  pylons_minimal:  Pylons minimal application template

To create your server use the paster create command:

$ paster create -t cows_server CowsServer csmlstore=$HOME/csml

Your new COWS project will be created in the CowsServer directory. You can now explore the directory structure. !TODO: basic Pylons outline?

Edit CowsServer/development.ini to set the host as and the port. Follow this general template:


# Change the host and port below
use = egg:Paste#http
host =
port = <port>
# This section must be added to make the server visible outside RAL
use = egg:PasteDeploy#prefix
prefix = /<username>
# Note the filter-with section must be added
use = egg:CowsServer
full_stack = true
cache_dir = %(here)s/data
filter-with = proxy

Replace <username> and <port> with values from the table below

Username Port
test 5010
nsb 5011
nico 5012
mlcu 5013
njcu 5014
mase 5015
wael 5016
lq 5017
pjk 5018
mggr 5019
petwa 5020
aprc 5021
monz 5022

You are now all ready to start your server. run the command:

$ paster serve development.ini

Try visiting<username>. You should see the COWS server catalogue.

Note: See TracBrowser for help on using the repository browser.