source: TI09-UKCollaboration/trunk/DS_Workshop/DS_Walkthrough.txt @ 5100

Subversion URL: http://proj.badc.rl.ac.uk/svn/ndg/TI09-UKCollaboration/trunk/DS_Workshop/DS_Walkthrough.txt@5100
Revision 5100, 5.1 KB checked in by spascoe, 11 years ago (diff)

Material for the NDG3 Data Services Workshop to be held on March 18th
2009.

Line 
1
2= Scientific Data Services Workshop: Day 2 =
3
4This tutorial will walk you through creating a combined WMS/WCS server
5for NetCDF data conforming to the CF conventions using the CEDA OWS
6Framework (COWS).
7
8== Setting Up your environment ==
9
10COWS is written in Python and requires a UNIX-like environment.  In
11order to run the examples in this workshop we have configured a server
12at RAL that everyone can log onto and run a COWS web server.  The
13server is {{{cirrus.badc.rl.ac.uk}}}.  Your username and password have
14been given to you on a separate sheet.
15
16To log into cirrus use the {{{putty}}} application.  !TODO: more details
17
18NOTE:: Although not required you might find it useful to enable the X
19Windowing system for your putty session.  To do this goto the
20tunneling section of the putty configuration and enable X11 forwarding.
21
22The workshop will guide you through how COWS is installed.  The first
23time you login you should create an isolated Python environment into
24which you will install the required components.  We will use the
25{{{virtualenv}}} tool for this::
26
27{{{
28cirrus$ virtualenv .
29New python executable in ./bin/python2.5
30Also creating executable in ./bin/python
31Installing setuptools............done.
32}}}
33
34Each time you login you need to import the virtualenv settings with the {{{source}}} command::
35{{{
36cirrus$ source ./bin/activate
37(user)cirrus$
38}}}
39NOTE:: Experienced UNIX users might want to put this in your ~/.profile file.
40
41Most COWS components can be automatically downloaded and installed
42from the internet with the {{{easy_install}}} command.  Components
43installed with {{{easy_install}}} are called eggs.  There are a few
44eggs that need manual compilation.  Therefore, for the purposes of
45this workshop all eggs have been gathered together in the directory
46{{{/home/spascoe/ds_workshop/eggs}}}.  To tell {{{easy_install}}} to
47look in this directory create the file {{{.pydistutils.cfg}}} in your
48home directory::
49
50'''.pydistutils.cfg'''
51{{{
52[easy_install]
53find-links = /home/spascoe/ds_workshop/eggs
54}}}
55
56You can now install COWS along with the improved Python shell
57{{{ipython}}}.  All COWS dependencies will be installed
58automatically::
59
60{{{
61$ easy_install cows ipython
62}}}
63
64== Generating CSML for the test dataset ==
65
66Some CF-NetCDF has been placed in /home/spascoe/ds_walkthrough/data.
67!TODO: describe datasets.  To generate CSML for this data use the
68{{{csmlscan}}} tool.  First create a configuration file::
69
70'''hadcm3.cfg'''
71{{{
72[dataset]
73dsID:hadcm3
74
75[features]
76type: GridSeries
77number: many
78
79[files]
80root: /home/spascoe/ds_workshop/data/hadcm3
81mapping: onetoone
82output: ./csml/hadcm3.xml
83printscreen:0
84
85[spatialaxes]
86spatialstorage:fileextract
87
88[values]
89valuestorage:fileextract
90[time]
91timedimension: time
92timestorage:inline
93}}}
94
95Create the destination directory {{{csml}}} and run {{{csmlscan}}}::
96
97{{{
98$ mkdir csml
99$ csmlscan -c hadcm3.cfg
100...
101********************************************************************
102CSML file is at: ./csml/hadcm3.xml
103********************************************************************
104}}}
105
106== Interacting with the CSML API ==
107
108!TODO
109
110== Create CowsServer ==
111
112COWS is built on the Pylons web framework.  Most command-line tasks in
113Pylons are done through the {{{paster}}} command.  When COWS was
114installed it automatically makes available the {{{cows_server}}}
115template to {{{paster}}}::
116
117{{{
118$ paster create --list-templates
119Available templates:
120  basic_package:   A basic setuptools-enabled package
121  cows_server:     A Pylons template to create CSML-enabled COWS server
122  paste_deploy:    A web application deployed through paste.deploy
123  pylons:          Pylons application template
124  pylons_minimal:  Pylons minimal application template
125}}}
126
127To create your server use the {{{paster create}}} command::
128
129{{{ 
130$ paster create -t cows_server CowsServer csmlstore=$HOME/csml
131}}}
132
133Your new COWS project will be created in the {{{CowsServer}}} directory.  You can now explore the directory structure.  !TODO: basic Pylons outline?
134
135Edit {{{CowsServer/development.ini}}} to set the host as cirrus.badc.rl.ac.uk and the port.  Follow this general template::
136
137'''development.ini'''
138{{{
139# Change the host and port below
140[server:main]
141use = egg:Paste#http
142host = 0.0.0.0
143port = <port>
144
145# This section must be added to make the server visible outside RAL
146[filter:proxy]
147use = egg:PasteDeploy#prefix
148prefix = /<username>
149
150# Note the filter-with section must be added
151[app:main]
152use = egg:CowsServer
153full_stack = true
154cache_dir = %(here)s/data
155filter-with = proxy
156}}}
157
158replace <username> and <port> with values from the table below
159
160||'''Username'''||'''Port'''||
161||test||        5010||
162||nsb||         5011||
163||nico||        5012||
164||mlcu||        5013||
165||njcu||        5014||
166||mase||        5015||
167||wael||        5016||
168||lq||          5017||
169||pjk||         5018||
170||mggr||        5019||
171||petwa||       5020||
172||aprc||        5021||
173||monz||        5022||
174
175You are now all ready to start your server.  run the command::
176
177{{{
178$ paster serve development.ini
179}}}
180
181Try visiting {{{http://cirrus.badc.rl.ac.uk/<username>}}}.  You should
182see the COWS server catalogue.i
Note: See TracBrowser for help on using the repository browser.