Version 7 (modified by dacosta, 6 years ago) (diff)


CEDA CMIP5 Web Processing Service

The  CEDA Web processing service allows registered users to run a set of useful functions (processes) on CMIP5 data under a simple Web Service.
Please, look at this page to know how to become a registered user.

The available processes intend to give the user the ability to tailor a sample of the CMIP5 data for his specific needs by extracting spacial subsets, time slices, averaging and regrinding the CMIP5 raw data.
Please take a look at the detailed example below.

1. Available processes

The  Processes link on the  CEDA Web Processing Service home page will redirected you to the list of available processes you are allowed to run on CMIP5 files.
Please, note that the processes which name does not contains the string "CMIP5" are not designed to read CMIP5 data.
The available processes can be split into 3 categories:

A ) Processes designed to select CMIP5 file:
Select CMIP5 Files
Select CMIP5 File Group

B ) Processes designed for space and time manipulation of selected CMIP5 files:
CMIP5 Global Averager
CMIP5 Regional Extractor
CMIP5 Regridder
CMIP5 Time Averager
CMIP5 Time Concatenator
CMIP5 Time Extractor

C ) Self-contained processes:
CMIP5 Global Averager With File Selection
CMIP5 Regional Extractor With File Selection
CMIP5 Regridder With File Selection
CMIP5 Time Averager With File Selection
CMIP5 Time Concatenator With File Selection
CMIP5 Time Extractor With File Selection

2. Run a process

Whatever the process the user may choose to run, he will be asked to fill in a number of parameters required by the process.
Some processes need only one parameter.
Bellow is the list of all possible parameters:

File Parameters
Input File Paths a list of paths pointing to CMIP5 files.
File Group URL the URL to a File Group XML file that describes a subset of the CMIP5 archive.
Time Parameters
Start Date the start date of the time range required.
End Date the end date of the time range required.
Space Parameters
Spatial Subset a geo-spatial bounding box.
Regrid To the regular lat/lon grid on to which the outputs will be regridded.

Parameters Input File Paths and File Group URL need adicional explanation, which is given in "Select CMIP5 files" bellow.

After attributing to the parameters their values, submit your job by clicking the "Submit" button.

3. Select CMIP5 files

There are a number of ways to tell a process which files it should take as input:

a) Using Input File Paths parameter

For example, you can paste in the following paths:


This requires you to know the exact path of the CMIP5 files you want to use.

b) Using File Group URL parameter

You should paste in the URL address of a XML file containing the paths of the CMIP5 files you want to use, for example:

The best way to produce this file is by using the process called Select CMIP5 Files or the process called Select CMIP5 File Group .

c) Using Select CMIP5 Files and Select CMIP5 File Group processes

These processes allow the user to select CMIP5 files from a set of models and ensemble members for a given experiment, variable and time period.
These processes return a webpage.
Use the link "Send to" to send the selected files to one of the CMIP5 processes: "CMIP5GlobalAverager" or "CMIP5Regridder".
Copy the link location of the file "cmip5_file_group.xml" and paste into the File Group URL box of the process you intend to use.

d) Using "With File Selection" named processes like  CMIP5 Time Concatenator With File Selection

In these processes the user does not need to fill in the Input File Paths or to run Select CMIP5 Files or Select CMIP5 File Group processes to fill in the File Group URL box. The user only needs to select CMIP5 files from a set of models and ensemble members for a given experiment, variable and time period and then submit the job.

4. Submit your job

This is a two step operation:

a) You need to click the "Submit" button at the end of the process page to proceed.
In return you are given the details of your command and an estimation of the volume of the output files and the time needed to compute them.
Please note that these are estimates and should only be interpreted as such.
You will be then asked if you are happy with the details of your command.

b) If you are happy with the details of your request please click again the "Submit" button to submit your request to the WPS scheduler.
You will be forwarded to the "job viewer" page at this point.
Note that large jobs may take a long time so you may wish to navigate away from this page: the WPS will continue to run your job regardless.

5. Downloading output files

Once the job has completed, or if for some reason it failed during processing, the WPS will refresh the "job viewer" page with all information necessary to download your data or resubmit your job.

It also notifies you by e-mail.

The e-mail will include a URL that links to a page from which you can download the output files from your job, and various text files with your outputs: a request metadata, disclaimer and various log files.

6. Example

Suppose you want to have monthly atmospheric pressure at sea level and monthly the sea surface temperature over the North Atlantic region ([-30,30] longitude and [30,60] latitude) simulated by bcc-csm1-1-m and bcc-csm1-1 models for the years 1900-2000 in a 2 degree resolution regular grid.

This request involves an area extraction, a time extraction and/or concatenation and a regrinding of the data for the given models and variables. In order to have a fast processing of your request, which also means less power consumption, you may want to start by doing the operation that discard most data. In this case that would be extracting the North Atlantic region. Therefore, you need to start by using the CMIP5 Regional Extractor With File Selection process

You also need to specify "Experiment family" and "Ensemble Member" values in order to complete your request. In this case we will assume that you tick at "historical" and "r1i1p1". If you feel that these parameters are obscure, please take a look at CMIP5 Data Reference Syntax. After this, the request page of the Regional Extraction process should have the following outlook:


Note that the form allow you to choose more than one value only for "Models" and "Ensemble member".
In case you need to extract 2 or more "Variables", 2 or more "Experiments" or 2 or more "Time frequencies" you should repeat the extraction procedure for each possibility.
Also note that "Strict treatment of multi-valued options" and "Strict Time Conformance" are set to "False". This allow the process to work with data that agrees but do not excatly match the parameters you have set.
Soon after submitting your Regional Extraction process to the WPS you will be redirect to a page similar to the one bellow:

No image "RegionalExtraction._outputjpg" attached to ExArchProcessing/CowsDocForUsers/Overview.