Version 9 (modified by astephen, 15 years ago) (diff)


The Data Extractor Interface (CGI and Web Service)

The Data Extractor (DX) has already been through many code revolutions is likely to continue to evolve in the near future. Hence the interface exposed to the outside world may still be revised due to external and internal requirements.

When we talk about the "DX Interface" it is important to clarify what is meant by that. One potential interface to users and applications is connection direct to the DX Web Service. In this case its methods are called directly and returned objects must be unpacked accordingly. In the short-term, people are more likely to think of the DX Interface as the arguments that can be sent to the DX User Interface (CGI client) via HTTP POST or GET.

This document (currently under development) discusses both interfaces and welcomes your feedback!

1. The DX Web Service interface (WSDL)

The WSDL for the DX is still in flux and still needs an overhaul. The main issues are:

  1. The code has so far only been tested with python ZSI-enabled client/server. That has allowed some interesting data structures that might be hard to encode in other languages/SOAP libraries.
  2. Even ZSI is quirky - sometimes you have to package array items within their own length-1 arrays in order to pass them via SOAP.
  3. The old WSDL model was very simple, mainly concentrating on "getOptions()" and "selectOptions()" methods. These are no longer considered very useful so it is likely we'll move to the more explicit "getDatasetOptions()" etc etc.
  4. The client application needs better error handling from the server so all return values should include an error flag and a string that contains the error. This is planned but not implemented yet.

Since this page was created a very simple schema has been suggested for the DX WSDL.

Given its fluid nature that is all we'll say about the DX WSDL for now. More pertinent to NDG is what you can send to the DX client as HTTP keyword-value pairs.

2. The DX User Interface CGI Arguments

Within NDG2 it is most likely that users and applications will talk to the DX User Interface (DXUI). Hence they will need to know what arguments can be sent via HTTP POST or GET when some pre-selection has been made that is then pushed to the DX. The information below is still draft so comments are welcome.

Recently I have modified the DX server to just express the axes of any variable/feature as "axis_<datasetGroup_n>.<dataset_n>.<variable_n>.<axis_n>". This makes it much more flexible for different types of feature/variable. However, from the client end it is slightly less accessible so the DXUI needs more intelligence.

You can pass the arguments:

variable_<n>=<variable_ID>    # you can call this "feature_<n>" if you want

Then I reckon I'll have to get it parse spatiotemporal stuff based on:




Also, you can specify format with:


That is all pretty much off the top of my head and I'll have to add a bit of code in the client to do it. However, should work.

An example would be:

In this way you could pass multiple variables/features. Inside the DX, it labels them systematically as:


However, you probably don't want to get into that from an external viewpoint. You just want to follow the basic strategy in the above URL.

NOTE: it might be bad practice to stick square (or any) brackets in the URL. If so, we can get around it by using (as the DXUI already does to get HTML form data across) <axis_id>_<n>_low and <axis_id>_<n>_high as the names.