Version 3 (modified by mggr, 13 years ago) (diff)


Working Grid Meeting notes: December 2006

Present: Sue, Helen, Siva, Mike, Stephen.

In spirit: Dom (has a cold, emailed in notes), Phil (away, updated status page).


List of actions

  • All DPs to update their setup pages as needed (PML, BADC, BODC?, NOCS)
  • Sue to ask Phil for a loose estimate of availability of an updated NDG security install to help with DP planning (ie. roughly January or March+).
  • Dom to see if the browse service provides appropriate secured access to CSML to meet DS/DX needs
  • Sue to raise delivery service to project board/AH for discussion (done, #613)
  • Stephen to send Helen notes on the current status of eggification
  • Helen to give us all another nudge on documentation


  • Try installing browse if anyone gets the time (and the Bryan)

Notes from meeting

Actions from last meeting

(all done)

Status updates

Points of note:

  • PML: Some work on security, now mostly waiting on newer security install. Working on getting updated MOLES available (soon). Updated eXist to 1.1.2. DP page needs update.
  • BADC: New MOLES catalogue implemented, people trained, population aimed for March. Otherwise mostly working on MDIP/discovery stuff. DP page correct but needs additional input.
  • BODC: Advised to wait for new security install for production machine. Roy restructuring MOLES backend database (mid Jan). New OAI & eXist on development server.
  • NOCS: Lots of backend work on restructuring metadata database - now capable of generating MOLES & DIF (some queries on specific elements to resolve), likely to generate records soon, but not in the 1000s. Alpha security in place, pre-alpha login code still around, but not tweaked yet - probably wait for new security install. DP page needs update.

Brief ticket review

  • Nothing especially stood out - everyone knows what they're doing. The main issues seem to be CSML2, MOLES (and other) population and (sometime) new security. Browse was also discussed a bit, with interest from several parties for local installs.

Discussion points

Security interlude

  • Phil made a wiki page detailing current status - TI12_Security/Security0612 - comments in the meeting included:
    • some minor discussion of post-NDG support
    • details on the progress of eggification from Stephen, also suggested that Twisted should be a pre-requisite rather than us eggifying it
    • some interest in DEWS' progress
    • general decision to wait for updated security installs prior to further DP work (as we're not yet serving restricted data it's not currently essential, so not blocking anyone directly)

Sue to ask Phil for a loose estimate of availability of an updated NDG security install to help with DP planning (ie. January or more like March).

CSML 2 status

Dom emailed in an update on CSML2, included verbatim here:

The V2 schema is more or less finished, but no sample instance documents
have been created yet. This is a process Andrew wants to do with data
providers (probably via email) and as a result of this the schema may
change slightly but not too much I hope. Andrew is away at an OGC
meeting this week so he won't get anything done on this before next
week at the earliest.

I've written a new parser for V2 CSML. So far it only handles GridSeries
and Point features. However the way I've written it means that it's
quicker to add classes now, so I hope to get all features supported in
the next couple of weeks. However the risk is that this is dependent on
any revisions Andrew makes to the schema.

There is therefore a worry that this will once again mean things are
tight for Ag for integration with the Data Extractor. For this reason
perhaps it's better that I concentrate on full parser/scanner/API support
for GridSeries features only and then finish everything post-Beta for
other features? This way we would have something working 'end to end' for
Beta, and we could make sure the schema is properly finalised before
writing too much code? Anyway, that's a discussion you might want to have.
  • Waiting on draft schemas/contact from Andrew
  • Complete parser support probably mainly a BADC decision, but Dom's suggestion sounds best

HDF read methods

  • re: blocker ticket #187 and access to HDF files
    • Fabio away, but will look at the ticket later
    • Some discussion on available libraries. Helen thinks they're likely to use their existing Fortran interface (and wrap it?).
    • OCCAM HDFs may be "impure" - possibly missing some metadata fields in the HDFs (available outside/a priori), so any read methods may not be useful for non-OCCAM HDFs.

Secured external access to CSML documents

  • Dom asked, re: #283 and external NDG-secured access to CSML documents, can the discovery service WS be extended to cope with this requirement or do we need a new interface? (verbatim text below)
    • Sue thinks that the browse service may provide this functionality - Dom to check?
      [WG] Should CSML (for BADC) be stored in eXist or in file system?
      I think we've agreed to store our CSML in eXist, but we need NDG-secured
      access. I.e. a service that fetches a CSML file from exist, checks the
      user credentials against the access control policy (stored in the CSML
      document) and returns the file if the user has the authority to read it.
      My WG question is this: can the discovery service WS Matt has written be
      extended for this, or do we need to write a separate interface? The
      service just needs to return whole documents based on ID (and I suppose
      it doesn't have to be a Web Service).


  • Helen noted there has been little/no activity in response to documentation team questions raised at the last all-hands, in particular the issue of which components belong in which logical "bundles" (types of DP)
    • Stephen to send Helen notes on the current status of eggification
    • Helen to give us all another nudge on documentation

Issues/possible problems

  • Delivery service
    • Sue to raise to project board/AH for discussion


  • Set date for next meeting
    • Following the metadata team's lead, evaluate the need for a meeting after the AH