NDG2 Plymouth 11-12 July 2006 - Peter's notes

Board Meeting

  • Finance - overall exactly on track with budget.
  • Highlight report - mostly on schedule at post-Alpha review. WG group were instrumental in Alpha system integration testing.
  • Next three months: need detailed definition of Beta products. Start playing with Vocab server? Discovery back end is broken, Matt Prichard to work on this. Population!
  • CSML future: Andrew presentation (CSML API and v2). CSML will be used by MetOffice? and other projects, e.g. FP6 MOTIIVE. Read Exeter communique on OGC etc. ( Development of 'feature types'. GML stable version 3.2. 'Operational affordance' features have to say what they can do. Can only extend one feature type but can inherit properties from multiple...? CSML will deliver core set of base types, derived feature types, basic services (api), schemas, tooling (parser/scanner).
  • So what should DPs do towards CSML? NOCS will be adapting scanner to deliver their HDF data rather than CF-netCDF. BODC is using QXF-read method from Matlab/Java?. Implement CSML v1 for now, then modify for v2. Swath very soon. Parser lets us play with CSML records prior to DX working - use that to help us write our scanner. If going from data->CSML, must write scanner in Python as this encodes the objects and attributes. If going from metadata->CSML, just need to plug in SQL extraction to Python scanner.
  • MOLES future: MOLES is for generating various discovery formats (and for metadata navigation), and will not be driven by external standards e.g. SensorML. x-links to include other docs (XHTML, SensorML, ...) verbatim within MOLES records, e.g for BODC calibration info. Could simplify/thin MOLES... Bryan proposes generating CSML data objects first, then flesh out MOLES from those (then DIF). Several issues to be sorted: Granules and crossover with CSML; schema for Stub-B; add coathanger facility; export to ISO19139; ...
  • Kerstin's job has changed, so Andrew taking over role.
  • MOLES Browse (python) knowledge transfer from Bryan to Dominic/Fabio?. Next stage is to deploy at DPs, configuring.
  • OAI issues with deletion in eXist. Keep with DLESE version or change to...?
  • e-Science All-Hands - no NDG papers... Poster on niESS stand/demo of MOLES browse - Roy certain days?, Mike G all days (free person for stand), Helen to provide poster (from GO-ESSP ppt), Helen/Fabio?, Michael H, Steve P?
  • Potential loss of Ag time on DX due to proposal to DEFRA (Hadley Centre climate impact visualisation), may involve NDG technology. If so Steve Pascoe to take on DX.
  • Need for documentation becoming more pressing...
  • Review outstanding tickets: WG team to lead integration, requesting and testing installs, developing installs that actually work for DPs; move away from central NDG portal, as every DP will have NDG search web service? combine browse/discovery GUIs; remove extra Trac milestones added by individual Products.
  • Future funding? Bryan is now on NERC Information Steering Group (ISG). So Discovery portal will be funded. NERC FAB - flexibility and agility in funding. NDGx...? No chance for NERC Consortium. JISC? Bi-partite proposals. Maintain NDG as club, possibly self-funded. Won't get more funding for population, but maybe for technical issues.

All-Hands Meeting

  • Delivery service (Steve P): ppt run through - bbFTP API in Python, transferring raw files or CSML instances including NDG authentication. Demo on Glue of ftp-like command-line client. Later pybbftp Python functions instead of command-line. May need further web-service DS to add functionality.
  • Vocabulary server (Michael/Roy?): 'what-lists <keyword>?'; populated with BODC Parameter Discovery, BODC Parameter Usage, GCMD Terms, SeaDAT, CF Terms later. Outputs XML for formating with stylesheet. Simple client only on livglue (Liverpool), but WS live at Roy can make available code for clients to install locally.
  • MOLES Browse GUI (Bryan) demo on Glue ( Links in with Ag's DX service, asking you to log in to your local site to give you NDG authorisation to continue into GeoSPLAT. Has security e.g. for COAPEC data.

Post-Alpha Review

  • Went through all pre/post/alpha tickets in priority order.
  • Keep eXist external access (through firewall), in addition to WS...?
  • Next milestone is External Product Definition, 31 Oct. 2006.

Project planning for Beta

  • CSML: All re-engineering for CSML v2 will be finished by pre-Beta. CSML generation fairly easy using Dominic's Scanner and Python.
  • MOLES: bounding box extension to cope with over-pole, PML may be asked for example.
  • XLink: links with attributes as to what you are expected to do with them, e.g. unidirectional, two-way, include image etc. (
  • Use cases (CompleteUseCases): working through Sue's CombinedUseCase.
    • Best practice is to generate CSML first, then MOLES. Scanner available for CSML but nothing generic for MOLES.
    • Talked about CSML read methods for various data formats, will hold 'Binary image format mtg' between Dom, Ag, PML. There was already a request from someone for '8-bit + scale' as a typical format. I asked for test rig, though Parser already has a 'ncdump' facility, could have 'ncview' for image preview. Ag said maybe data should be converted on-the-fly to CDAT rather than netCDF?
    • Data Extractor DX will be installed on DP, otherwise have to transfer entire files to user when they only need subsets.
    • Delivery Service DS will be installed on DP, if they want to deliver high-bandwidth data. Exploring OGC WCS and WFS web services.
    • Visualisation GeoSPLAT can be run local to DP (to get branding and logging) but doesn't have to be.
  • Documentation: HowTo - e.g. PML say how to generate MOLES/DIF from RDBS, in Wiki or existing doc format; ExternalProductDefinition - an attempt to determine what will be released in NDG2 so what will need documenting.
  • Next meeting: 3-4 Oct. 2006.

Peter actions

  • Re-read PML sample CSML - on Composite maps should be called SST, not CompositeSST.
  • Test out MOLE Browser on Glue -
  • Investigate CrossOver? Office for Linux for $60 where you can run MS Office ( which Bryan uses. Bryan successfully uses Evolution for Exchange on Linux. Also Citrix for Windows on Linux, or Terminal Server.
  • Circulate MetOffice? communique on OGC to Steve/Mike?.
  • Check Mike going to e-Science AHM - holiday/interrisk mtg conflict?
  • Get access to or install our own NDG vocab client - ask Michael/Roy?. See #416.
  • All pre/post/alpha tickets must be closed or moved to next milestone.
  • Meeting Mike/Mark?/me to press ahead with population - when?
  • Do we need Feature Type in our metadataDB to pass through to MOLES records?
  • Cross-domain cookies (#383) - we need those for NEODAAS too, ask Phil how to.
  • See here for UML tutorial and free editor:
  • Discussions with Helen - will we generate CSML on the fly and/or store records in eXist?
  • Find out about Delivering Environmental Web Services (DEWS) - DTI funded project involving NERC. (
  • Talk to Dominic about how to generate CSML. We will organise 'Binary image format mtg'.