wiki:WorkingGrid/DataProviderSetup/PML/POLJan07

Version 12 (modified by pmiller, 13 years ago) (diff)

--

Peter's notes on NDG Meetings at POL 17-18 Jan. 2007

PML Actions

  • Peter - Test Bryan's Discovery search (available end Jan), then Browse GUI.
  • All - Fix PML DIF (then MOLES) records using Discovery search.
  • Peter - Talk to Phil re cross-domain cookie advice for NEODAAS link-up.
  • Peter - circulate proposal for intelligent (Google) ordering of discovered records.
  • Mike - discuss with Andrew/Dom? how to progress 8-bit read methods.
  • Peter - discuss CSML2 Swath feasture with Andrew.

Project board

Issues

  • Progress: sluggish e.g. security, metadata production. Bryan says we should just go with the security development version if that works, even if this is a 'moving target'. Staffing problem with people getting dragged into other projects rather than NDG deliverables. Bryan says we should be driving the development now with science questions, which can now be asked at this stage, e.g. I want to compare these two datasets. Sue should be nastier about missed deadlines.
  • Staffing: Fabio may leave before end of NDG. Bryan says we should get no-cost extension, extend Andrew's work on CSML, ... Could be synchronisation issues between delayed components and DP effort.
  • DX: Steven Pascoe is re-engineering DX as a lightweight interface for DEWS, using mapplotter not CDAT. Ag has DX working for two feature types (overlay trajectory on plot), but must use Dom's CSML interface to hide the issues like a new CSML feature type for PML data files.

What is important to PML? We are doing OK with the metadata population (MOLES, DIF, CSML) - Sue says a week ago we jumped to 1,400 DIFs! (Bryan says granularity issues.) We need access to Browse interface to play with. Security is not as important as logging. We want to be able to extract/visualise our satellite data via CSML, DX, GeoSPLAT. If delivery via GeoSPLAT is problematic, what about OGC interfaces to WxS?

Lots of discussion (in Bryan's mindmap) on the technical questions that need resolving tomorrow, particularly regarding DX, CSML, ...

We will host services: Browse interface GUI, WCS- or DX-based backend, WCS- or DC-based frontend GUI (ie WMS Client or GeoSPLAT), ...

  • Metadata: Discussion on Roy's granularity issues, multiple related data entities for All Cruises->Individual cruises->CTD's, etc. Only the 'All Cruises' aggregated MOLES record will be discoverable as a DIF. Argh - Bryan hasn't implemented related entities in Browse (because he hasn't got a Stub-B schema...) so you can't actually get to the individual cruise yet! Suggested adding 'Discoverable' boolean parameter to each MOLES record.
  • I repeated my view that the DP deciding how to aggregate all their datasets is unlikely to be a successful approach, both in terms of getting a quality selection of DIFs from the DP, and in users being impressed with a Discovery search resulting in 'All cruises', 'All SeaWiFS data globally', etc., particularly if these have a gap for the time/region of interest. I believe all the MOLES records should have DIFs, and Discovery should do a better job like Google in indexing and ordering the results. E.g. if user searches 'chlor' then the aggregated datasets would be a sensible result, but if user searches 'chlor' + 1998 + North Sea then they should see a few matching cruises and a few closely matching satellite datasets (SeaWiFS North Sea 1998, SeaWiFS global 1998). Bryan thinks that this would be nice but won't be done in NDG2. I maintain there should be a relatively easy way to add this to Discovery, to avoid DP's having to mess with their datasets after they've gone to the trouble of releasing them. E.g. Are the DIF related record fields being used? Don't think so. Peter must write report if he wants this to happen in NDG2, and it must be really easy.
  • Risk review.
  • Issues review. The web-site integration with Dundee for NEODAAS I said is not an issue on NDG2 timescale. Even when the authentication system changes we can still keep the RSDAS website going for NDG purposes.

Continuation of board meeting

  • Documentation requirements: This was previously covered in the External Product Definitions mindmap, which needs Helen to prioritise and assign tasks.
  • See what QUEST are doing on QESA (Earth system atlas) - fancy website etc. Helen? go to QUEST open science meeting.
  • MOLES issues, if Kev is to leave project soon then his knowledge must be documented and transferred. There is a Stub-B schema in svn.

All-hands meeting

  • Rant from Sue about being behind on milestones.
  • Discovery website: demo of simple/advanced search. Revealed our records - 846 hits for temperature - immediate problems with data summary, lower case title, repository 'PIM Miller' and related record links. Need access to discovery search in order to debug these - Bryan says available on Glue by 25 Jan.
  • Security update (Phil): DEWS design. Using standard web services interfaces (e.g. Twisted) should be much more robust. Phil much happier with Beta, DP's like us should install that rather than continue with buggy Alpha. Something about role-mapping. Bryan suggests leaving Security out of the DX/etc. testing for few weeks, leave placeholders in code. Security Client stub code date? DP able to (re)install security date? Cross-domain cookie issues - talk to Phil for advice on NEODAAS.
  • Vocab server (Roy): new API v2 live bv end Feb.
  • CSML (Andrew): CSML2 powerpoint overview. Station name missing, or more general problem of passing relevant metadata through to CSML via 'metadata' tag, e.g. for finding and labelling in DX/vis stages. Documentation coming... though CSML1 doc was very comprehensive. DP's need to also talk with Dom for advice on CSML generation. Talk to Peter and Helen re Swath feature.

Peter's proposal for search grouping