Version 83 (modified by selatham, 14 years ago) (diff)


NDG Closedown Meeting (draft) 14th November 2007

  • NDG History (NDG1/2)
  • NDG2 'The plan'
  • NDG2 Objectives - have these been met?
  • What is left to clear up as part of NDG2 (end-of-Dec 2007)
    • Moving discovery to Production Service (NERC Data Discovery Service)
  • What goes over to OMII?
  • What goes over to NERC Portals?
  • Is there anything left with no-where to go?
  • Finances
  • Lessons Learnt
  • Any other reporting requirements.

07th November 2007

  • CSML progress - BODC/PML/NOCS
  • Browse activity discovery, related entities of the same type.
  • Service Binding / Related URLs
  • Security

17th October 2007

  • Bryan, Stephen not present
  • BODC
    • Vocab 1.1 WS service is up and accessible internally, but there are problems accessing outside firewall. BODC Web manager looking into it. Geoff is checking status. Advise that Bryan checks every so often. The test clients work though.
    • Browse is working.(although some BODC records do not have MOLES and this situation isn't handled very well)
    • Phil will update their pylons stack on LivJade? remotely.
    • Liz Bradshaw is working on suitable subsetting for th Nat Tide Gauge dataset.
    • Roy working on spec for gettng their MOLES to 1.3 and MOLES-to-granules.
    • Ray getting up-to-speed on Siva's work particularly the MOLES creation code.
    • Sue to visit Ray at BODC next Tuesday.
  • NOCS
    • Problems with Cdat-lite install. Phil will try installing full CDAT at NOCS (in their ndg virtual host).
  • Plymouth
    • Peter to ticket issue with Browse not showing related DataEntities? when viewing a DPT, ObsStn? or Activity.

3rd October 2007

  • Sue, Bryan, Dom and Stephen away
  • Focus on security issues:
    • Phil to be away in next few weeks on paternity leave
    • security documentation is being updated. Phil has created pages with epydoc see:
    • PML status: Mike is likely to have some time free now to concentrate on completing it
    • BODC: Phil has remote access re-established. Ray will be working on security - get up to speed when Phil returns from leave
  • Other issues:
    • PML: Peter - records don't seem to have updated correctly - still 'seeing' 6 month old records. Old ones start with 'All' eg. 'AllSeaWiFS...'
    • Browse service is not available - Sue/Matt/Bryan??
    • Peter would like to see e.g.s showing relationship data granules to data sets and how to define the boundary

29th August 2007

  1. What is outstanding for Pylons Discovery/Browse/Security?/WMS/WCS installs?
    • BADC Andrew Harwood doing this with help from Bryan
    • PML - Ports, Certificates,
    • BODC - Phil tried security session manager port from BADC yesterday but still unavailable
    • NOCS - Phil has permissions but needs sudo root access to install services + co-ordinate with Bruce for MyProxy install on db machine
    • #859
    • BODC pylons install somewhere accessible
    • BODC still security session manager/port problems
    • PML open port
    • PML security instructions, re-create certificates
    • NOCS Bryan/Phil? have permissions.
  2. Content
    • PML MOLES-DIF-CSML metadata production pipe broken?
  3. CSML
    • Dom visited NOCS - progress on HDF reading
    • Siva coding !raggedSection features
    • Trajectories need testing in WCS
  4. Issues of note

22nd August 2007

  1. What is outstanding for Pylons Discovery/Browse/Security?/WMS/WCS installs?
  • Bryan's egging /config work
  • BODC Port for Session Manager
  • BODC pylons install somewhere accessible
  • PML open port
  • PML security instructions, re-create certificates
  • NOCS let Bryan/Phil? do install?
  1. Content
  • esp for testing WCS/WMS
  1. CSML
  • Dom visited NOCS
  • Has Siva started creating CSML now that FeatureTypes? sorted?
  1. Issues of note

8th August 2007

I expect work to have progressed on

  • Discovery front-end including OpenLayers? map,
  • integration of security
  • and therefore improved egg for install
  • possibly PML, BODC trying install.

25th July

  1. Metaticket covered?


  1. Installation of Discovery/Browse? pylons version (as-is) is priority for DataProviders?. Note the old version is to be ditched everywhere, including glue.
  2. Phil to create instructions for re-starting pylons version of Security now on glue (for when he is on holiday).
  3. Fabio to document/hand over/ review tickets.
  4. Siva prompt Roy re. use case for vocab search.
  5. BODC to identify a/some test datasets which they can create full set of metadata for.
  6. Helen progress HDF read methods as matter of urgency.
  7. Sue arrange handover of MOLES XQueries with Kev and Steve Donegan.

18th July


  1. Dominic to look into issues with config files and egging in the pylons stack.
  2. Discovery is included in the stack - is this a problem? Can discovery be turned off? Sue raise as issue for Bryan.
  3. Phil still working on Security integration to the stack.
  4. Dominic will put new stack in startup for glue.
  5. Sue/Dominic/Phil? fix on ports for services. All DP's will need in their configs for cross DP browsing etc.
  6. BODC Moles records are still 1.2.4. Will this cause problems at Browse? Sue raise issue.

11th July 2007


  1. Pylons installation - Now that python2.4 and 2.5 installed on glue, Dominic to install pylons stack on glue (Phil will do Security bit). Then write wiki install page for DataProvider to reference.
  2. DataProviders should then install the stack and do local configuration as required.
  3. Pylons stack includes:-
    1. Discovery
    2. Browse
    3. Security
    4. WCS
    5. WMS and matplotlib
    6. cache management(bespoke - not pylons version at the moment)
    7. CSML
    8. paste, wsgi, numeric
  4. Security will require ports to be opened at DataProviders. See Phil's e-mail.
  5. NOCS CSML production is a blocker issue. Helen to progres.
  6. Plymouth (now called NEODAAS) - Peter needs to populate the security roles (and do the mapping?).
  7. Sue investigate if there's a bug in Discovery display of temporal info(there's a ticket).
  8. Note Holidays/away July - Peter, Helen, Stephen, Bryan, Roy for some/all of rest of July.

4th July 2007


  1. Bryan - email new config settings around (and anything else reqd) to get browse to talk to BODC exist.
  2. Phil - install python2.5 on glue
    1. Then new pylons version of discovery,browse,security, WCS, WMS, cache can be added to the stack.
    2. Then egg can be created for installing the lot.
  3. Everyone - CSML needs to be populated and link to actual data. Where will CSML go. Where is data.

20 June 2007

  1. Browse status
  2. WMS & WCS
    1. status
    2. Integration(browse)
    3. metadata requirements for testing
  3. NDG Closedown timetable
  4. decide about next Wed mtg


  1. Bryan - email new config settings around (and anything else reqd) to get browse to talk to BODC exist.
  2. Mike - let us know when PML exist has been moved and should be working.
  3. Fabio - chase up port opening for exist at NOCS.
  4. Bryan, Dom, Stephen - discuss cache management.
  5. All - check your alpha security role mappings are still valid.
  6. All - cancel next week's AG (27th June)

13 June 2007

  1. Security - Critical Path for Beta+Security

All-hands 4th June 2007. Venue Southampton. Time 10:00am to 3:30pm.

  1. Project Status Issues:
    1. Financial and staffing situation
    2. ...and therefore Project Timescale
  2. Technical/Deployment? Status Reports plus potential deliverables
    1. Discovery including NERC and MDIP status (Sue & Bryan )
    2. ...closely followed by Browse (BNL & All)
    3. Vocabulary Services (Roy)
    4. CSML2 (Dominic)
    5. Data Services & Visualisation (Stephen & Dominic)
    6. Security (Phil)
    7. Metadata population - Discovery, Moles and CSML (All)
  3. Lunch
  4. Collaborations requirements (Andrew lead)
    2. NERC Discovery Service
    3. MDIP
  5. Deliverables discussion
    1. inc. indentification of blockers
  6. Decisions/Timetabling?
    1. re-hash tasks etc on trac as go
    2. what do we do in NDG2?
    3. What's left for futures?
  7. AOB

30th May 2007

Note - Sue and Bryan can't make this meeting. If you need them to answer questions then note them for later e-mail or ticketing. Mike away too.

  1. Browse is working with BADC (with simplified xqueries and tuned exist db). Need to get browse working with other DPs ASAP/by Monday. Are there any outstanding obstacles?
    1. PML has working browse, but needs info on tuning exist and is missing some exist libraries ("voclib"?).
    2. BODC has also got working NDG-Browse on a local Sandbox, and copied "xqueryLib" Collection from SVN.
  2. Sample data for CSMLing/overlay. Have all DPs got an idea of what target data they will be CSMLing?
  3. AOB

16th May 2007

9th May 2007

  1. Browse (Aim: to have PML browse working via MOLES basic rather than stub-B)
    • Report (Bryan doesn't think he can make this meeting): As discussed on list, we can't really use MOLES basic either, as that has no names. Kev working on simplified xquery, which we hope we'll have asap. See how that goes.)

2nd May 2007

  1. Report on CSML service meeting last Friday (OwsNotes)
  2. Security schema and MOLES (held over from last week)
  3. Doing away with Stub-B
  4. Browse Status:
    1. Code (need to get rid of stub-b)
    2. Deployment (code:PML, BODC in sandbox, SOC nearly Mod_Python problems, BADC Chinook unstable)
    3. Content (PML best for testing)
  5. Cataloging Coordinate Reference Systems

25th April 2007

  1. Separate security schema from MOLES?
  2. Browse Status
  3. Finance status, end date of current activities.

18th April 2007

  1. DX GUI #647
  2. Eggs and software control
  3. Update on status of security package:
    1. Egg status
    2. DEWS
      • delivery of large datasets
      • Java clients and use of proxy certificates
    3. MyProxy as a  Certificate Authority
    4. NDG readiness: Session Manager, Simple CA, Logging and Login services (PHP)
    5. AHM Paper
  4. (mggr->dom)  CSML raw extract proposal
  5. FastCGI /modpython installations
  6. Status, clarification for ServiceBinding

11th April 2007

No meeting

4th April 2007

  1. Discovery status update.
  2. Objectives for Roy at April 19(30th?) MDIP meeting.
  3. Status, clarification and action plan for implementing ServiceBinding.
  4. FastCGI /modpython installations
  5. NDG security installation status (inc twisted)
  6. CSML status - tooling, creation at !DPs
  7. Planning rapid development of DX/Geosplat replacement. (point to DX/Geoslpat in meantime.)
  8. Date of next meeting (who's away Easter week?)

28th February 2007

  1. Part of the Discovery refactoring is the change across to the official vocab namespaces. This means that all MOLES producers should use these.

14th February 2007

  1. Mod_Python status at SOC?
  2. Completion of SystemStatus page?
  3. Status of MOLES refactoring and ticket destruction?
  4. Status of ServiceBinding?

7th February 2007

  1. Working Grid
    1. Information Status
      1. Where is everyone in their metadata production?
    2. Status and Plans for MOLES
      1. Now that glue has discovery  up, we find that the mini-MOLES conversion have a lot of problems. For examples:
        1. Location keywords become paleo whatsits. (see any NEODC record before and after)
        2. Parameters do not survive the conversion. (see any record).
        3. Actions required: everyone to look at docs and advise Kev on specific problems (this week). Kev to refactor xqueries to support easier adjustment (this week). Kev to fix as many as possible by next week.
    3. Status and plans for discovery
      1. Need to get service binding sorted, ensure MOLES can support it, and ensure that the transformations keep it, then, and only then, will work on browse.
      2. Actions: Bryan to provide service binding final decision (this week).
      3. Status of FastCGI at data providers?
  2. Documentation:
    1. Status of the two-pager. Bryan has provided a revised version to Helen. What next?

Meeting Outcomes

  1. Bryan to finalise ServiceBinding (this week)
  2. Bryan to produce a table of metadata and system status for updating by all parties (DONE)
  3. Bryan to investigate wsgi and mod_python to avoid fastcgi problems at soc (and probably elsewhere). Other options include proxying to lightpd or something else. (DONE, mod_python should be ok, see comment on ticket:639)
  4. Kevin to complete refactoring of xqueries into exist libraries (this week)
  5. All to provide examples of problems with lossy conversions into or out of MOLES (we expect lossy conversion out of MOLES where MOLES has more than the output format, but where the MOLES content ought to end up in the output, we need to tell Kev about it). All problems to be raised as specific tickets. (this week and ongoing)
  6. Sue to get MDIP examples into the discovery exist (this week)
  7. Kev to fix as many of the ticketed problems as possible next week, prioritising any MDIP ones (by next meeting)
  8. Helen to ring Bryan re QUEST and the two-pager (asap).
  9. Sue and Bryan to further investigate memory problems in the discovery ingestion.