wiki:WorkingGrid/DataProviderSetup/PMLOct06

Version 11 (modified by mhenning, 13 years ago) (diff)

--

NDG Progress, as of October 2006 (mhen)

NDG/MOLES

  • The PML MOLES Database has been redesigned. Triggers and postgres-specific extensions (such as arrays) have been removed and relationships have been made explicit. This should make it (slightly) easier to understand, and also more portable. A graphical representation of the new database structure is attached.

  • The database can now be fully populated with data entities, by using a few perl scripts and unix tools.

mhen@pmpc828 ndg_moles # ./scanner.pl modis meris avhrr seawifs | sort | uniq | ./group_by_year.pl | ./ndg_moles_insert.pl

Currently, the database has one MOLES record for each year of data for a particular sensor and area:

mhen=# SELECT production_tool.description AS Sensor, COUNT(data_entity.*) AS RecordCount FROM ndg.data_entity, ndg.data_entity_deployment, ndg.production_tool, ndg.deployment WHERE deployment.production_tool_id = production_tool.production_tool_id AND data_entity_deployment.data_entity_id = data_entity.data_entity_id AND data_entity_deployment.deployment_id = deployment.deployment_id GROUP BY(production_tool.description);
 sensor  | recordcount 
---------+-------------
 AVHRR   |         617
 MERIS   |          32
 MODIS   |          81
 SeaWiFS |         260
(4 rows)

mhen=# 

(Total 1054)

NDG/CSML

  • CSML records can now be generated, by using a combination of perl and python scripts:

mhen@pmpc828 ndg_csml_gen # ./ndg_csml_granule meris200607zx | metatrans.pl --format NDG_CSML | ./ndg_csml_dataset_gen.py

The CSML record generated by the above scripts can be seen here:  meris200607zx

Attachments