Changeset 3020 for TI01-discovery

Timestamp:

16/11/07 16:32:51 (13 years ago)

Author:

selatham

Message:

New DP configs. Deal with colons in IDs

Location:

TI01-discovery/trunk/ingestAutomation/OAIBatch

Files:

• ceh_config.properties (added)
• oai_ingest.py (modified) (2 diffs)
• tpac_config.properties (modified) (1 diff)
• wdcc_config.properties (modified) (1 diff)

TI01-discovery/trunk/ingestAutomation/OAIBatch/oai_ingest.py

@@ -188 +188 @@
                 else:
                     new_filename = outdir + "/" +datacentre_namespace+ "__"+datacentre_format+ "__"+ ident +".xml"
+                    new_filename=new_filename.replace(":","-")
                 print "original file = %s, newfile = %s" %(original_filename, new_filename)
                 commandline = "cp "+original_filename+ " " +new_filename
@@ -231 +232 @@
 
 # Then run the minimum moles creator for each discovery record
-# Put records in ./DIF2MOLES with original filename
+# Put records in ./DIF2MOLES with same filename
 filenames = os.listdir(outdir)
 for filename in filenames:

TI01-discovery/trunk/ingestAutomation/OAIBatch/tpac_config.properties

@@ -1 @@
-# Define host_OAI as the string that OAI adds to the filenames after harvesting
-
-# BODC = oai%3Agrid.bodc.nerc.ac.uk%3A
-# SOC = oai%3Aoai.noc.soton.ac.uk%3A
-# NCAR = oai%3Aucar.ncar.scd.cdp%3A
-# PML = oai%3Apml.nerc.ac.uk%3A
-# TPAC = oai%3Atpac.org.au%3A
-
+#Note - don't have any blank lines in this config file
+# Define host_OAI as the string that OAI adds to the filenames after harvesting (not neccessary anymore?)
+#
 host_OAI=oai%3Atpac.org.au%3A
+#
+#Define host_path as the full directory name where this data centre's records will be harvested into.
+host_path /usr/local/jakarta-tomcat/webapps/oai/WEB-INF/harvested_records/ngportal-sf-utas-edu-au-dif
+#
+#Define groups - portal groups for limiting searches by 'group of datacentres'.
+#groups
+#
+#Define which format is harvested from the data centre (one only)
+format DIF
+#
+#Define the data providers namespace
+namespace tpac.edu.au
+#
+#Say whether this is a full NDG data Provider who uses NDG identifiers etc.

TI01-discovery/trunk/ingestAutomation/OAIBatch/wdcc_config.properties

@@ -1 +1 @@
 #Note - don't have any blank lines in this config file
-# Define host_OAI as the string that OAI adds to the filenames after harvesting
-# BODC = oai%3Agrid.bodc.nerc.ac.uk%3A
-# SOC = oai%3Aoai.noc.soton.ac.uk%3A
-# NCAR = oai%3Aucar.ncar.scd.cdp%3A
-# PML = oai%3Apml.nerc.ac.uk%3A
-# TPAC = oai%3Atpac.org.au%3A
-# WDCC = ?
-host_OAI = oai%3Agrid.wdcc.nerc.ac.uk%3A
+#host_OAI =
 #
 #Define host_path as the full directory name where this data centre's records will be harvested into.
-host_path /home/tomcat/oai/WEB-INF/harvested_records/uranus.dkrz.de/8080/oai/provider/dif
+host_path /usr/local/jakarta-tomcat/webapps/oai/WEB-INF/harvested_records/uranus-dkrz-de-8080-dif
+#
+#Define groups - portal groups for limiting searches by 'group of datacentres'.
+#groups
+#
+#Define which format is harvested from the data centre (one only)
+format DIF
+#
+#Define the data providers namespace
+namespace wdcc.dkrz.de
+#
+#Say whether this is a full NDG data Provider who uses NDG identifiers etc.