source: TI01-discovery/trunk/ingestAutomation @ 3785

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @3785   12 years selatham gather required utilities. improve elementree imports in keywordAdder. put …
(edit) @3779   12 years selatham retiring some old code
(edit) @3777   12 years selatham removed development SpaceTimeIngestPostgisFromMOLES
(edit) @3205   12 years selatham make celementree imports more robust
(edit) @3194   12 years selatham bug fix - non-existent reference kicking about.
(edit) @3179   12 years selatham tidying up a bit
(edit) @3177   12 years selatham adding new coverages extraction to postgis tables.
(edit) @3172   12 years selatham change working directory early, so that finds pwds file.
(edit) @3168   12 years selatham Re-write to use the pythonic dif2moles xquery rather than Java. extract …
(edit) @3101   12 years selatham Cope with slashes in id. More DP config files
(edit) @3021   12 years selatham Another new DP. Tidied ingest.
(edit) @3020   12 years selatham New DP configs. Deal with colons in IDs
(edit) @2972   12 years selatham bgs config
(edit) @2856   12 years selatham corrected pmls config file
(edit) @2508   13 years selatham various improvements
(edit) @2424   13 years selatham fixed bug in spatiotemp ingest where no dates
(edit) @2423   13 years selatham fixed bug in ingest - not clearing out intermediate directory.
(edit) @2421   13 years selatham Bug in inserting with non-existant dates
(edit) @2378   13 years selatham more config files
(edit) @2338   13 years selatham Corrected keyword to be added.
(edit) @2331   13 years selatham updated bgs config file.
(edit) @2329   13 years selatham Using keywordAdder correctly. sorted spatiotemporal null problem.
(edit) @2324   13 years selatham new SchemaNameSpace? corrector
(edit) @2314   13 years selatham correcting nocs config file.
(edit) @2306   13 years selatham colon to underscores translation.
(edit) @2300   13 years selatham new config files
(edit) @2273   13 years selatham new datacentre config files
(edit) @2263   13 years selatham nocs config etc.
(edit) @2262   13 years selatham cope with empty spatiotemporal tags
(edit) @2252   13 years selatham Copes with MDIP format. Some new config file
(edit) @2088   13 years selatham implementing keywordAdder.
(edit) @2067   13 years selatham Plugging in new d2boneoff.
(edit) @2066   13 years selatham continued development.
(edit) @2002   13 years selatham Cope with no dates OR no coords.
(edit) @1988   13 years selatham Changing config files
(edit) @1987   13 years selatham changed namespace. Added ndgDataProvider.
(edit) @1971   13 years selatham Changes for getting id for filename from inside Discovery record.
(edit) @1954   13 years selatham cope with multiple bounding boxes.
(edit) @1949   13 years selatham dealing with missing content
(edit) @1948   13 years selatham added date stamp
(edit) @1916   13 years selatham now works
(edit) @1898   13 years selatham further ingest work
(edit) @1889   13 years selatham getting auto ingest working
(edit) @1880   13 years selatham putting all new parts in
(edit) @1869   13 years selatham further work on ingest
(edit) @1837   13 years selatham half-done keywordAdder.
(edit) @1821   13 years selatham error trapping
(edit) @1809   13 years selatham namespace declaration problem.
(edit) @1797   13 years selatham Do a bit of lat/lon parsing. Actually ingests now
(edit) @1794   13 years selatham doing ingest of spatio temporal information into postgres
(edit) @1788   13 years selatham remove some debug prints
(edit) @1769   13 years selatham Added a config file for neodc
(edit) @1768   13 years selatham Removed use of oaiClean. Just do renaming. Use data centres namespace …
(edit) @1755   13 years selatham corrected boolean.
(edit) @1754   13 years selatham made ndgdoc style wrapping optional. Turned off in script
(edit) @1620   13 years selatham fix bug where 'in' directory not being cleared initially.
(edit) @1619   13 years selatham Allow no specification of groups/keywords in config file
(edit) @1599   13 years selatham oaiClean.py included <doc> element around ingested discovery document.
(edit) @1593   13 years selatham Cope with different incoming formats. Also add keywords from config file.
(edit) @1591   13 years selatham Uses oaiClean.py instead of java clean-up code. Also deals with upgrade …
(edit) @1590   13 years selatham created a python version of the discovery record cleaning code. Also wraps …
(edit) @1579   13 years selatham Changes for upgrade and re-deployment of eXist db in slightly different …
(edit) @1481   13 years selatham DLESE upgrade meant harvest location changed
(edit) @1478   13 years selatham new version of DLESE jOAI changed harvesting default location
(edit) @1260   13 years selatham tweaked nocs config file
(edit) @1255   13 years selatham setting up nocs config file
(edit) @1172   14 years selatham fixing bug in directory tidying
(edit) @1126   14 years selatham more stuff for mass file handling
(edit) @1121   14 years selatham bug fix in copying to backup directories
(edit) @1076   14 years selatham putting in pwd protection
(edit) @1066   14 years selatham needs to cope with large numbers of files
(edit) @1046   14 years selatham config file for wdcc data centre
(edit) @1045   14 years selatham setting up wdcc data centre stuff
(edit) @962   14 years selatham typing error in options
(edit) @956   14 years selatham securing the exist backup script
(edit) @916   14 years root correct spelling
(edit) @909   14 years root Had to put the -p option in the exist backup now
(edit) @863   14 years selatham bug fix to oai_ingest and updated pml and bodc config files
(edit) @823   14 years selatham Updateing config files for bodc, badc
(edit) @741   14 years selatham clear out old harvests and exit when nothing to harvest
(edit) @729   14 years selatham delete old Automation directories not required
(edit) @728   14 years selatham archive Marta's old OAO automation stuff
(edit) @727   14 years selatham archive Marta's old OAO automation stuff
(edit) @726   14 years selatham archive Marta's old OAO automation stuff
(edit) @724   14 years selatham move source code back to OAIBatch
(edit) @723   14 years selatham move source code back to OAIBatch
(edit) @722   14 years selatham move source code back to OAIBatch
(edit) @721   14 years selatham adding ingest scripts, config files etc
(edit) @720   14 years selatham adding database backup scripts
(add) @719   14 years selatham re-arranging ingest automation stuff
Note: See TracRevisionLog for help on using the revision log.