source: TI01-discovery/trunk/ingestAutomation @ 3785

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @3785   13 years selatham gather required utilities. improve elementree imports in keywordAdder. put …
(edit) @3779   13 years selatham retiring some old code
(edit) @3777   13 years selatham removed development SpaceTimeIngestPostgisFromMOLES
(edit) @3205   13 years selatham make celementree imports more robust
(edit) @3194   13 years selatham bug fix - non-existent reference kicking about.
(edit) @3179   13 years selatham tidying up a bit
(edit) @3177   13 years selatham adding new coverages extraction to postgis tables.
(edit) @3172   13 years selatham change working directory early, so that finds pwds file.
(edit) @3168   13 years selatham Re-write to use the pythonic dif2moles xquery rather than Java. extract …
(edit) @3101   13 years selatham Cope with slashes in id. More DP config files
(edit) @3021   13 years selatham Another new DP. Tidied ingest.
(edit) @3020   13 years selatham New DP configs. Deal with colons in IDs
(edit) @2972   13 years selatham bgs config
(edit) @2856   13 years selatham corrected pmls config file
(edit) @2508   14 years selatham various improvements
(edit) @2424   14 years selatham fixed bug in spatiotemp ingest where no dates
(edit) @2423   14 years selatham fixed bug in ingest - not clearing out intermediate directory.
(edit) @2421   14 years selatham Bug in inserting with non-existant dates
(edit) @2378   14 years selatham more config files
(edit) @2338   14 years selatham Corrected keyword to be added.
(edit) @2331   14 years selatham updated bgs config file.
(edit) @2329   14 years selatham Using keywordAdder correctly. sorted spatiotemporal null problem.
(edit) @2324   14 years selatham new SchemaNameSpace? corrector
(edit) @2314   14 years selatham correcting nocs config file.
(edit) @2306   14 years selatham colon to underscores translation.
(edit) @2300   14 years selatham new config files
(edit) @2273   14 years selatham new datacentre config files
(edit) @2263   14 years selatham nocs config etc.
(edit) @2262   14 years selatham cope with empty spatiotemporal tags
(edit) @2252   14 years selatham Copes with MDIP format. Some new config file
(edit) @2088   14 years selatham implementing keywordAdder.
(edit) @2067   14 years selatham Plugging in new d2boneoff.
(edit) @2066   14 years selatham continued development.
(edit) @2002   14 years selatham Cope with no dates OR no coords.
(edit) @1988   14 years selatham Changing config files
(edit) @1987   14 years selatham changed namespace. Added ndgDataProvider.
(edit) @1971   14 years selatham Changes for getting id for filename from inside Discovery record.
(edit) @1954   14 years selatham cope with multiple bounding boxes.
(edit) @1949   14 years selatham dealing with missing content
(edit) @1948   14 years selatham added date stamp
(edit) @1916   14 years selatham now works
(edit) @1898   14 years selatham further ingest work
(edit) @1889   14 years selatham getting auto ingest working
(edit) @1880   14 years selatham putting all new parts in
(edit) @1869   14 years selatham further work on ingest
(edit) @1837   14 years selatham half-done keywordAdder.
(edit) @1821   14 years selatham error trapping
(edit) @1809   14 years selatham namespace declaration problem.
(edit) @1797   14 years selatham Do a bit of lat/lon parsing. Actually ingests now
(edit) @1794   14 years selatham doing ingest of spatio temporal information into postgres
(edit) @1788   14 years selatham remove some debug prints
(edit) @1769   14 years selatham Added a config file for neodc
(edit) @1768   14 years selatham Removed use of oaiClean. Just do renaming. Use data centres namespace …
(edit) @1755   14 years selatham corrected boolean.
(edit) @1754   14 years selatham made ndgdoc style wrapping optional. Turned off in script
(edit) @1620   14 years selatham fix bug where 'in' directory not being cleared initially.
(edit) @1619   14 years selatham Allow no specification of groups/keywords in config file
(edit) @1599   14 years selatham oaiClean.py included <doc> element around ingested discovery document.
(edit) @1593   14 years selatham Cope with different incoming formats. Also add keywords from config file.
(edit) @1591   14 years selatham Uses oaiClean.py instead of java clean-up code. Also deals with upgrade …
(edit) @1590   14 years selatham created a python version of the discovery record cleaning code. Also wraps …
(edit) @1579   14 years selatham Changes for upgrade and re-deployment of eXist db in slightly different …
(edit) @1481   14 years selatham DLESE upgrade meant harvest location changed
(edit) @1478   14 years selatham new version of DLESE jOAI changed harvesting default location
(edit) @1260   15 years selatham tweaked nocs config file
(edit) @1255   15 years selatham setting up nocs config file
(edit) @1172   15 years selatham fixing bug in directory tidying
(edit) @1126   15 years selatham more stuff for mass file handling
(edit) @1121   15 years selatham bug fix in copying to backup directories
(edit) @1076   15 years selatham putting in pwd protection
(edit) @1066   15 years selatham needs to cope with large numbers of files
(edit) @1046   15 years selatham config file for wdcc data centre
(edit) @1045   15 years selatham setting up wdcc data centre stuff
(edit) @962   15 years selatham typing error in options
(edit) @956   15 years selatham securing the exist backup script
(edit) @916   15 years root correct spelling
(edit) @909   15 years root Had to put the -p option in the exist backup now
(edit) @863   15 years selatham bug fix to oai_ingest and updated pml and bodc config files
(edit) @823   15 years selatham Updateing config files for bodc, badc
(edit) @741   15 years selatham clear out old harvests and exit when nothing to harvest
(edit) @729   15 years selatham delete old Automation directories not required
(edit) @728   15 years selatham archive Marta's old OAO automation stuff
(edit) @727   15 years selatham archive Marta's old OAO automation stuff
(edit) @726   15 years selatham archive Marta's old OAO automation stuff
(edit) @724   15 years selatham move source code back to OAIBatch
(edit) @723   15 years selatham move source code back to OAIBatch
(edit) @722   15 years selatham move source code back to OAIBatch
(edit) @721   15 years selatham adding ingest scripts, config files etc
(edit) @720   15 years selatham adding database backup scripts
(add) @719   15 years selatham re-arranging ingest automation stuff
Note: See TracRevisionLog for help on using the revision log.