source: TI01-discovery/branches/ingestAutomation-upgrade/OAIBatch @ 3869

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @3869   12 years cbyrom Add code to delete records if they weren't properly ingested - to allow …
(edit) @3868   12 years cbyrom Add documentation file with description of new ingestion process.
(edit) @3865   12 years cbyrom Fix keywordAdder so that processing is properly escaped if errors occur …
(edit) @3862   12 years cbyrom Fix workflow so that transforms are only ran when actually required - i.e. …
(edit) @3861   12 years cbyrom Clear out redundant code and rename main script, oai_document_ingester - …
(edit) @3860   12 years cbyrom Add new function, escapeSpecialCharacters() to PostgresRecord? to correct …
(edit) @3858   12 years cbyrom Properly update doc when modifications made + remove redundant code + …
(edit) @3857   12 years cbyrom Add 'debug' mode to ingest script - to allow more verbose logging.
(edit) @3855   12 years cbyrom Add new config file, ingest.config - to store details of the postgres DB …
(edit) @3854   12 years cbyrom Add helper methods to SpatioTemporalData? for easy retrieval of …
(edit) @3853   12 years cbyrom Update python scripts to call the new stored procedures, to properly …
(edit) @3852   12 years cbyrom Add port to db_connect and adjust set up of connection to allow this to be …
(edit) @3847   12 years cbyrom Create new object, SpatioTemporalData?, to wrapper temporal and spatial …
(edit) @3846   12 years cbyrom Ajudst DAO and Record classes to throw errors rather than catching them - …
(edit) @3845   12 years cbyrom Fix creation of backup dirs.
(edit) @3844   12 years cbyrom Improve checking of return vals from db calls - to nullify these if …
(edit) @3839   12 years cbyrom Add script to run ingest for all avaiable config files. Make …
(edit) @3838   12 years cbyrom Remove unecessary files and dirs - including all xquery stuff since this …
(edit) @3837   12 years cbyrom Add saxon library - required for the xquery transforms.
(edit) @3836   12 years cbyrom Re-organise location of config.properties files to make codebase neater.
(edit) @3835   12 years cbyrom Remove logger - replaced by inbuilt python logger.
(edit) @3821   12 years cbyrom Fix a few problems - including referencing the xquery libraries; these …
(edit) @3820   12 years cbyrom Remove unecessary file.
(edit) @3817   12 years cbyrom Add default logging support + create new version of ingest script, …
(edit) @3816   12 years cbyrom Add workflow to allow creation of moles doc from whatever starting point …
(edit) @3815   12 years cbyrom Improve the FileUtilities? - using 'find' to get the various required …
(edit) @3814   12 years cbyrom Fill out functionality for DAO - including check for uniqueness of …
(edit) @3810   12 years cbyrom Create Logger class to standardise output - and allow this to be switched …
(edit) @3809   12 years cbyrom Improve PostgresRecord? class to generalise the transforming of docs into …
(edit) @3808   12 years cbyrom Rename class - DAL -> DAO (data access object) - to make more correctly …
(edit) @3800   12 years cbyrom Upgraded version of ingest codebranch - including major refactoring of the …
(copy) @3797   12 years cbyrom Upgraded version of ingest codebranch - including major refactoring of the …
copied from TI01-discovery/trunk/ingestAutomation/OAIBatch:
(edit) @3779   12 years selatham retiring some old code
Note: See TracRevisionLog for help on using the revision log.