source: TI01-discovery/branches/ingestAutomation-upgrade/OAIBatch @ 4257

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @4257   11 years cbyrom Fix handling of ndg hosted data - properly reading config settings from …
(edit) @4224   11 years sdonegan Added extra try-catch to deal with missing temporal coverage in original …
(edit) @4223   11 years sdonegan Updated comments regarding fix for missing temporalCoverage in DIF
(edit) @4222   11 years sdonegan Removed file/folder
(edit) @4221   11 years sdonegan Updated bgs DPPP record
(edit) @4208   11 years sdonegan updated configuration files for new tomcat
(edit) @4207   11 years sdonegan test update
(edit) @4206   11 years sdonegan test update on branch
(edit) @4027   11 years cbyrom Add setup script for oai ingester + remove dependencies in this project on …
(edit) @3998   11 years cbyrom Additional 'todo' improvements.
(edit) @3972   11 years cbyrom Use the short filename in the postgres DB for storing the original …
(edit) @3967   11 years cbyrom Update code to allow the extraction of authors, parameters and scope from …
(edit) @3913   12 years cbyrom Updates to readme doc.
(edit) @3912   12 years cbyrom Adjust logging in oai_document_ingester + adjust exception handling, …
(edit) @3910   12 years cbyrom Fix problems with molesReadWrite - including escaping of special …
(edit) @3909   12 years cbyrom Add default logger use to keywordAdder + tidy up code.
(edit) @3908   12 years cbyrom Add local moles schema - for use with molesReadWrite.
(edit) @3907   12 years cbyrom Add rollback to db funcs - to allow processing to continue correctly if an …
(edit) @3906   12 years cbyrom Add further documentation - including filling out 'to do' section with …
(edit) @3869   12 years cbyrom Add code to delete records if they weren't properly ingested - to allow …
(edit) @3868   12 years cbyrom Add documentation file with description of new ingestion process.
(edit) @3865   12 years cbyrom Fix keywordAdder so that processing is properly escaped if errors occur …
(edit) @3862   12 years cbyrom Fix workflow so that transforms are only ran when actually required - i.e. …
(edit) @3861   12 years cbyrom Clear out redundant code and rename main script, oai_document_ingester - …
(edit) @3860   12 years cbyrom Add new function, escapeSpecialCharacters() to PostgresRecord? to correct …
(edit) @3858   12 years cbyrom Properly update doc when modifications made + remove redundant code + …
(edit) @3857   12 years cbyrom Add 'debug' mode to ingest script - to allow more verbose logging.
(edit) @3855   12 years cbyrom Add new config file, ingest.config - to store details of the postgres DB …
(edit) @3854   12 years cbyrom Add helper methods to SpatioTemporalData? for easy retrieval of …
(edit) @3853   12 years cbyrom Update python scripts to call the new stored procedures, to properly …
(edit) @3852   12 years cbyrom Add port to db_connect and adjust set up of connection to allow this to be …
(edit) @3847   12 years cbyrom Create new object, SpatioTemporalData?, to wrapper temporal and spatial …
(edit) @3846   12 years cbyrom Ajudst DAO and Record classes to throw errors rather than catching them - …
(edit) @3845   12 years cbyrom Fix creation of backup dirs.
(edit) @3844   12 years cbyrom Improve checking of return vals from db calls - to nullify these if …
(edit) @3839   12 years cbyrom Add script to run ingest for all avaiable config files. Make …
(edit) @3838   12 years cbyrom Remove unecessary files and dirs - including all xquery stuff since this …
(edit) @3837   12 years cbyrom Add saxon library - required for the xquery transforms.
(edit) @3836   12 years cbyrom Re-organise location of config.properties files to make codebase neater.
(edit) @3835   12 years cbyrom Remove logger - replaced by inbuilt python logger.
(edit) @3821   12 years cbyrom Fix a few problems - including referencing the xquery libraries; these …
(edit) @3820   12 years cbyrom Remove unecessary file.
(edit) @3817   12 years cbyrom Add default logging support + create new version of ingest script, …
(edit) @3816   12 years cbyrom Add workflow to allow creation of moles doc from whatever starting point …
(edit) @3815   12 years cbyrom Improve the FileUtilities? - using 'find' to get the various required …
(edit) @3814   12 years cbyrom Fill out functionality for DAO - including check for uniqueness of …
(edit) @3810   12 years cbyrom Create Logger class to standardise output - and allow this to be switched …
(edit) @3809   12 years cbyrom Improve PostgresRecord? class to generalise the transforming of docs into …
(edit) @3808   12 years cbyrom Rename class - DAL -> DAO (data access object) - to make more correctly …
(edit) @3800   12 years cbyrom Upgraded version of ingest codebranch - including major refactoring of the …
(copy) @3797   12 years cbyrom Upgraded version of ingest codebranch - including major refactoring of the …
copied from TI01-discovery/trunk/ingestAutomation/OAIBatch:
(edit) @3779   12 years selatham retiring some old code
Note: See TracRevisionLog for help on using the revision log.