source: TI01-discovery/branches/ingestAutomation-upgrade/OAIBatch @ 8912

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @6200   10 years sdonegan updated config for non oai feed ingest
(edit) @6195   10 years sdonegan debig
(edit) @6194   10 years sdonegan Added thread id so abstractdocumentingester.py can distinguish between …
(edit) @6193   10 years sdonegan debug
(edit) @6192   10 years sdonegan debug
(edit) @6190   10 years sdonegan debug
(edit) @6189   10 years sdonegan Added ability to take processing info from a config file (not in svn for …
(edit) @6188   10 years sdonegan debug
(edit) @6187   10 years sdonegan Added ability to take processing info from a config file (not in svn for …
(edit) @6186   10 years sdonegan Added ability to take processing info from a config file (not in svn for …
(edit) @6162   10 years sdonegan Update feed ingester for consistency with production oai ingest (non local …
(edit) @6154   10 years sdonegan all bugs in controller script seem to be fixed…
(edit) @6137   10 years sdonegan Further debugs w.r.t non local file handling
(edit) @6136   10 years sdonegan Further debugs w.r.t non local file handling
(edit) @6135   10 years sdonegan Further debugs w.r.t non local file handling
(edit) @6133   10 years sdonegan Further debugs w.r.t non local file handling
(edit) @6132   10 years sdonegan fixing bugs in config file handling
(edit) @6131   10 years sdonegan took out useless pkg_resources handling of config file. use normal …
(edit) @6130   10 years sdonegan Updated handling of directories - put in absolute production paths in badc …
(edit) @6128   10 years sdonegan Took out local directory references and put absolute paths where necessary
(edit) @6100   10 years sdonegan updated reporting
(edit) @6099   10 years sdonegan Updated reporting
(edit) @6093   10 years sdonegan added back in the individual dc reporting - not sure why it was dropped!!
(edit) @6092   10 years sdonegan fixed bug with lineseparator
(edit) @6091   10 years sdonegan Updated error log handling - configured for use in production environment …
(edit) @6031   10 years sdonegan Updated NDG redirection service to production on …
(edit) @6017   10 years sdonegan Updated redirectUrlchanger class so deals with different input formats …
(edit) @5976   10 years sdonegan updated keyword provision when calling method
(edit) @5847   10 years sdonegan fixed bug in getting latest revision date & distinguishing from creation …
(edit) @5846   10 years sdonegan Fixed bug so handles special chars in dataset title and handling of …
(edit) @5737   10 years sdonegan Adjusted filename handling and id's to fix problem with presenting …
(edit) @5736   10 years sdonegan Sorted bug in handling errors where no records available to delete
(edit) @5716   10 years sdonegan New method to allow deletion of all records in Discovery database …
(edit) @5714   10 years sdonegan Fixing big with mdip handling via oai_info_editor (lower case mdip format)
(edit) @5713   10 years sdonegan Udating location
(edit) @5687   10 years sdonegan Updated to handle lower case dif format spec from info editor..
(edit) @5686   10 years sdonegan withdrawing explict DeleteRecord? activation - handled by postgresql …
(edit) @5603   10 years sdonegan small update to reporting
(edit) @5602   10 years sdonegan Debugged ATOM feed ingestion to deal with revised eXist atom architecture …
(edit) @5600   10 years sdonegan Debugged ATOM feed ingestion to deal with revised eXist atom architecture
(edit) @5587   10 years sdonegan Debugging integreattion
(edit) @5586   10 years sdonegan Debugging integreattion
(edit) @5585   10 years sdonegan Updated feed handling to use new reporting methods as developed for …
(edit) @5584   10 years sdonegan Updated feed handling to use new reporting methods as developed for …
(edit) @5583   10 years sdonegan Updated reporting to cmd line on oai_document_ingester.py
(edit) @5582   10 years sdonegan Updated reporting to give correct numbers on ingests
(edit) @5580   10 years sdonegan
(edit) @5553   10 years sdonegan Removed file/folder
(edit) @5548   10 years sdonegan Renamed and sorted config files to match names on oai provider info page …
(edit) @5547   10 years sdonegan replace these original config files with normalised ones to oai_provider …
(edit) @5542   10 years sdonegan Adjust messaging so FULL report is made available to info editor for email
(edit) @5537   10 years sdonegan Updated to allow inclusion of end date ordering
(edit) @5524   10 years sdonegan Updated to allow extraction of end date into special column to allow …
(edit) @5523   10 years sdonegan synchronising dev with buildouts for info editor operations
(edit) @5509   10 years sdonegan dodgy update wiped out recent amendments... restored this version
(edit) @5508   10 years sdonegan Removed file/folder
(edit) @5506   10 years sdonegan corrected bug in indentation…
(edit) @5498   10 years sdonegan debugged problem with previous version - good ol indentation error missed
(edit) @5496   10 years sdonegan update to latest working merged ingest
(edit) @5495   10 years sdonegan Removed file/folder
(edit) @5464   10 years sdonegan Allow start date to be inserted into new column in original doc to allow …
(edit) @5418   10 years sdonegan Class needed for individual deletion of records from postgres database via …
(edit) @5417   10 years sdonegan needed for deletion of recs - individually - check
(edit) @5416   10 years sdonegan Support extra abstractdocumentingester.py functionality - deletions and …
(edit) @5415   10 years sdonegan Support extra abstractdocumentingester.py functionality - deletions and …
(edit) @5414   10 years sdonegan Contains merged code to allow reporting of ingest, use of ndg redirect, …
(edit) @5413   10 years sdonegan Now includes classes to handle ingest identityTransforms by xquery and …
(edit) @5352   11 years sdonegan
(edit) @5351   11 years sdonegan testing config
(edit) @5326   11 years sdonegan Class to aid in creation of object to aid in extraction of ID plus other …
(edit) @5297   11 years cbyrom Add additional instructions for setting up ingester + improve the setup …
(edit) @5252   11 years cbyrom Simplify error handling, improve output logging + standardise use of upper …
(edit) @5248   11 years cbyrom Update ingest scripts for use with the OAIInfoEditor harvest functionality …
(edit) @5243   11 years cbyrom Adjust logging and output of error in ingest scripts.
(edit) @5218   11 years cbyrom Tidy up docs + fix bug with wrongly scoped variable + remove unecessary …
(edit) @5167   11 years sdonegan updates to allow oai_document_ingester to work
(edit) @5153   11 years sdonegan debugged numfilesproc error
(edit) @4980   11 years cbyrom Fix ingester to use new object structure correctly and more efficiently. …
(edit) @4962   11 years cbyrom Remove breakpoint.
(edit) @4948   11 years cbyrom Adjust ingest scripts to use the new ndg common client suite + add new …
(edit) @4854   11 years cbyrom Add new ingest script - to allow ingest of DIF docs from eXist hosted atom …
(edit) @4714   11 years sdonegan SJD cock up (4712)- put Calums latest (4664) back as head revision!
(edit) @4712   11 years sdonegan Updated to compile a report on number of files successfully (or not) …
(edit) @4711   11 years sdonegan Updated to collate information from each data centre ingest report and …
(edit) @4664   11 years cbyrom Update code to reflect refactoring of ndgUtils.
(edit) @4423   11 years cbyrom Move FileUtilities? module to ndgUtils to allow re-use elsewhere + un-oo …
(edit) @4258   11 years cbyrom Extend idget function to cope with MDIP records and implement usage of …
(edit) @4257   11 years cbyrom Fix handling of ndg hosted data - properly reading config settings from …
(edit) @4224   11 years sdonegan Added extra try-catch to deal with missing temporal coverage in original …
(edit) @4223   11 years sdonegan Updated comments regarding fix for missing temporalCoverage in DIF
(edit) @4222   11 years sdonegan Removed file/folder
(edit) @4221   11 years sdonegan Updated bgs DPPP record
(edit) @4208   11 years sdonegan updated configuration files for new tomcat
(edit) @4207   11 years sdonegan test update
(edit) @4206   11 years sdonegan test update on branch
(edit) @4027   11 years cbyrom Add setup script for oai ingester + remove dependencies in this project on …
(edit) @3998   12 years cbyrom Additional 'todo' improvements.
(edit) @3972   12 years cbyrom Use the short filename in the postgres DB for storing the original …
(edit) @3967   12 years cbyrom Update code to allow the extraction of authors, parameters and scope from …
(edit) @3913   12 years cbyrom Updates to readme doc.
Note: See TracRevisionLog for help on using the revision log.