Changeset 5255

06/05/09 12:13:12 (10 years ago)

Extend OAI Info editor documentation and help pages + improve the
setup config + extend test suite and fix problem with setting
email addresses in providerinfo.

8 edited


  • TI01-discovery/trunk/OAIInfoEditor/README.txt

    r5226 r5255  
    1 This file is for you to describe the oai_info_editor application. Typically 
    2 you would include information such as the information below: 
     3The following should be installed to run the OAI Info editor: 
     41. Python2.5 - see 
     52. Pylons - see 
     63. Genshi - see 
     74. OAIBatch - see [or main trunk when things have been properly merged] 
     85. ndgCommon - see 
     10NB, it is easier to setup the prereqs by using easy_install - 
     11- then you can just run, easy_install pylons, easy_install genshi, etc. 
    413Installation and Setup 
     15Once the prereqs are installed and the codebase is extracted to a local: 
    7 Install ``oai_info_editor`` using easy_install:: 
     17i) Adjust the content of the config files in oai_info_editor directory as follows: 
    9     easy_install oai_info_editor 
     20- adjust the host and port settings appropriately 
     21- turn debug on/off 
     22- adjust logging level of server output (adjust the logger_root settings) 
    11 Make a config file as follows:: 
    13     paster make-config oai_info_editor config.ini 
     28- server - NB, this should match the host and port settings in the development.ini file 
     29- mailserver - SMTP server for sending mails 
     30- proxyServer - proxy server to use, if required 
    15 Tweak the config file as appropriate and then setup the application:: 
     33- appDataFileName - name of the data file used to store the provider info in - NB, this 
     34is suffixed with '_provider_info_data.xml' 
     35- appDataFileDir - the directory, relative to the run directory, to store the 
     36change history data and any parser data files 
     37- backupFileDir - the directory, relative to the run directory, to store backups of 
     38the data files in the appDataFileDir 
     39- harvestDir - the directory to harvest OAI documents to 
    17     paster setup-app config.ini 
     42- userIDs - include a list of user IDs which have access to the OAI Info Editor 
    19 Then you are ready to go. 
     44NB, each user ID should then have its own section - e.g. if userIDs is as follows: 
     46userIDs: calum bob fred 
     48there needs to be three futher sections, [calum], [bob] and [fred]  
     50- the content of these additional sections should include the following: 
     51- pw - the password matching the user ID 
     52- providerNames - a semicolon delimeted list of provider names that the user ID has 
     53view/edit access rights to.  NB, if the user can view/edit everything, providerNames 
     54should be set to the value, admin 
     56- e.g. for the above example: 
     59pw: gloopy 
     60providerNames: admin 
     63pw: bob 
     64providerNames: provider1 
     67pw: fred 
     68providerNames: provider2;provider3 
     70- calum is then an admin user and bob can view/edit provider1 info, whereas fred 
     71can view/edit provider2 AND provider3 info. 
     73ii) Add a config file for the postgres DB used by the discovery service; this is required 
     74when documents are harvested via the editor - since once the harvest is complete, the 
     75data is automatically ingested into the discovery service.  The file should be in 
     76the oai_info_editor directory and be called, 'ingest.config'.  The contents should 
     77be of the format: 
     79databaseName hostName userName password portNumber 
     81(NB, portNumber is optional) 
     83iii) From the oai_info_editor directory, run: 
     85paster serve --reload development.ini 
     88Using the Editor 
     90A help guide is available from the web app - click on the link, 'help' for detailed 
     91instructions on how to use. 
     96A suite of unit tests is available under oai_info_editor/tests - the file,,  
     97in this directory, can be used to discover and run all the tests via the python nose module -  
     98just run it as a python app. 
     101 Further Notes 
     103i) The editor uses the OAIBatch codebase to do the ingest - using the same 
     104script, oai_document_ingester, that is used by the discovery service 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/editor.config

    r5242 r5255  
    17 # specify where the persisted data, and backups of this, should be stored 
     17# specify where the persisted data and backups of this, should be stored.  also specify 
     18# local dir to store harvested data 
    1819appDataFileName: oai_info_data 
    1920appDataFileDir: oaiInfoEditorData 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/model/

    r5251 r5255  
    20 def createProviderInfoWithData(providerName, repositoryInfos, providerEmail = None): 
     20def createProviderInfoWithData(providerName, repositoryInfos, providerEmail = ''): 
    2121    ''' 
    2222    Factory method to create ProviderInfo object with set values 
    55     def setData(self, providerName, repositoryInfos, providerEmail = None): 
     55    def setData(self, providerName, repositoryInfos, providerEmail = ''): 
    5656        ''' 
    5757        Simple method to set data on object 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/templates/oai_info_editor_help.html

    r5242 r5255  
    8989                            the name of the provider [mandatory] - NB editable only by admin users 
    9090                            </dd> 
     91                            <dt><b>Provider Email</b></dt><dd> 
     92                            the email address of the provider - NB, this is primarily used for communicating 
     93                            the results of the ingest process (section <a href="${h.url_for('harvest')}"> 
     94                        Harvesting Data</a>). 
     95                            </dd> 
    9196                            <dt><b>Repository Name</b></dt> 
    9297                            <dd> 
    194199                    </p> 
    195200                    <p> 
    196                     The outcome of both of these operations, together with basic details (e.g. 
    197                     number of records harvested) is then displayed to the user. 
     201                    The outcome of both of the harvest operation, together with basic details (e.g. 
     202                    number of records harvested) is then displayed to the user.  If there is 
     203                    a provider email address available, the results of the ingest will be mailed 
     204                    to that; NB, the ingest is rather a slow process so it is instigated asynchronously. 
    198205                    </p> 
    199206                    <p> 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/lib/

    r5253 r5255  
    2020        setUpDir(VALID_HARVEST_DIR) 
    21         self.h = Harvester() 
     21        self.h = Harvester(VALID_MAIL_SERVER) 
    2323    def testBasicConstructorWithKeyword(self): 
    24         h = Harvester(outDir = VALID_HARVEST_DIR) 
     24        h = Harvester(VALID_MAIL_SERVER, outDir = VALID_HARVEST_DIR) 
    2525        self.assertEquals(VALID_HARVEST_DIR, h.outDir) 
     26        self.assertEquals(VALID_MAIL_SERVER, h.mailServer) 
    2728    def testBasicConstructor(self): 
    2829        self.assertEquals(None, self.h.outDir) 
     30        self.assertEquals(VALID_MAIL_SERVER, self.h.mailServer) 
    3032    def testInvalidInputHarvestRepository(self): 
    3638        self.assertFalse(status) 
    3739        self.assertEquals('Error: The request for data resulted in an invalid response from the provider. Error: Server returned HTTP response code: 400 for URL:', out) 
    38         self.assertEquals('tmp/rep2-ac-uk-DIF', dir) 
     40        self.assertEquals('%s/rep2-ac-uk-DIF' %VALID_HARVEST_DIR, dir) 
    4042    def testHarvestRepository(self): 
    4345        self.assertTrue(status) 
    4446        self.assertTrue(out.find('Harvest of is complete') > -1) 
    45         self.assertEquals('tmp/badc-nerc-ac-uk-dif', dir) 
     47        self.assertEquals('%s/badc-nerc-ac-uk-dif' %VALID_HARVEST_DIR, dir) 
    4749    def tearDown(self): 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/testdata/test_provider_info_collection.xml

    r5251 r5255  
    22   <version>1</version> 
    3    <harvestDir>/tmp/harvest_me_now/</harvestDir> 
     3   <harvestDir>tmp_harvest_dir</harvestDir> 
    44   <providerInfo name="provider1"> 
    55          <email></email> 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/testdata/

    r5251 r5255  
    4040HARVEST_COMMAND = 'java %s -cp %sDLESETools.jar:%sjdom-b7.jar:%sxercesImpl.jar:%sxml-apis.jar org.dlese.dpc.oai.harvester.Harvester %s %s %s'\ 
    4141    %('%s', LIB_DATA_DIR, LIB_DATA_DIR, LIB_DATA_DIR, LIB_DATA_DIR, '%s', '%s', '%s') 
     43VALID_MAIL_SERVER = '' 
    4345VALID_REP_INFO_FILE = TEST_DATA_DIR + 'test_repository_info.xml' 
  • TI01-discovery/trunk/OAIInfoEditor/

    r5226 r5255  
    1313    #author_email='', 
    1414    #url='', 
    15     install_requires=["Pylons>="], 
     15    install_requires=["Pylons>=", "ndgCommon", "genshi"], 
    1616    packages=find_packages(exclude=['ez_setup']), 
    1717    include_package_data=True, 
Note: See TracChangeset for help on using the changeset viewer.