Changeset 5255 for TI01-discovery


Ignore:
Timestamp:
06/05/09 12:13:12 (10 years ago)
Author:
cbyrom
Message:

Extend OAI Info editor documentation and help pages + improve the
setup config + extend test suite and fix problem with setting
email addresses in providerinfo.

Location:
TI01-discovery/trunk/OAIInfoEditor
Files:
8 edited

Legend:

Unmodified
Added
Removed
  • TI01-discovery/trunk/OAIInfoEditor/README.txt

    r5226 r5255  
    1 This file is for you to describe the oai_info_editor application. Typically 
    2 you would include information such as the information below: 
     1Prerequisites 
     2================== 
     3The following should be installed to run the OAI Info editor: 
     41. Python2.5 - see http://www.python.org/download/ 
     52. Pylons - see http://docs.pythonweb.org/display/pylonsdocs/Installing+Pylons 
     63. Genshi - see http://genshi.edgewall.org/wiki/Download 
     74. OAIBatch - see http://proj.badc.rl.ac.uk/ndg/browser/TI01-discovery/branches/ingestAutomation-upgrade [or main trunk when things have been properly merged] 
     85. ndgCommon - see http://proj.badc.rl.ac.uk/ndg/browser/ndgCommon 
     9 
     10NB, it is easier to setup the prereqs by using easy_install - http://peak.telecommunity.com/DevCenter/EasyInstall 
     11- then you can just run, easy_install pylons, easy_install genshi, etc. 
    312 
    413Installation and Setup 
    514====================== 
     15Once the prereqs are installed and the codebase is extracted to a local: 
    616 
    7 Install ``oai_info_editor`` using easy_install:: 
     17i) Adjust the content of the config files in oai_info_editor directory as follows: 
    818 
    9     easy_install oai_info_editor 
     19development.ini 
     20- adjust the host and port settings appropriately 
     21- turn debug on/off 
     22- adjust logging level of server output (adjust the logger_root settings) 
    1023 
    11 Make a config file as follows:: 
     24editor.config 
     25adjust: 
    1226 
    13     paster make-config oai_info_editor config.ini 
     27[SERVERS] 
     28- server - NB, this should match the host and port settings in the development.ini file 
     29- mailserver - SMTP server for sending mails 
     30- proxyServer - proxy server to use, if required 
    1431 
    15 Tweak the config file as appropriate and then setup the application:: 
     32[DATA_STORE] 
     33- appDataFileName - name of the data file used to store the provider info in - NB, this 
     34is suffixed with '_provider_info_data.xml' 
     35- appDataFileDir - the directory, relative to the run directory, to store the 
     36change history data and any parser data files 
     37- backupFileDir - the directory, relative to the run directory, to store backups of 
     38the data files in the appDataFileDir 
     39- harvestDir - the directory to harvest OAI documents to 
    1640 
    17     paster setup-app config.ini 
     41[USER_LIST] 
     42- userIDs - include a list of user IDs which have access to the OAI Info Editor 
    1843 
    19 Then you are ready to go. 
     44NB, each user ID should then have its own section - e.g. if userIDs is as follows: 
     45 
     46userIDs: calum bob fred 
     47 
     48there needs to be three futher sections, [calum], [bob] and [fred]  
     49 
     50- the content of these additional sections should include the following: 
     51- pw - the password matching the user ID 
     52- providerNames - a semicolon delimeted list of provider names that the user ID has 
     53view/edit access rights to.  NB, if the user can view/edit everything, providerNames 
     54should be set to the value, admin 
     55 
     56- e.g. for the above example: 
     57 
     58[calum] 
     59pw: gloopy 
     60providerNames: admin 
     61 
     62[bob] 
     63pw: bob 
     64providerNames: provider1 
     65 
     66[fred] 
     67pw: fred 
     68providerNames: provider2;provider3 
     69 
     70- calum is then an admin user and bob can view/edit provider1 info, whereas fred 
     71can view/edit provider2 AND provider3 info. 
     72  
     73ii) Add a config file for the postgres DB used by the discovery service; this is required 
     74when documents are harvested via the editor - since once the harvest is complete, the 
     75data is automatically ingested into the discovery service.  The file should be in 
     76the oai_info_editor directory and be called, 'ingest.config'.  The contents should 
     77be of the format: 
     78 
     79databaseName hostName userName password portNumber 
     80 
     81(NB, portNumber is optional) 
     82 
     83iii) From the oai_info_editor directory, run: 
     84 
     85paster serve --reload development.ini 
     86 
     87 
     88Using the Editor 
     89------------------ 
     90A help guide is available from the web app - click on the link, 'help' for detailed 
     91instructions on how to use. 
     92 
     93 
     94Tests 
     95------------ 
     96A suite of unit tests is available under oai_info_editor/tests - the file, nostTests.py,  
     97in this directory, can be used to discover and run all the tests via the python nose module -  
     98just run it as a python app. 
     99 
     100 
     101 Further Notes 
     102----------------- 
     103i) The editor uses the OAIBatch codebase to do the ingest - using the same 
     104script, oai_document_ingester, that is used by the discovery service 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/editor.config

    r5242 r5255  
    1515 
    1616[DATA_STORE] 
    17 # specify where the persisted data, and backups of this, should be stored 
     17# specify where the persisted data and backups of this, should be stored.  also specify 
     18# local dir to store harvested data 
    1819appDataFileName: oai_info_data 
    1920appDataFileDir: oaiInfoEditorData 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/model/providerinfo.py

    r5251 r5255  
    1818 
    1919 
    20 def createProviderInfoWithData(providerName, repositoryInfos, providerEmail = None): 
     20def createProviderInfoWithData(providerName, repositoryInfos, providerEmail = ''): 
    2121    ''' 
    2222    Factory method to create ProviderInfo object with set values 
     
    5353 
    5454 
    55     def setData(self, providerName, repositoryInfos, providerEmail = None): 
     55    def setData(self, providerName, repositoryInfos, providerEmail = ''): 
    5656        ''' 
    5757        Simple method to set data on object 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/templates/oai_info_editor_help.html

    r5242 r5255  
    8989                            the name of the provider [mandatory] - NB editable only by admin users 
    9090                            </dd> 
     91                            <dt><b>Provider Email</b></dt><dd> 
     92                            the email address of the provider - NB, this is primarily used for communicating 
     93                            the results of the ingest process (section <a href="${h.url_for('harvest')}"> 
     94                        Harvesting Data</a>). 
     95                            </dd> 
    9196                            <dt><b>Repository Name</b></dt> 
    9297                            <dd> 
     
    194199                    </p> 
    195200                    <p> 
    196                     The outcome of both of these operations, together with basic details (e.g. 
    197                     number of records harvested) is then displayed to the user. 
     201                    The outcome of both of the harvest operation, together with basic details (e.g. 
     202                    number of records harvested) is then displayed to the user.  If there is 
     203                    a provider email address available, the results of the ingest will be mailed 
     204                    to that; NB, the ingest is rather a slow process so it is instigated asynchronously. 
    198205                    </p> 
    199206                    <p> 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/lib/testharvester.py

    r5253 r5255  
    1919 
    2020        setUpDir(VALID_HARVEST_DIR) 
    21         self.h = Harvester() 
     21        self.h = Harvester(VALID_MAIL_SERVER) 
    2222         
    2323    def testBasicConstructorWithKeyword(self): 
    24         h = Harvester(outDir = VALID_HARVEST_DIR) 
     24        h = Harvester(VALID_MAIL_SERVER, outDir = VALID_HARVEST_DIR) 
    2525        self.assertEquals(VALID_HARVEST_DIR, h.outDir) 
     26        self.assertEquals(VALID_MAIL_SERVER, h.mailServer) 
    2627         
    2728    def testBasicConstructor(self): 
    2829        self.assertEquals(None, self.h.outDir) 
     30        self.assertEquals(VALID_MAIL_SERVER, self.h.mailServer) 
    2931 
    3032    def testInvalidInputHarvestRepository(self): 
     
    3638        self.assertFalse(status) 
    3739        self.assertEquals('Error: The request for data resulted in an invalid response from the provider. Error: Server returned HTTP response code: 400 for URL: http://rep2.ac.uk?verb=Identify', out) 
    38         self.assertEquals('tmp/rep2-ac-uk-DIF', dir) 
     40        self.assertEquals('%s/rep2-ac-uk-DIF' %VALID_HARVEST_DIR, dir) 
    3941 
    4042    def testHarvestRepository(self): 
     
    4345        self.assertTrue(status) 
    4446        self.assertTrue(out.find('Harvest of http://badc.nerc.ac.uk/badc_oai/provider is complete') > -1) 
    45         self.assertEquals('tmp/badc-nerc-ac-uk-dif', dir) 
     47        self.assertEquals('%s/badc-nerc-ac-uk-dif' %VALID_HARVEST_DIR, dir) 
    4648     
    4749    def tearDown(self): 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/testdata/test_provider_info_collection.xml

    r5251 r5255  
    11<providerInfoCollection> 
    22   <version>1</version> 
    3    <harvestDir>/tmp/harvest_me_now/</harvestDir> 
     3   <harvestDir>tmp_harvest_dir</harvestDir> 
    44   <providerInfo name="provider1"> 
    55          <email>BOB@Bob.com</email> 
  • TI01-discovery/trunk/OAIInfoEditor/oai_info_editor/tests/testdata/testconstants.py

    r5251 r5255  
    4040HARVEST_COMMAND = 'java %s -cp %sDLESETools.jar:%sjdom-b7.jar:%sxercesImpl.jar:%sxml-apis.jar org.dlese.dpc.oai.harvester.Harvester %s %s %s'\ 
    4141    %('%s', LIB_DATA_DIR, LIB_DATA_DIR, LIB_DATA_DIR, LIB_DATA_DIR, '%s', '%s', '%s') 
     42 
     43VALID_MAIL_SERVER = 'outbox.rl.ac.uk' 
    4244 
    4345VALID_REP_INFO_FILE = TEST_DATA_DIR + 'test_repository_info.xml' 
  • TI01-discovery/trunk/OAIInfoEditor/setup.py

    r5226 r5255  
    1313    #author_email='', 
    1414    #url='', 
    15     install_requires=["Pylons>=0.9.6.2"], 
     15    install_requires=["Pylons>=0.9.6.2", "ndgCommon", "genshi"], 
    1616    packages=find_packages(exclude=['ez_setup']), 
    1717    include_package_data=True, 
Note: See TracChangeset for help on using the changeset viewer.