source: TI01-discovery/trunk/OAIInfoEditor/README.txt @ 5473

Subversion URL: http://proj.badc.rl.ac.uk/svn/ndg/TI01-discovery/trunk/OAIInfoEditor/README.txt
Revision 5473, 8.2 KB checked in by cbyrom, 10 years ago (diff)

Add buildout configuration files + extend documentation + outline
in docs about the master doc test and real file.

Line 
1Prerequisites
2==================
3The following should be installed to run the OAI Info editor:
41. Python2.5 - see http://www.python.org/download/
52. Pylons - see http://docs.pythonweb.org/display/pylonsdocs/Installing+Pylons
63. Genshi - see http://genshi.edgewall.org/wiki/Download
74. OAIBatch - see http://proj.badc.rl.ac.uk/ndg/browser/TI01-discovery/branches/ingestAutomation-upgrade [or main trunk when things have been properly merged]
85. ndgCommon - see http://proj.badc.rl.ac.uk/ndg/browser/ndgCommon
9
10NB, it is easier to setup the prereqs by using easy_install - http://peak.telecommunity.com/DevCenter/EasyInstall
11- then you can just run, easy_install pylons, easy_install genshi, etc.
12
13[NB, THERE IS NOW A buildout.cfg FILE INCLUDED IN THE TOP LEVEL TO USE WITH zc.buildout
14- THIS CAN BE USED INSTEAD OF INSTALLING THE EGGS MANUALLY]
15
16
17Installation and Setup
18======================
19Once the prereqs are installed and the codebase is extracted to a local dir:
20
21i) Copy and adjust the content of the following config files, from the base directory to the
22directory to run the app from (taken to be 'oai_info_editor' here), as follows:
23
24development.ini (or secured.ini if using secured version - see security section, below)
25- adjust the host and port settings appropriately
26- turn debug on/off
27- adjust logging level of server output (adjust the logger_root settings)
28
29editor.config
30adjust:
31
32[SERVERS]
33- server - NB, this should match the host and port settings in the development.ini file
34- mailserver - SMTP server for sending mails
35- proxyServer - proxy server to use, if required
36
37[DATA_STORE]
38- appDataFileName - name of the data file used to store the provider info in - NB, this
39is suffixed with '_provider_info_data.xml'
40- appDataFileDir - the directory, relative to the run directory, to store the
41change history data and any parser data files
42- backupFileDir - the directory, relative to the run directory, to store backups of
43the data files in the appDataFileDir
44- harvestDir - the directory to harvest OAI documents to
45
46policy.xml - see the security section for configuring this
47
48
49ii) Add a config file for the postgres DB used by the discovery service; this is required
50when documents are harvested via the editor - since once the harvest is complete, the
51data is automatically ingested into the discovery service.  The file should be in
52the oai_info_editor directory and be called, 'ingest.config'.  The contents should
53be of the format:
54
55databaseName hostName userName password portNumber
56
57(NB, portNumber is optional)
58
59iii) From the oai_info_editor directory, run:
60
61paster serve --reload development.ini
62
63
64Running with security
65----------------------------
66The application is configured to work with the openid based security system available
67with the ndg.security package.  To set it up with this, complete the following:
68
691) Get Phil Kershaw to generate specific security certificate and key for your app
70- these will be refered to, below as app-security.crt and app-security.key, respectively.
71Also get the server encryption certification - we will call this cacert.pem, below.
72 
732) Install the various ndg.security eggs - this can be done, e.g. via:
74
75sudo easy_install -Uf http://ndg.nerc.ac.uk/dist/ ndg_security ndg_security_common ndg_security_server ndg_security_test
76
773) In the installed eggs, go to:
78
79ndg/security/test/integration/authz
80
81and edit the securityservices.ini file as follows:
82
83a) Set the following line (or choose an alternative openid provider):
84
85openid.relyingparty.signinInterface.initialOpenID = https://ndg3beta.badc.rl.ac.uk/openid
86
87b) Set the following:
88
89openid.relyingparty.certFilePath = %(testConfigDir)s/pki/wsse-server.crt
90openid.relyingparty.priKeyFilePath = %(testConfigDir)s/pki/wsse-server.key
91openid.relyingparty.priKeyPwd =
92openid.relyingparty.caCertDirPath = %(testConfigDir)s/ca
93
944) Ensure the crt and key files are available in the config directory:
95
96ndg/security/test/config/pki
97
98and the cacert.pem is in:
99
100ndg/security/test/config/ca
101
102- also, adjust the port number and hostname, if required; this is where this security
103service will be visible from.
104
1055) Start the securityservice app by running:
106
107ndg/security/test/integration/authz/securityservicesapp.py
108
109- this should start the security server - with the logging saying something like:
110
111serving on 0.0.0.0:7443 view at http://127.0.0.1:7443
112
113- if not, diagnose and fix the problems (most likely install issues).
114
1156) Adjust the content of OAIInfoEditor/secured.ini as follows:
116
117a)
118authN.redirectURI = http://localhost:7443/verify
119- this should point to the service set up to run in (5)
120
121b)
122pip.caCertFilePathList=%(here)s/ca/cacert.pem
123- this should point at the local version of cacert.pem - i.e. ensure there is a copy
124of this in the local app ca dir
125
126c)
127pip.wssecurity.signingCertFilePath=%(here)s/pki/app-security.crt
128pip.wssecurity.signingPriKeyFilePath=%(here)s/pki/app-security.key
129pip.wssecurity.caCertFilePathList=%(here)s/ca/cacert.pem
130- again, ensure there are copies of the .crt and .key files in the local pki directory
131
132d)
133pip.wssecurity.addTimestamp=False
134- this avoids a current issue with ntp time synchronisation
135
1367) The app should then be able to be started in secure mode via:
137
138paster serve --reload secured.ini
139
140
141Adding new users + enforcing security
142-------------------------------------------
143Security is enforced by the policy.xml file; this sets what access is required to
144view what URLs.  When a new provider user is added to the system, the template in the
145policy file needs to be updated.  This should look something like:
146
147    <Target>
148        <URIPattern>^/.*badc$</URIPattern>
149        <Attributes>
150            <Attribute>urn:badc:security:authz:1.0:attr:oai_editor:badc</Attribute>
151        </Attributes>
152        <AttributeAuthority>
153            <uri>https://ndg3beta.badc.rl.ac.uk/AttributeAuthority</uri>
154        </AttributeAuthority>
155    </Target>
156
157
158- in this case, any URL which ends in 'badc' requires the user to have the 'badc' role
159in order to view.  Every new provider in the system (e.g. by an admin user) will need
160a new target added to enforce the required security.  Phil Kershaw can then set up
161any required users for this provider.
162
163In the case of admin users, all admin functions are accessed by appending '?admin=1'
164parameter to the URL - and this is handled by the target:
165
166    <Target>
167        <URIPattern>^/.*?admin=1$</URIPattern>
168        <Attributes>
169            <Attribute>urn:badc:security:authz:1.0:attr:oai_editor:admin</Attribute>
170        </Attributes>
171        <AttributeAuthority>
172            <uri>https://ndg3beta.badc.rl.ac.uk/AttributeAuthority</uri>
173        </AttributeAuthority>
174    </Target>
175
176- this keeps things simple.  New admin users just need to have the role,
177
178urn:badc:security:authz:1.0:attr:oai_editor:admin
179
180added.
181
182NB, the current system contains the following users (user ID, openID url, roles):
183
184cbyrom         https://ndg3beta.badc.rl.ac.uk/openid/Calum.Byrom        urn:badc:security:authz:1.0:attr:oai_editor:admin
185aharwood     https://ndg3beta.badc.rl.ac.uk/openid/Andrew.Harwood  urn:badc:security:authz:1.0:attr:oai_editor:badc
186rlowry          https://ndg3beta.badc.rl.ac.uk/openid/Roy.Lowry            urn:badc:security:authz:1.0:attr:oai_editor:bodc
187
188- their login passwords are available from Phil Kershaw.
189
190
191Using the Editor
192------------------
193A help guide is available from the web app - click on the link, 'help' for detailed
194instructions on how to use.
195
196
197Tests
198------------
199A suite of unit tests is available under oai_info_editor/tests - the file, nostTests.py,
200in this directory, can be used to discover and run all the tests via the python nose module -
201just run it as a python app.
202
203
204 Further Notes
205-----------------
206i) The editor uses the OAIBatch codebase to do the ingest - using the same
207script, oai_document_ingester, that is used by the discovery service
208ii) NB, this work replaces the use of the jOAI harvester web app.  Although the jOAI codebase has
209an exposed java API to complete harvesting (which is used by this editor), in a consistent manner
210to the web app, it doesn't have any public API to curate the data contained by the web app.  Since
211the only additional functionality offered by the jOAI system was increased logging, and since this
212wasn't being used anyway, it was considered simpler just to write a new system to allow
213curation of provider info.
Note: See TracBrowser for help on using the repository browser.