Changes between Version 50 and Version 51 of Software/MSI/ConfigurationNeptune


Ignore:
Timestamp:
25/11/10 11:30:45 (9 years ago)
Author:
sdonegan
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Software/MSI/ConfigurationNeptune

    v50 v51  
     1  
     2 
    13= Neptune Configuration = 
     4 
    25[[PageOutline]] 
     6 
    37Neptune is the test server for NDG3 (MSI). 
    4 {{{ 
     8 
     9{{{ 
     10 
    511$ cat /etc/issue 
    612 
     13  
     14 
    715Welcome to openSUSE 10.3 (X86-64) - Kernel \r (\l). 
    8 }}} 
     16 
     17}}} 
     18 
    919Neptune has an alias ndg3beta.badc.rl.ac.uk. 
    1020 
     21  
     22 
    1123== Known Issues == 
     24 
    1225 * ssh login fails with: 
    13 {{{ 
     26 
     27{{{ 
     28 
    1429$ ssh neptune.badc.rl.ac.uk 
     30 
    1531Password: 
     32 
    1633Permissions on the password database may be too restrictive. 
    1734 
     35  
     36 
    1837Password: 
    19 }}} 
     38 
     39}}} 
     40 
    2041   ... indicates the store mount has failed.  Contact [mailto:sddcs@rl.ac.uk SDDCS] 
     42 
    2143 * wildly inaccurate dates causing unpredictable behaviour.  Contact [mailto:sddcs@rl.ac.uk SDDCS] 
    2244 
     45  
     46 
    2347== Python Configuration == 
     48 
    2449System default is Python 2.5 in `/usr/bin`.  Under SuSE, the site package location is customised to `/usr/local/lib64/python2.5/site-packages` with `/usr/lib64/python2.5/distutils/distutils.cfg`. 
    2550 
     51  
     52 
    2653Application packages will be installed separately to avoid version conflicts and maintenance problems with a single package area.  `virtualenv` or `zc.buildout` could achieve this.  `virtualenv` is easy to set-up with `mod_wsgi` - see [wiki:Software/MSI/ConfigurationNeptune/ApacheConfiguration Apache Configuration].  `zc.buildout` enables overriding control over package versions to define a package and version combination to make a stable deployment.  `zc.buildout` [http://pypi.python.org/pypi/collective.recipe.modwsgi collective.recipe.modwsgi] enables integration with `mod_wsgi`.  `zc.buildout` is currently the preferred means of configuration (17/06/2009). 
    2754 
     55  
     56 
    2857=== Virtualenv === 
     58 
    2959virtualenv bootstrap: 
    30 {{{ 
     60 
     61{{{ 
     62 
    3163$ virtualenv --no-site-packages myenv 
    32 }}} 
     64 
     65}}} 
     66 
    3367... failed with this message: 
    34 {{{ 
     68 
     69{{{ 
     70 
    3571TEST FAILED: /usr/local/lib64/python2.5/site-packages/ does NOT support .pth files  
    36 }}} 
     72 
     73}}} 
     74 
     75  
    3776 
    3877This is a known problem with SuSE: 
    3978 
     79  
     80 
    4081http://groups.google.com/group/python-virtualenv/browse_thread/thread/aa69f8b738d23652 
    4182 
     83  
     84 
    4285This discussion suggests commenting out the prefix setting in `/usr/lib64/python2.5/distutils/distutils.cfg`, but a less intrusive option is to override the setting by creating an [http://docs.python.org/install/index.html#alternate-installation-unix-the-prefix-scheme alternative] config file `setup.cfg` or `~/.pydistutils.cfg`: 
    4386 
    44 {{{ 
     87  
     88 
     89{{{ 
     90 
    4591$ cat > ./setup.cfg 
     92 
    4693[build_py] 
     94 
    4795optimize=0 
    4896 
     97  
     98 
    4999[install] 
     100 
    50101prefix=/mypath/myenv 
     102 
    51103optimize=0 
    52 }}} 
     104 
     105}}} 
     106 
    53107Then ... 
    54 {{{ 
     108 
     109{{{ 
     110 
    55111$ virtualenv --no-site-packages myenv 
    56 }}} 
     112 
     113}}} 
     114 
     115  
    57116 
    58117Move the `setup.cfg` file to within the `virtualenv` directory so that it doesn't interfere with other components:  
     118 
    59119{{{  
     120 
    60121$ mv ./setup.cfg ./myenv  
     122 
    61123}}}  
     124 
    62125Install setuptools `cd`ing to `myenv` directory first to ensure that `setup.cfg` is picked up:  
     126 
    63127{{{  
     128 
    64129$ cd ./myenv $ wget http://peak.telecommunity.com/dist/ez_setup.py 
     130 
    65131$ ./bin/python ./ez_setup.py 
    66 }}} 
     132 
     133}}} 
     134 
     135  
    67136 
    68137=== zc.buildout === 
     138 
    69139See the [wiki:Software/MSI/ConfigurationNeptune#DiscoveryBrowserInterface Discovery service browser interface section] for details. 
    70140 
     141  
     142 
    71143== Apache Configuration == 
     144 
    72145See: [wiki:Software/MSI/ConfigurationNeptune/ApacheConfiguration Apache Configuration] page. 
    73146 
     147  
     148 
    74149== Application Configuration == 
    75150 
     151  
     152 
    76153=== Discovery Browser Interface === 
     154 
    77155The discovery service has been deployed with a buildout script (as of 04/06/2009).  The configuration is in `/usr/local/ndg-discovery`.  The procedure to set up was: 
    78156 
     157  
     158 
    79159 1. Install zc.buildout and [http://pypi.python.org/pypi/collective.recipe.modwsgi/1.1 collective.recipe.modwsgi]: 
     160 
    80161    {{{ 
     162 
    81163$ sudo /usr/local/bin/easy_install zc.buildout 
     164 
    82165$ sudo /usr/local/bin/easy_install collective.recipe.modwsgi 
     166 
    83167    }}} 
     168 
    84169 1. The Apache config needs altering to add a mount point for the Discovery service: 
     170 
    85171    {{{ 
     172 
    86173WSGIScriptAlias /services /srv/www/wsgi_scripts/discovery.wsgi 
     174 
    87175    }}} 
     176 
    88177 1. Create the buildout.cfg `/usr/local/ndg-discovery`: 
     178 
    89179    {{{ 
     180 
    90181[buildout] 
     182 
    91183parts = DiscoveryServiceProGlueMirror 
    92184 
     185  
     186 
    93187# Configuration mirroring eggs as currently deployed on proglue 
     188 
    94189[DiscoveryServiceProGlueMirror] 
     190 
    95191recipe = collective.recipe.modwsgi 
    96192 
     193  
     194 
    97195eggs = ows_server==0.0.0dev_r5354 
     196 
    98197        ows_common==0.1dev_r2969 
     198 
    99199        ndgUtils==0.1.2.dev_r4896 
     200 
    100201        csml==2.0b_r3046 
     202 
    101203        cdat_lite>=4.1.2_0.2.5 
     204 
    102205#       cdat_lite==4.1.2_0.2.5 
     206 
    103207        Pylons==0.9.6.1 
     208 
    104209        WebHelpers==0.3.2 
     210 
    105211config-file = ${buildout:directory}/service.ini 
     212 
    106213find-links = http://ndg.nerc.ac.uk/dist 
     214 
    107215        http://ndg.nerc.ac.uk/dist/archivedcsml 
     216 
    108217    }}} 
     218 
    109219    The `DiscoveryServiceProGlueMirror` part mirrors the configuration on `proglue`: 
     220 
    110221     * Explicit `Pylons` and `WebHelpers` versions were set to avoid `webhelpers` '`auto_link`' `AttributeError`. 
     222 
    111223     * `cdat_lite` should be set to version 4 but this wouldn't install.  Version 5 installs but there is a known error with cdms imports.  This will be fixed with an upgrade to the latest version of discovery. 
     224 
    112225 1. To generate the configuration run buildout from `/usr/local/ndg-discovery`. 
     226 
    113227    {{{ 
     228 
    114229$ /usr/local/bin/buildout 
     230 
    115231    }}} 
     232 
    116233    This will create a WSGI script in `./parts/DiscoveryServiceProGlueMirror/wsgi` 
     234 
    117235 1. The script as generated could be improved to enable logging by adding a line to include `fileConfig` from `paste.script.util.logging_config` e.g.: 
     236 
    118237    {{{ 
     238 
    119239from paste.deploy import loadapp 
     240 
    120241from paste.script.util.logging_config import fileConfig 
    121242 
     243  
     244 
    122245configFilePath = '/usr/local/myapp/service.ini' 
     246 
    123247fileConfig(configFilePath) 
    124248 
     249  
     250 
    125251application = loadapp('config:%s' % configFilePath) 
     252 
    126253    }}} 
     254 
    127255    This could be done by customising the `collective.recipe.modwsgi` recipe.  As an interim measure. a Makefile makes the changes using an ugly set of `sed` commands :)  The Makefile also includes targets to install the script into the right area for Apache to pick up: 
     256 
    128257    {{{ 
     258 
    129259WSGI_DIR=/srv/www/wsgi_scripts 
     260 
    130261WSGI_SCRIPT_NAME=discovery.wsgi 
     262 
    131263WSGI_SCRIPT_IN_FILE=./parts/DiscoveryService/wsgi 
     264 
    132265TMP_FILE=${WSGI_SCRIPT_IN_FILE}.tmp 
    133266 
     267  
     268 
    134269install_wsgi: add_logging 
     270 
    135271        @echo installing WSGI script ... 
     272 
    136273        cp ${TMP_FILE} ${WSGI_DIR}/${WSGI_SCRIPT_NAME} 
     274 
    137275        @echo Done. 
    138276 
     277  
     278 
    139279add_logging: 
     280 
    140281        @echo Altering WSGI script file to include logging functionality ... 
     282 
    141283        @cat ${WSGI_SCRIPT_IN_FILE} | sed s/"application = loadapp(\"config:"/"from paste.script.util.logging_config import fileConfig\n\nconfigFilePath 
     284 
    142285 = \""/g |sed s/"\")"/"\"\nfileConfig(configFilePath)\n\napplication = loadapp(\"config:\"+configFilePath)"/g > ${TMP_FILE} 
     286 
    143287        @echo Done. 
    144288 
     289  
     290 
    145291http_proxy=http://wwwcache.rl.ac.uk:8080 
    146292 
     293  
     294 
    147295buildout: 
     296 
    148297        export http_proxy=${http_proxy}; /usr/local/bin/buildout 
     298 
    149299    }}} 
     300 
    150301    To build and install then, 
     302 
    151303    {{{ 
     304 
    152305    $ make buildout 
     306 
    153307    $ make 
     308 
    154309    }}} 
     310 
    155311    `mod_wsgi` set up in daemon mode will automatically reload the script content without the need to restart Apache. 
     312 
    156313 1. Nb. Ensure `debug = false` in the ini file config!  Check `ndgDiscovery.config` `server` setting consistent with the mount point set by `WSGIScriptAlias` in the Apache config file. 
    157314 
     315  
     316 
     317== Tomcat configuration == 
     318 
     319In order to minimize the number of installed Tomcat a "multiple instances" configuration has been setup. 
     320 
     321Apache Tomcat 6.0.24  (CATALINA_HOME: /opt/apache-tomcat-6.0.24) 
     322 
     323Java6 (JAVA_HOME: /opt/SDK6/jdk) 
     324 
     325  
     326 
    158327=== Discovery Web Service === 
    159328 
    160329To run the Discovery backend server the following has been installed: 
    161330 
    162 Apache Tomcat 6.0.18  (/usr/local/apache/apache-tomcat-6.0.18) 
    163  
    164 Tomcat management info can be found in conf/tomcat-users.xml 
    165  
    166 Apache Axis Axis2-1.4 (/usr/local/axis2/axis2-1.4)  (Axis2.war deployed within tomcat) 
    167  
    168 Apache Ant 1.7.1 (/usr/local/ant) 
    169  
    170 Updates to /etc/init.d/tomcat have been made to allow tomcat to stop/start Discovery service backend.  Following environment variables need to be set: 
    171  
    172 export JAVA_HOME=/usr/java/jdk1.5.0_08 
    173 export JRE_HOME=/usr/java/jdk1.5.0_08/jre 
    174 export CATALINA_HOME=/usr/local/apache/apache-tomcat-6.0.18 
    175 export AXIS2_HOME=/usr/local/axis2/axis2-1.4 
    176  
    177 Updates to catalina.sh in $CATALINA_HOME/bin/ need to be made: 
    178  
    179 JAVA_OPTS="-Xmx1024m -Dhttp.proxyPort=8080 -Dhttp.proxyHost=wwwcache.rl.ac.uk -Dhttp.nonProxyHosts=\"*.rl.ac.uk|localhost\"" 
    180 JRE_HOME="/usr/java/jdk1.5.0_08/jre" 
     331  
     332 
     333 * develop configuration (CATALINA_BASE: /opt/tomcatInstances/develop) 
     334 
     335  
     336 
     337 * Apache Axis Axis2-1.4 ($CATALINA_BASE/webapps/axis2)  (Axis2.war deployed within tomcat) 
     338 
     339  
     340 
     341 * A configuration file ($CATALINA_BASE/bin/setenv.sh) defines 
     342 
     343export JAVA_HOME=/opt/SDK6/jdk 
     344 
     345export JRE_HOME=$JAVA_HOME/jre 
     346 
     347export CATALINA_HOME=/opt/apache-tomcat-6.0.24 
     348 
     349export CATALINA_BASE=/opt/tomcatInstances/develop 
     350 
     351export JAVA_OPTS="-Xmx512M -Xms64M -Dfile.encode=UTF-8 -XX:MaxPermSize=256M -Dhttp.proxyPort=8080 -Dhttp.proxyHost=wwwcache.rl.ac.uk -Dhttp.nonProxyHosts=\"*.rl.ac.uk|localhost\"" 
     352 
     353export AXIS2_HOME=$CATALINA_BASE/webapps/axis2/axis2-1.4 
     354 
     355  
     356 
     357* The db resource ($CATALINA_BASE/conf/context.xml) has to contains the following resurces 
     358 
     359  
     360 
     361    <Resource name="jdbc/discoveryDB" auth="Container" 
     362 
     363          type="javax.sql.DataSource" driverClassName="org.postgresql.Driver" 
     364 
     365          url="jdbc:postgresql://neptune.badc.rl.ac.uk:5432/discovery" 
     366 
     367          removeAbandoned="true" removeAbandonedTimeout="60" logAbandoned="true" 
     368 
     369          username="xxxx" password="yyyy" maxActive="100" maxIdle="30" maxWait="10000"/> 
     370 
     371  
     372 
     373  
     374 
     375   <Resource name="jdbc/searchLogDB" auth="Container" 
     376 
     377          type="javax.sql.DataSource" driverClassName="org.postgresql.Driver" 
     378 
     379          url="jdbc:postgresql://neptune.badc.rl.ac.uk:5432/searchLog" 
     380 
     381          removeAbandoned="true" removeAbandonedTimeout="60" logAbandoned="true" 
     382 
     383          username="gggg" password="zzzz" maxActive="100" maxIdle="30" maxWait="10000"/> 
     384 
     385  
     386 
     387  
     388 
     389* The service /etc/init.d/tomcat has to been made to allow tomcat to stop/start Discovery service backend. I should define the following environment variables: 
     390 
     391export CATALINA_HOME=/opt/apache-tomcat-6.0.24 
     392 
     393export CATALINA_BASE=/opt/tomcatInstances/develop   
     394 
     395  
     396 
     397* This service will run on port 8080 
     398 
     399  
     400 
     401=== GeoNetwork === 
     402 
     403To run the Geonetwork application the following has been installed: 
     404 
     405  
     406 
     407* develop (CATALINA_BASE: /opt/tomcatInstances/gnTomcatBase) 
     408 
     409  
     410 
     411* A configuration file ($CATALINA_BASE/bin/setenv.sh) defines 
     412 
     413export JAVA_HOME=/opt/SDK6/jdk 
     414 
     415export JRE_HOME=$JAVA_HOME/jre 
     416 
     417export CATALINA_HOME=/opt/apache-tomcat-6.0.24 
     418 
     419export CATALINA_BASE=/opt/tomcatInstances/geonetwork 
     420 
     421export JAVA_OPTS="-Xmx512M -Xms64M -Dfile.encode=UTF-8 -XX:MaxPermSize=256M -Dhttp.proxyPort=8080 -Dhttp.proxyHost=wwwcache.rl.ac.uk -Dhttp.nonProxyHosts=\"*.rl.ac.uk|localhost\"" 
     422 
     423export AXIS2_HOME=$CATALINA_BASE/webapps/axis2/axis2-1.4 
     424 
     425  
     426 
     427* Updates to /etc/init.d/geonetwork have been made to allow GeoNetwork's tomcat to stop/start Discovery service backend.   
     428 
     429Following environment variables need to be set: 
     430 
     431export CATALINA_HOME=/opt/apache-tomcat-6.0.24 
     432 
     433export CATALINA_BASE=/opt/tomcatInstances/geonetwork 
     434 
     435  
     436 
     437* This service will run on port 8280 
     438 
     439  
    181440 
    182441==== Postgres database ==== 
    183442 
     443  
     444 
    184445Postgres 8.3.1 has been installed in the standard location: /usr/local/pgsql. The database data files and configuration files are stored in the 'data' subdirectory. The postgis extension has also been installed. This database is not currently being backed up. 
    185446 
     447  
     448 
    186449Postgres should automatically start on reboot via /etc/init.d/postgresql file. 
    187450 
     451  
     452 
    188453To start/stop/restart/reload postgres: 
    189454 
     455  
     456 
    190457As linux user 'postgres':   
    191458 
     459  
     460 
    192461/usr/local/pgsql/bin/pg_ctl ''start/stop/restart/reload/status'' -D /usr/local/pgsql/data 
    193462 
     463  
     464 
    194465'''OR''', As 'root': 
    195466 
     467  
     468 
    196469/etc/init.d/postgresql ''start/stop/restart/status'' 
    197470 
     471  
     472 
    198473Passwords can be found in the Secrets box. 
    199474 
     475  
     476 
     477  
    200478 
    201479=== Discovery NDG URL redirection service === 
    202480 
     481  
     482 
    203483The NDG redirection service redirects links to the original URL, once the original URL has been replaced during discovery ingestion with the relevant details.  Before the required link is redirected details on the link and associated dataset are recorded within a logging database as well as a counter incremented in the original database for the relevant dataset. 
    204484 
     485  
     486 
    205487A converted redirected URL looks like this: 
    206488 
     489  
     490 
    207491http://ndg3beta.badc.rl.ac.uk/URL_redirect_NDG3/ndgURLredirect/redirect?url=http%3A//www.neodc.rl.ac.uk&amp;docID=neodc.nerc.ac.uk%3ADIF%3Adataent_11924879127625221&amp;docTitle=NCAVEO%20field%20experiment%20data 
    208492 
     493  
     494 
    209495and redirects the link to the original specified url at http://www.neodc.rl.ac.uk/ .  Dataset title and unique id are also encoded so as to allow easy dissemination of which datasets produce the most traffic. 
    210496 
     497  
     498 
    211499The NDG redirection service endpoint is: http://ndg3beta.badc.rl.ac.uk/URL_redirect_NDG3/ndgURLredirect/redirect and requires the addition of 3 parameters to allow parsing and update of the database: 
    212500 
     501  
     502 
    213503url, docID & docTitle.  These need to be encoded using UTF-8 so they can be parsed correctly by the service. 
    214504 
     505  
     506 
    215507The service is available as [http://ndg.nerc.ac.uk/dist/ndgRedirect-1.0.0dev_r5460-py2.5.egg egg] which can be run within a paster buildouts session (you will need to unpack service.ini as well as config files for both the url tracking database and the main discovery database - not in the egg - contact steve.donegan@stfc.ac.uk).  Dependancies etc as required are detailed in the configs below 
    216508 
     509  
     510 
    217511Buildout script to run paster on 8081: 
    218512 
     513  
     514 
    219515TBC 
    220516 
     517  
     518 
    221519Run as mod_wsgi script with buildout 
    222520 
     521  
     522 
    223523TBC 
    224524 
     525  
     526 
     527  
    225528 
    226529=== COWS === 
     530 
    227531There is a virtualenv based python under /usr/local/cows_virtualenv 
     532 
    228533This python has numpy and matplotlib installed. This python is then used to run buildout scripts for both the COWS WXS Services, and the COWS Client code. 
    229534 
     535  
     536 
    230537==== COWS Services ==== 
     538 
    231539A buildout script is in /usr/local/cowsserver_buildout/ 
    232540 
    233 {{{ 
     541  
     542 
     543{{{ 
     544 
    234545[buildout] 
     546 
    235547parts = cows_server_app 
    236548 
     549  
     550 
    237551[cows_components] 
     552 
    238553recipe = zc.recipe.egg 
     554 
    239555interpreter = py 
    240556 
     557  
     558 
    241559find-links = http://ndg.nerc.ac.uk/dist 
    242560 
     561  
     562 
    243563eggs= 
     564 
    244565    Pylons==0.9.6.1 
     566 
    245567    csml 
     568 
    246569    shapely==1.0.12 
     570 
    247571    ndg-security-server 
     572 
    248573    cows 
    249574 
     575  
     576 
    250577[cows_server_app] 
     578 
    251579recipe = collective.recipe.modwsgi 
     580 
    252581find-links = 
     582 
    253583        /usr/local/cows_virtualenv/cows_server_app/dist 
     584 
    254585        http://ndg.nerc.ac.uk/dist 
     586 
    255587        http://dist.repoze.org/ 
    256588 
     589  
     590 
    257591eggs= 
     592 
    258593  Pylons==0.9.6.1 
     594 
    259595  csml 
     596 
    260597  shapely==1.0.12 
     598 
    261599  AuthKit==0.4.3ndg_r174 
     600 
    262601  ndg-security-server 
     602 
    263603  cows 
     604 
    264605  PIL 
     606 
    265607  OGCTestBed 
     608 
    266609  SQLAlchemy 
     610 
    267611config-file = /usr/local/cows_virtualenv/cows_server_app/development.ini 
    268612 
    269  
    270 }}} 
     613  
     614 
     615  
     616 
     617}}} 
     618 
     619  
    271620 
    272621There is a bootstrap script in the cowsserver_buildout directory, to run it use: 
    273 {{{ 
     622 
     623{{{ 
     624 
    274625 /usr/local/cows_virtualenv/bin python bootstrap.py 
    275 }}} 
     626 
     627}}} 
     628 
    276629This creates a custom buildout script.  Run this: 
    277 {{{ 
     630 
     631{{{ 
     632 
    278633 /usr/local/cowsserver_buildout/bin/buildout 
    279 }}} 
     634 
     635}}} 
     636 
     637  
    280638 
    281639The generated WSGI script `./parts/cows_server_app/wsgi` should be installed in the Apache WSGI scripts directory. 
    282640 
     641  
     642 
    283643Note that the pylons ini file is in: 
    284 {{{ 
     644 
     645{{{ 
     646 
    285647/usr/local/cows_virtualenv/cows_server_app/development.ini 
    286 }}} 
     648 
     649}}} 
     650 
     651  
    287652 
    288653/usr/local/cows_virtualenv/cows_server_app/ contains a lightweight Pylons app that sets up the controllers for COWS. This app was created using COWS project-templates. 
    289654 
     655  
     656 
    290657TODO: The bootstrap script needs modifying so it adds logging capabilities to the final WSGI document (as per Phil's makefile). 
    291658 
     659  
     660 
    29266105/08/2009 - Phil has added a similar Makefile.  The procedure for update and install is: 
    293 {{{ 
     662 
     663{{{ 
     664 
    294665$ make bootstrap 
     666 
    295667$ make 
    296 }}} 
     668 
     669}}} 
     670 
     671  
    297672 
    298673==== COWS WMS/WCS Client ==== 
    299674 
     675  
     676 
    300677There is a buildout config for the 'cowsclient' app here: 
     678 
    301679/usr/local/cowsclient_buildout 
    302680 
    303 {{{ 
     681  
     682 
     683{{{ 
     684 
     685  
    304686 
    305687[buildout] 
    306688 
     689  
     690 
    307691parts = cows_client_app 
    308692 
     693  
     694 
    309695[cows_client_app] 
     696 
    310697recipe = collective.recipe.modwsgi 
     698 
    311699find-links = 
     700 
    312701        http://ndg.nerc.ac.uk/dist 
     702 
    313703eggs= 
     704 
    314705  AuthKit==0.4.3ndg_r174 
     706 
    315707  ndg-security-server 
     708 
    316709  cowsclient 
     710 
    317711config-file = ${buildout:directory}/appconfig/cowsclient.ini 
    318712 
    319 }}} 
     713  
     714 
     715}}} 
     716 
     717  
    320718 
    321719Again this should be run using the virtualenv python and bootscript.py 
    322 {{{ 
     720 
     721{{{ 
     722 
    323723/usr/local/cows_virtualenv/bin/python bootstrap.py 
    324 }}} 
     724 
     725}}} 
     726 
     727  
    325728 
    326729This creates a custom buildout script.  Run this: 
    327730 
    328 {{{ 
     731  
     732 
     733{{{ 
     734 
    329735 /usr/local/cowsserver_buildout/bin/buildout 
    330 }}} 
     736 
     737}}} 
     738 
     739  
    331740 
    332741The generated WSGI script `./parts/cows_client_app/wsgi` should be installed in the Apache WSGI scripts directory. 
    333742 
     743  
     744 
    334745Alternatively, use the Makefile: 
    335 {{{ 
     746 
     747{{{ 
     748 
    336749$ make bootstrap 
     750 
    337751$ make 
    338 }}} 
     752 
     753}}} 
     754 
     755  
    339756 
    340757=== pyDAP === 
     758 
    341759pyDAP 3.0 is installed in `/usr/local/dap`.  This is being updated (17/06/2009) to use a `zc.buildout` configuration: 
    342760 
    343 {{{ 
     761  
     762 
     763{{{ 
     764 
    344765[buildout] 
     766 
    345767parts = pyDAP 
    346768 
     769  
     770 
    347771# Configuration mirroring eggs as currently deployed on proglue 
     772 
    348773[pyDAP] 
     774 
    349775recipe = collective.recipe.modwsgi 
    350776 
     777  
     778 
    351779eggs = ndg_security 
     780 
    352781       Pydap==3.0.b.4 
     782 
    353783       pydap.handlers.netcdf==0.4.4 
     784 
    354785       pydap.responses.netcdf==0.1 
     786 
    355787config-file = ${buildout:directory}/etc/service.ini 
    356788 
     789  
     790 
    357791find-links = http://ndg.nerc.ac.uk/dist 
    358 }}} 
     792 
     793}}} 
     794 
     795  
    359796 
    360797The eggs list includes NetCDF response and handler plugins and NDG Security filter to intercept requests. 
    361798 
     799  
     800 
    362801=== Security === 
     802 
    363803There are two components: 
     804 
    364805 1. '''Security Services''': an application running security services such as OpenID, Attribute Authority and Session Management   
     806 
    365807 1. '''Application Filters''': handler filters which are configured with existing applications to protect them 
    366808 
     809  
     810 
     811  
    367812 
    368813The first is installed in it's own mod_wsgi application running over HTTPS.  For the second, there are filters configured to secure COWS and pyDAP services.  
    369   
     814 
     815  
     816 
    370817==== Security Services ==== 
     818 
    371819This is installed using the same technique as described above for the Discovery Service Browser interface: a buildout script installs the eggs required in `/usr/local/ndg-security/eggs` and creates a `mod_wsgi` script.  A Makefile installs the script in the script location set-up for Apache `mod_wsgi` scripts.   The script is mounted via a `WSGIScriptAlias` directive in the Apache config file. 
     820 
    372821{{{  
     822 
    373823[buildout] 
     824 
    374825parts = NDGSecurity 
    375826 
     827  
     828 
    376829[NDGSecurity] 
     830 
    377831recipe = collective.recipe.modwsgi 
     832 
    378833interpreter = py 
    379834 
     835  
     836 
    380837# SQLAlchemy is used by OpenID / Session Manager for authentication Database 
     838 
    381839eggs = 
     840 
    382841    AuthKit==0.4.3ndg_r174 
     842 
    383843    ndg_security 
     844 
    384845    ndg_security_test 
     846 
    385847    SQLAlchemy 
     848 
    386849config-file = ${buildout:directory}/config/securityservices.ini 
     850 
    387851find-links = http://ndg.nerc.ac.uk/dist 
    388 }}} 
     852 
     853}}} 
     854 
     855  
    389856 
    390857See the `README` file in `/usr/local/ndg-security` for additional configuration information. 
    391858 
     859  
     860 
    392861==== Application Filters ==== 
     862 
    393863Each application secured with NDG Security is configured with security filters to intercept requests to them.  These are applied by making settings in the Paste ini file for the application.  This ini file extract shows how the main application is configured at the end of a WSGI pipeline fronted with Authentication and Authorisation filters: 
    394864 
    395 {{{ 
     865  
     866 
     867{{{ 
     868 
    396869[pipeline:main] 
     870 
    397871pipeline = AuthenticationFilter AuthorizationFilter myApp 
    398 }}} 
     872 
     873}}} 
     874 
     875  
    399876 
    400877`AuthenticationFilter`, `AuthorizationFilter` and `myApp` refer to sub-sections in the file where the specific settings are for each individual component. 
    401878 
     879  
     880 
    402881The filters make use of services running in the Security Services application stack described above.  The authentication filter is configured to invoke the OpenID Relying Party interface running in the Security Services stack.  This prompts the user to enter the OpenID for their home site. 
    403882 
     883  
     884 
    404885The authorisation filter is configured with an XML policy file which sets which request URIs are to be secured.  It also makes callouts to the Attribute Authority and Session Manager web services which similarly, run on the Security Services stack. 
    405886 
     887  
     888 
    406889pyDAP and COWS services are secured using this configuration. 
    407890 
     891  
     892 
    408893=== Vocab Term Editor === 
     894 
    409895Configuration is set in `/usr/local/vocab-editor`.  Ownership should be set so that the Apache user has read permission.   
    410 {{{ 
     896 
     897{{{ 
     898 
    411899[buildout] 
     900 
    412901parts = Vocab_Term_Editor 
    413902 
     903  
     904 
    414905# Configuration mirroring eggs as currently deployed on proglue 
     906 
    415907[Vocab_Term_Editor] 
     908 
    416909recipe = collective.recipe.modwsgi 
     910 
    417911extra-paths = ${buildout:directory}/passwords 
    418912 
     913  
     914 
    419915eggs =  ndgCommon==0.1.1.dev_r5445 
     916 
    420917        Pylons==0.9.6.2 
     918 
    421919        PasteScript 
     920 
    422921        WebHelpers==0.3.2 
     922 
    423923        VocabTermEditor==0.0.0dev_r6101 
     924 
    424925        Routes==1.7.3 
     926 
    425927        pysvn 
     928 
    426929        PyGreSQL 
     930 
    427931        AuthKit==0.4.3ndg_r174 
     932 
    428933        ndg_security_server 
     934 
    429935config-file = ${buildout:directory}/production.ini 
     936 
    430937find-links = http://ndg.nerc.ac.uk/dist 
     938 
    431939             ./pysvn/pysvn-1.7.0/dist 
    432 }}} 
     940 
     941}}} 
     942 
    433943PySVN is not installable from an egg so an egg was created locally for PySVN from the tar ball distribution and this referenced with the `find-links` option above. 
    434944 
     945  
     946 
    435947==== PySVN Egg creation ==== 
     948 
    436949PySVN is not available as an egg so it has to be adapted (all commands 
     950 
    437951executed as root)... 
    438952 
     953  
     954 
    439955Get Dependencies - the svn development package: 
    440 {{{ 
     956 
     957{{{ 
     958 
    441959$ yast2 
    442 }}} 
     960 
     961}}} 
     962 
    443963 * Navigate to `Software` -> `Software Management` 
     964 
    444965 * <Alt+S> to search and enter 'subversion' 
     966 
    445967 * pick svn-devel package and select with <Alt+T> and '+' key. 
     968 
    446969 * <Alt+U> to update 
     970 
    447971 * <Alt+N> - no in response to option to install or remove more packages 
     972 
    448973 * <Alt+Q> to quit 
    449974 
     975  
     976 
    450977Get tarball distribution: 
    451 {{{ 
     978 
     979{{{ 
     980 
    452981$ wget http://pysvn.barrys-emacs.org/source_kits/pysvn-1.7.0.tar.gz 
     982 
    453983$ tar zxvf pysvn-1.7.0.tar.gz 
     984 
    454985$ cd pysvn-1.7.0 
    455 }}} 
     986 
     987}}} 
     988 
    456989Follow to [http://pysvn.tigris.org/issues/show_bug.cgi?id=86] to get details 
     990 
    457991of how to patch PySVN to make it eggable.  See [http://pysvn.tigris.org/nonav/issues/showattachment.cgi/19/pysvn-egg.patch patch] 
    458992 
     993  
     994 
    459995for the patch.  Unfortunately this refers to older version of PySVN but it's 
     996 
    460997still possible to hack the changes into 1.7.0(!).  Additionally hacking 
     998 
    461999necessary to correctly link to libcom_err in /lib64 directory.  Follow the 
     1000 
    4621001list of steps in [http://pysvn.tigris.org/issues/show_bug.cgi?id=86 issue 86] entry to create the egg. 
    4631002 
     1003  
     1004 
    4641005==== Build and Installation ==== 
     1006 
    4651007A Makefile in the installation directory can be used to call `zc.buildout` and install the WSGI script created: 
    466 {{{ 
     1008 
     1009{{{ 
     1010 
    4671011$ make buildout 
     1012 
    4681013$ make 
    469 }}} 
     1014 
     1015}}} 
     1016 
    4701017The Apache configuration is set up with a `WSGIScriptAlias` to pick up the script from the target location. 
     1018 
     1019