wiki:NDGBrowseHowTo

Version 15 (modified by lawrence, 12 years ago) (diff)

more detailed instructions on installing browse/discovery

How to Install the NDG Browse Code

[This page is NOT FINISHED ... and is not a real guide to do anything YET, further, I *KNOW* this doesn't work as of January 25th]

(Intended for *full* NDG data providers: that is, data providers who *do* have their own browse repository of MOLES documents.)

Note that the new browse and discovery code is integrated using paste, so you will need to ensure you have all the paste code installed in your python. In the following instructions, I'll assume you have access to your python, and that it exists at /your/path/bin/python. You may not need all these steps depending on the state of your system, and you may have needed to ensure that you have a http proxy variable set! Note also that much of this will be vastly simplified when we eggify the installation!

  1. Setup fastcgi. Your apache configuration should have something like this in it:
    <IfModule mod_fastcgi.c>
      Alias /retrieve "/var/www/fastcgi/ndg.fcgi/retrieve"
      Alias /browse "/var/www/fastcgi/ndg.fcgi/browse"
      Alias /discovery "/var/www/fastcgi/ndg.fcgi/discovery"
      Alias /layout "/var/www/fastcgi/ndg.fcgi/layout"
      <Directory /var/www/fastcgi>
        SetHandler fastcgi-script
        Order allow,deny
        Allow from all
      </Directory>
      # following socket ought to be setup by paste via ndg.ini (bnl)
      FastCgiExternalServer /var/www/fastcgi/ndg.fcgi -socket /tmp/ndg_fastcgi.soc
    </IfModule>
    
  2. Make sure the pieces we need from paste are installed:
    /your/path/bin/easy_install paste
    /your/path/bin/easy_install PasteDeploy
    /your/path/bin/easy_install PasteScript
    /your/path/bin/easy_install flup
    /your/path/bin/easy_install wsgiutils
    
  3. If your python is 2.4 or less (it comes as standard in 2.5), you'll need elementtree:
    /your/path/bin/easy_install elementtree
    
  4. And now for the big one, we need a recent version of ZSI. I've used the default today, which pulled ZSI-2.0RC3. Watch out for this one because a later one might break. Let me know how this goes!:
    /your/path/bin/easy_install 
    
  5. Get the contents of TI07-MOLES/trunk/PythonCode/wsgi and put them into a suitable directory on your webserver, and change the ownership of the directory to apache (or whatever account you run your website under). This can most easily be done with the following command (which will give you a subversion working copy):
    svn co http://glue.badc.rl.ac.uk:/ndgsvn/TI07-MOLES/trunk/PythonCode/wsgi
    
  6. cd into that directory!
  7. Modify 00deploy.sh so that MYPYTHONBIN points to your python:
    MYPYTHONBIN=/your/path/bin/
    
  8. Modify the contents of ndg.ini so that the socket matches the description in your fastcgi setup, and it has the right configDir in the [DEFAULT] section. Here, for example, is what is running on glue (where the wsgi code is directory /var/www/ndg):
    [server:main]
    #use = egg:PasteScript#wsgiutils
    #host = localhost.localdomain
    #port = 8001
    use = egg:PasteScript#flup_fcgi_thread
    socket = /tmp/ndg_fastcgi.soc
    
    [DEFAULT]
    configDir = /var/www/ndg/
    
  9. Create a passwords.txt file and make sure it exists in the directory from which you run the deploy script (and check that you can't get access to that directory via the web!). It is for allowing the browse code access to your exist repository and should have the following format:
    your.repository.name exist_access_usercode password
    
  10. You will need to modify the ndgDiscovery.config file as well. You should only need to modify the entries in the default and layout sections.
  11. At this point you can check the underlying transport works, try
    python ndgSearch.py
    
    1. It should run some unittests (at the moment one might fail, but some should succeed), if not, then you have a problem with the underlying transport:
    2. You may find that you have a firewall problem, if you need to, you should set an http_proxy variable to get through, eg:
      http_proxy=http://wwwcache.rl.ac.uk:8080;export http_proxy
      
  12. You may need to make changes to 001deploy.sh (in particular the path to your python, and potentially you may not be running apache under the user apache).
  13. Make sure that the directory that holds all this code provides read/write access to the user apache (or whatever user your webserver runs under).
  14. Now you can try and run the service:
    ./001deploy.sh
    
  15. At the moment one needs to restart the process by hand after a system reboot.

Troubleshooting

The last step should produce a log file in your directory, and a file called paster.pid. The latter has the pid of the process. If it doesn't exist, something has gone wrong with your deployment, check the log files in your directory. If they don't exist, suspect that your webserver either has failed to fire up fastcgi or that apache can't write into your directory!

If you have trouble setting up wsgi and fastcgi, see if you can get the wsgiEnvTest code working. You will need to modify wsgiEnvTest.sh for your python path, possibly your web server user and modify wsgiEnvTest.ini for your wsgi server environment, but you should then be able to point your browser at  http://yourhost/browse, and see the wsgi environment variables!