Changes between Initial Version and Version 1 of TI12_Security/EggifyingNDGsecurity


Ignore:
Timestamp:
22/11/06 17:17:19 (14 years ago)
Author:
pjkersha
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • TI12_Security/EggifyingNDGsecurity

    v1 v1  
     1= Eggifying NDG Security = 
     2NDG packages security included must be as easy to install as possible if they are to get take up by users and institutions.   
     3 
     4The use of MyProxy, set-up of PKI and the number of 3rtd party python package dependencies makes the security installation more complicated.  As of the Alpha version, security had a distutils based setup.  This made installation of the python source more straightforward but does not address the issue of the installation of the required 3rd party packages.  Also, there is a need to separate server and client side code.  The client side is more light weight and so making this distinct makes the installation process much less onerous for those only interested in client side functionality e.g. application developers wishing to NDG-secure a resource. 
     5 
     6Use of Python Eggs (See http://peak.telecommunity.com/DevCenter/PythonEggs and http://peak.telecommunity.com/DevCenter/setuptools) should address some of these issues: 
     7 * setup scripts configured with the external dependencies they need and with the ability to automatically download and install these 
     8 * separate client and server side eggs 
     9 
     10== NDG Security Installation Status as of Alpha Stage == 
     11 * flat package structure: a single `NDG` package containing all the code 
     12 * distutils based setup to install it 
     13 * Separate installation required for !MyProxy, Globus !SimpleCA client package and OpenSSL and for 3rd party python dependencies. 
     14 
     15== How to make NDG Security Installation Easier == 
     16 * Separate eggs for client and server 
     17 * eggs have dependencies set for the 3rd party python packages they need so that these are found and automatically installed on setup 
     18 * Keep !MyProxy, Globus packages and OpenSSL install processes separate otherwise too complex.  - Egg setups could however be configured to detect whether these are in place and fail or give a warning if not there. 
     19 
     20== Proposed Package Structure == 
     21At the Python code review (http://proj.badc.rl.ac.uk/ndg/wiki/Packages) we agreed a hierarchical package structure.  Applying this to security instead of having a single `NDG` package we now have `ndg.security`. 
     22 
     23Also separate client and server packages would be useful for security.  For example, 
     24 
     25 1. a data provider wants to deploy an Attribute Authority but doesn't need session management.  They install the ndg.security.server.attAuthority egg. 
     26 1. Bryan's situation: he's developing the Discovery CGI.  He's only interested in client bindings to security services so he wants a ndg.security.client egg. 
     27 
     28For client and server installations there are now separate `client` and `server` packages.  However, there a `common` package is also needed because some modules are required by both.  Finally, the Attribute Authority, Session Manager and Simple CA services are distinct and so can be divided into separate packages [Log and Gatekeeper may be added].  The resulting structure is, 
     29 
     30{{{ 
     31#!python 
     32ndg/ 
     33    security/ 
     34        client/ 
     35            AttAuthority/ 
     36            SessionMgr/ 
     37        common/ 
     38        server/ 
     39            AttAuthority/ 
     40            SessionMgr/ 
     41            ca/ 
     42}}} 
     43 
     44== Special Namespace Packages == 
     45Python egg setuptools enables a special kind of package known as a ''namespace package''.  This can be thought of simply as a placeholder to support a required package structure.  It comes into play in the above since `ndg` and `security` don't actually need any modules ''but'' they are important in preserving the structure of the hierarchy and allowing distinct namespaces for NDG python software e.g. 
     46 
     47 * `ndg.security` - NDG Security namespace 
     48 * `ndg.utils` - NDG utilities namespace 
     49 * `ndg.dx` - namespace for NDG Data Extractor 
     50 
     51== Sub dividing Security into separate Eggs == 
     52setuptools modifies and extends the distutils setup.py concept.  A setup.py creates a .egg file.  (.eggs use zip file format and can be accessed using `unzip`).  This has consequences for the directory structure under the SVN `trunk/`.  There needs to be separate arrangements of the ndg/security directory structure for each of the eggs we want to define.  The structure below illustrates: 
     53 
     54{{{ 
     55#!python 
     56trunk/ 
     57    python/ 
     58        setup.py (contains settings to make egg for ''whole'' of NDG security) 
     59 
     60        ndgSecurityServer/ 
     61            setup.py (contains settings to make server egg) 
     62            ndg/ 
     63                __init__.py (with special namespace package declaration) 
     64                security/ 
     65                    __init__.py (with namespace package declaration) 
     66                    server/ 
     67                        __init__.py (no namespace package declaration) 
     68                        ... server sub packages and modules *.py ... 
     69 
     70        ndgSecurityClient/ 
     71            setup.py (contains settings to make client egg) 
     72            ndg/ 
     73                __init__.py (with namespace package declaration) 
     74                security/ 
     75                    __init__.py (with namespace package declaration) 
     76                    client/ 
     77                        __init__.py (no namespace package declaration) 
     78                        ... client sub packages and modules *.py ... 
     79 
     80        ndgSecurityCommon/ 
     81            setup.py (contains settings to make client egg) 
     82            ndg/ 
     83                __init__.py (with namespace package declaration) 
     84                security/ 
     85                    __init__.py (with namespace package declaration) 
     86                    common/ 
     87                        __init__.py (no namespace package declaration) 
     88                        ... common sub packages and modules *.py ... 
     89}}} 
     90 
     91 * `python/setup.py` makes the overall NDG security egg - the kitchen sink :) 
     92 * `python/ndgSecurityServer/setup.py` makes the NDG server egg.  It is has setting to make it dependent on the `common/` egg because this code is also needed.  When `setup.py` is run, it will get and install the `python/ndgSecurityServer\common` egg.   
     93 * `python/ndgSecurityCommon/setup.py` makes the `common` egg.  It can be thought of as a virtual egg as no one would need to install it on it's own.  It's only ever installed by virtue of it being a dependency for the client and server eggs. 
     94 * `python/ndgSecurityClient/setup.py` makes the client egg.  The same applies here as with the server egg: it is dependent on the `common` egg. 
     95 
     96This makes for an unwieldy directory structure but is consistent with the idea that each egg should be independently downloadable. 
     97 
     98== Making `setup.py` Settings == 
     99Keywords to the `setup` function specify the packages to be included in the egg, namespace packages, and other dependencies 
     100 
     101`client/setup.py`: 
     102 
     103{{{ 
     104#!python 
     105setup( 
     106  name = 'ndg_security_client', 
     107  ... 
     108  install_requires = ['ndg_security_common'], 
     109  namespace_packages = ['ndg', 'ndg.security'], 
     110  packages = ['ndg.security.client', ...], 
     111  ... 
     112) 
     113}}} 
     114 
     115`server/setup.py`: 
     116 
     117{{{ 
     118#!python 
     119setup( 
     120  name = 'ndg_security_server', 
     121  ... 
     122  Install_requires = ['ndg_security_common'], 
     123  namespace_packages = ['ndg', 'ndg.security'], 
     124  packages = ['ndg.security.server', ...], 
     125  ... 
     126) 
     127}}} 
     128 
     129`common/setup.py`: 
     130 
     131{{{ 
     132#!python 
     133setup( 
     134  name = 'ndg_security_common', 
     135  ... 
     136  install_requires = [], # plus any other dependencies 
     137  namespace_packages = ['ndg', 'ndg.security'], 
     138  packages = ['ndg.security.common', ...], 
     139  ... 
     140) 
     141}}} 
     142 
     143== When Dependencies Go Wrong == 
     144Third party python packages required by Python can be specified by giving their special name as used in the Python Cheese Shop.  However, this may not work and it may be necessary to give `setup.py` the explicit URL where a particular package can be downloaded.  Also, some packages may not install correctly or not follow the rules of `distutils`.  To be completed... mention `setup.cfg`. 
     145 
     146== Setup for Data files, Configuration files and Scripts == 
     147To be completed... 
     148 
     149 
     150 
     151 
     152 
     153 
     154 
     155 
     156 
     157 
     158 
     159 
     160 
     161 
     162