Changes between Version 37 and Version 38 of CdatLite


Ignore:
Timestamp:
25/05/12 10:26:34 (7 years ago)
Author:
spascoe
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • CdatLite

    v37 v38  
    1 = NEWS = 
    2  
    3  '''2011-06-24''':: 
    4   [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0rc2.tar.gz cdat_lite-6.0rc2.tar.gz] has superseded 6.0rc1 because it left out a recent upgrade to cdunifpp. 
    5  
    6   [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0rc1.tar.gz cdat_lite-6.0rc1.tar.gz] is available.  This release supports {{{nc-config}}} as a mechanism 
    7   for detecting your NetCDF installation.  It has code merged from the latest CDAT master branch.  The version scheme is diverging from CDAT since  
    8   users were confused by the "alpha" tag.  Although CDAT is still alpha the cdms2 package is generally stable. 
    9  '''2011-03-18''':: 
    10   [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0.alpha-4.tar.gz cdat_lite-6.0.alpha-4.tar.gz] is available.  This release fixes a problem 
    11   when recompiling the package for different NetCDF libraries. 
    12  '''2011-03-11''':: 
    13   Source code has moved to Git.  See the [http://proj.badc.rl.ac.uk/gitweb?p=cdat_lite.git gitweb view]. 
    14  '''2010-11-05''':: 
    15   [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0.alpha-1.tar.gz cdat_lite-6.0.alpha-1.tar.gz] is available.  This release 
    16   merges in the latest CDAT-6 master branch which includes some important bug fixes for x86_64 platforms.  It also includes 
    17   the CF checker module for checking NetCDF-CF compliance.  Se below for details. 
    18  
    19  '''2010-03-16''':: 
    20   [http://ndg.nerc.ac.uk/dist/cdat_lite-5.2-1.tar.gz cdat_lite-5.2-1.tar.gz] is available.  This release includes minor fixes to enable cdat_lite to build with [http://pypi.python.org/pypi/pip pip]. 
    21  
    22  '''2010-01-12''':: 
    23   There is a [http://mail.scipy.org/pipermail/numpy-discussion/2010-January/047867.html significant bug in numpy-1.4.0] affecting the !MaskedArray implementation.  Therefore cdat_lite shouldn't be used with numpy-1.4.0. 
    24  
    25  '''2009-09-23''':: 
    26   cdat-lite-5.2 has been released.  Select you favourite method of installation: 
    27     1. Use {{{easy_install cdat-lite}}} 
    28     1. [http://ndg.nerc.ac.uk/dist/cdat_lite-5.2.tar.gz Download Manually] from the NDG Egg repository 
    29     1. Visit the [http://pypi.python.org/pypi/cdat-lite cdat-lite page] on the Python Package Index 
    30  
    31  '''2009-09-15''':: 
    32   cdat-lite-5.2rc1 is out and [http://pypi.python.org/pypi/cdat-lite available on pypi] or the [http://ndg.nerc.ac.uk/dist NDG Egg repository] 
    33  
    34  
    35 {{{ 
    36 #!rst 
    37  
    38 .. contents:: 
    39  
    40 .. sectnum:: 
    41  
    42 Cdat-lite is a Python package for managing and analysing climate 
    43 science data.  It is a subset of the Climate Data Analysis Tools 
    44 (CDAT_) developed by PCMDI_ at Lawrence Livermore National Laboratory. 
    45  
    46 Cdat-lite aims to compliment CDAT by focussing on it's core data 
    47 management and analysis components and by offering a radically 
    48 different installation system to CDAT.  As a result it is much more 
    49 lightweight (hence the name): CDAT's source distribution is the order 
    50 of 1Gb whereas cdat-lite is under 5Mb. 
    51  
    52 Cdat-lite is designed to work with the `CF checker`_ package. 
    53  
    54 .. _CDAT: http://www2-pcmdi.llnl.gov/cdat 
    55 .. _PCMDI: http://www2-pcmdi.llnl.gov/ 
    56 .. _`CF checker`: http://pypi.python.org/pypi/cfchecker 
    57  
    58  
    59 cdat-lite versioning 
    60 ==================== 
    61  
    62 Cdat-lite is a project that tracks versions of 2 other projects (CDAT 
    63 and cdunifpp).  From version 6.0rc1 the cdat-lite version will not be 
    64 based directly on the CDAT version.  This is because CDAT updates it's 
    65 version very seldomly and stays as an "alpha" distribution for long 
    66 periods when the parts included in cdat-lite are generally stable. 
    67  
    68 Full details of which versions of CDAT and cdunifpp a cdat-lite 
    69 distribution includes is available in the setup.py file and the 
    70 PKG_INFO metadata. 
    71 Installing cdat-lite 
    72 ==================== 
    73  
    74 cdat-lite is distributed as a tarball available from the `cdat-lite 
    75 homepage`_ on the `NERC Data Grid wiki` .  It is also installable 
    76 using the ``easy_install`` tool.  If you are familiar with 
    77 ``easy_install`` try this super-quick installation recipe:: 
    78  
    79   $ export NETCDF_HOME=/usr/local/netcdf 
    80   # Required if using a NetCDF4 compiled with HDF5 
    81   $ export HDF5_HOME=/usr/local/hdf5 
    82   $ easy_install cdat_lite  
    83  
    84  
    85 Dependencies 
    86 ------------ 
    87  
    88 To install cdat-lite you will need: 
    89  
    90  1. `Python 2.5.x`_.  cdat-lite has not been tested on 2.6 but may 
    91  work (feedback would be gratefully received).  It is unlikely to work on 3.0. 
    92  
    93  2. `setuptools`_.  cdat-lite will attempt to download and install 
    94  setuptools if it is missing but it is safer to install it first. 
    95  
    96  3. `NetCDF-3.x`_ or greater.  cdat-lite should work with any 
    97  relatively modern NetCDF3 installation on your system provided it is 
    98  compiled as a shared library.  It will also work with NetCDF4 
    99  configured in various different ways, including embedded OPeNDAP 
    100  mode. 
    101  
    102  4. If you want to run the short test suite you will need nose_ 
    103  
    104 .. _`Python 2.5.x`: http://www.python.org/download/releases/2.5.4 
    105 .. _`setuptools`: http://pypi.python/org/setuptools 
    106 .. _`NetCDF-3.x`: http://www.unidata.ucar.edu/software/netcdf/ 
    107 .. _nose: http://somethingaboutorange.com/mrl/projects/nose/ 
    108  
    109 Selecting your NetCDF installation 
    110 ---------------------------------- 
    111  
    112 cdat-lite will work with NetCDF3 or NetCDF4 but because it is 
    113 referenced by shared libraries (the python C extension modules) it 
    114 must be compiled as position independent code.  If you have a NetCDF4 
    115 installation you almost certainly are using shared libraries and even 
    116 if you wish to use NetCDF3 it is probably easiest to install NetCDF as 
    117 a shared library (use ``--enable-shared`` in the NetCDF ``configure`` 
    118 script).  Alternatively, you can configure NetCDF with:: 
    119  
    120   $ ./configure --with-pic ... 
    121  
    122 If you are using NetCDF4 you will also need to configure HDF5 with ``--enable-shared`` or ``--with-pic``. 
    123  
    124 If you have the command ``nc-config`` in your path cdat-lite will 
    125 detect all library and include dependencies.  Otherwise cdat-lite will 
    126 look for a NetCDF installation in several places. 
    127  
    128 If your NetCDF is installed somewhere unusual, or if you want to 
    129 select a specific installation, set the NETCDF_HOME variable.  E.g.:: 
    130  
    131   # sh users 
    132   $ export NETCDF_HOME=/usr/local/netcdf 
    133   # csh users 
    134   $ setenv NETCDF_HOME /usr/local/netcdf 
    135  
    136 If you are using NetCDF4 cdat-lite will also look for your HDF5 
    137 installation which you can configure in a similar way:: 
    138  
    139   # sh users 
    140   $ export HDF5_HOME=/usr/local/hdf5 
    141   # csh users 
    142   $ setenv HDF5_HOME /usr/local/hdf5 
    143  
    144 For compatibility with the ``netcdf4-python`` package cdat-lite also accepts ``NETCDF4_DIR`` AND ``HDF5_DIR`` as synonims for these environment variables. 
    145  
    146 Note, you don't need these environment variables set to run cdat_lite, 
    147 although the libraries must be findable by your system's dynamic 
    148 linker.  This can be configured by setting ``LD_LIBRARY_PATH`` or using ``ldconfig``. 
    149  
    150 Running the installer 
    151 --------------------- 
    152  
    153 If you have all the dependencies in place you can try using 
    154 ``easy_install`` to automatically download and install cdat_lite.  Make sure you have access to the internet, with the appropriate HTTP proxy settings, and do:: 
    155  
    156   $ easy_install cdat-lite 
    157  
    158 Alternatively you might want to see what you are installing :-).  In 
    159 this case either download the tarball__ or use ``easy_install`` to do it for you:: 
    160  
    161   $ easy_install -eb . cdat-lite 
    162   # The cdat-lite tarball will be downloaded unpacked into you current directory 
    163  
    164 Now from the distribution directory run the build and install steps separately:: 
    165  
    166   $ python setup.py bdist_egg 
    167   $ easy_install dist/cdat-lite*.egg 
    168  
    169 __ `cdat-lite homepage`_ 
    170  
    171  
    172 .. _`installing locally`: 
    173  
    174 Installing as an unprivileged user 
    175 ---------------------------------- 
    176  
    177 If you don't have write access to your python distribution you can use 
    178 the tool virtualenv_ to create a local python environment with 
    179 it's own ``easy_install`` executable which you can then use to install 
    180 cdat-lite.  In combination with ``NETCDF_HOME``, ``HDF5_HOME`` and 
    181 ``LD_LIBRARY_PATH`` it should be possible to install all dependencies 
    182 of cdat-lite locally.  See the virtualenv_ for details on 
    183 installation or try this recipe after downloading the virtualenv:: 
    184  
    185   # From virtualenv distribution directory 
    186   $ ./virtualenv.py <virtualenv-path> 
    187   $ cd <virtualenv-path> 
    188   $ source bin/activate 
    189   (venv)$ easy_install cdat-lite 
    190  
    191 .. _virtualenv: http://pypi.python.org/pypi/virtualenv 
    192  
    193 Platform-specific installation notes 
    194 ------------------------------------ 
    195  
    196 OS X 
    197 '''' 
    198  
    199 Christopher Lee contributed the following experiences installing on OS X 10.6.7. 
    200  
    201 My particular Macbook has an Intel CPU, and the default on the Mac is 
    202 to compile for the architecture x86_64. In order to override this 
    203 (because python is 32 bit, and the netcdf libraries I use are also 32 
    204 bit) I needed to pass in "-arch i386" to the compiler. I also needed 
    205 the little endian flag '-DBYTESWAP' when compiling the netcdf 
    206 interface (inside libcdms). The -DBYTESWAP flag should be included by 
    207 the libcdms configure script, where there is a section for 'darwin' 
    208 (OS X), but it's currently configured without BYTESWAP (line 
    209 6182). The problem here is that OS X used to run on PowerPC CPUs, 
    210 which don't need the BYTESWAP flag. I'm not sure if this is your 
    211 configure script or if it's from the cdat package. 
    212  
    213 I included the -arch i386 and -DBYTESWAP in the setup.py in the 
    214 libcdms section, and the setup works fine. 
    215  
    216 After running python setup.py build ; python setup.py install ; I 
    217 still get an error when importing cdms2. This problem is caused by the 
    218 way that libcdms is linked to the netcdf libraries. The 'normal' Mac 
    219 method is to link with absolute paths, but libcdms is linked with 
    220 relative paths (the libraries are references with @rpath). The result 
    221 is that LD_LIBRARY_PATH environment variable is often empty. I'm not 
    222 sure how to fix this in the 'Mac' way with absolute paths, but I added 
    223 my $NETCDF_HOME/lib directory to the variable and cdms2 now imports 
    224 without error. 
    225  
    226  
    227 Testing the installation 
    228 ======================== 
    229  
    230 cdat-lite ships with a small set of tests designed to verify that it 
    231 has been built successfuly.  These tests require the testing framework 
    232 nose_.  Once cdat-lite is installed just run:: 
    233  
    234   $ nosetests cdat_lite 
    235  
    236 When run from cdat-lite's distribution directory nosetests will run 
    237 slightly differently, running some tests that are known to fail at the 
    238 moment.  To disable this behaviour do: 
    239  
    240   $ nosetests --config='' 
    241  
    242 .. _`cdat-lite homepage`: http://proj.badc.rl.ac.uk/ndg/wiki/CdatLite 
    243 .. _`NERC Data Grid wiki`: http://proj.badc.rl.ac.uk/ndg/wiki 
    244  
    245  
    246  
    247 FAQ 
    248 === 
    249  
    250 What is CDAT? 
    251 ------------- 
    252  
    253 CDAT_ is a large suite of open source tools distributed by PCMDI_ for 
    254 the management and analysis of climate data.  It includes several 
    255 visualisation components and the graphical user interface VCDAT. 
    256  
    257 What is the difference between CDAT and cdat-lite? 
    258 -------------------------------------------------- 
    259  
    260 Differences between CDAT and cdat-lite can be classified as 
    261 differences in scope, i.e. which packages are included, and installation system. 
    262  
    263 cdat-lite contains the 'cdms2' package and a few related 
    264 packages.  It does not include the 'vcs' visualisation package or the 
    265 VCDAT graphical user interface.  As of v5.1.1-0.3pre3 the included 
    266 packages are: 
    267  
    268  * cdms2 
    269  
    270  * cdtime 
    271  
    272  * cdutil 
    273  
    274  * genutil 
    275  
    276  * ncml 
    277  
    278  * Properties 
    279  
    280  * regrid2 
    281  
    282  * unidataa 
    283  
    284  * xmgrace 
    285  
    286 CDAT bundles virtually all dependencies together in it's source 
    287 distribution -- even Python itself.  This has it's advantages as it 
    288 simplifies satisfying dependencies and avoids version conflicts 
    289 between dependencies.  However, if you want to integrate CDAT's data 
    290 management components into your existing Python architecture CDAT can 
    291 be overkill. 
    292  
    293  
    294 What has changed between cdat-lite-4.x and cdat-lite-5.x? 
    295 --------------------------------------------------------- 
    296  
    297 If you are a cdat-lite-4 user (or a CDAT 4 user) you have a big 
    298 migration job on your hands.  CDAT-4 uses the ``Numeric`` package for 
    299 arrays which has been out of date and unmaintained for a long time 
    300 now.  It is known to have problems on 64bit architectures. 
    301  
    302 How does cdat-lite track changes to CDAT? 
    303 ----------------------------------------- 
    304  
    305 cdat-lite tries to release major new versions shortly after new 
    306 versions of CDAT.  Sometimes CDAT-trunk contains important fixes that 
    307 should be applied so that the latest cdat_lite can run ahead of 
    308 official CDAT releases (although sometimes CDAT recommends you build 
    309 from trunk anyway). 
    310  
    311 The one exception is the UK Met. Office PP file support which is 
    312 usually updated in cdat_lite before CDAT.  In all cases the exact 
    313 build versions of CDAT and cdunifpp will be stated in the 
    314 distribution's ``setup.py`` file. 
    315  
    316 How can I use CMOR2 with cdat-lite? 
    317 ----------------------------------- 
    318  
    319 We are interested to hear any with experience of using CMOR2 with 
    320 cdat-lite but it should be as simple as downloading the distribution 
    321 and installing it in parallel with:: 
    322  
    323   # From the CMOR install directory 
    324   $ python setup.py install 
    325  
    326 How can I use OPeNDAP with cdat-lite? 
    327 ------------------------------------- 
    328  
    329 OPeNDAP support is an experimental feature of cdat-lite at the moment. 
    330 Unlike CDAT you don't select OPeNDAP explicitly during installation 
    331 but cdat-lite will inherit any OPeNDAP support embedded into the 
    332 NetCDF4 library.  Recent beta releases of NetCDF4 provides a switch to 
    333 transparently use OPeNDAP. 
    334  
    335 How do I install cdat-lite as an unprivileged user? 
    336 --------------------------------------------------- 
    337  
    338 See `installing locally`_ 
    339  
    340 Which versions of NetCDF does cdat-lite support? 
    341 ------------------------------------------------ 
    342  
    343 TODO 
    344  
    345  
    346  
    347 }}} 
     1= Page Moved = 
     2Cdat Lite is now hosted on the [http://proj.badc.rl.ac.uk/cedaservices:CEDA Services wiki] at http://proj.badc.rl.ac.uk/cedaservices/wiki/CdatLite.