Context Navigation

← Previous Change
Wiki History
Next Change →

Changes between Version 37 and Version 38 of CdatLite

Timestamp:: 25/05/12 10:26:34 (7 years ago)
Author:: spascoe
Comment:: --

Legend:

: Unmodified
: Added
: Removed
: Modified

CdatLite

-                      v37
+                      v38
+= NEWS =
+ '''2011-06-24'''::
+  [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0rc2.tar.gz cdat_lite-6.0rc2.tar.gz] has superseded 6.0rc1 because it left out a recent upgrade to cdunifpp.
+  [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0rc1.tar.gz cdat_lite-6.0rc1.tar.gz] is available.  This release supports {{{nc-config}}} as a mechanism
+  for detecting your NetCDF installation.  It has code merged from the latest CDAT master branch.  The version scheme is diverging from CDAT since
+  users were confused by the "alpha" tag.  Although CDAT is still alpha the cdms2 package is generally stable.
+ '''2011-03-18'''::
+  [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0.alpha-4.tar.gz cdat_lite-6.0.alpha-4.tar.gz] is available.  This release fixes a problem
+  when recompiling the package for different NetCDF libraries.
+ '''2011-03-11'''::
+  Source code has moved to Git.  See the [http://proj.badc.rl.ac.uk/gitweb?p=cdat_lite.git gitweb view].
+ '''2010-11-05'''::
+  [http://ndg.nerc.ac.uk/dist/cdat_lite-6.0.alpha-1.tar.gz cdat_lite-6.0.alpha-1.tar.gz] is available.  This release
+  merges in the latest CDAT-6 master branch which includes some important bug fixes for x86_64 platforms.  It also includes
+  the CF checker module for checking NetCDF-CF compliance.  Se below for details.
+ '''2010-03-16'''::
+  [http://ndg.nerc.ac.uk/dist/cdat_lite-5.2-1.tar.gz cdat_lite-5.2-1.tar.gz] is available.  This release includes minor fixes to enable cdat_lite to build with [http://pypi.python.org/pypi/pip pip].
+ '''2010-01-12'''::
+  There is a [http://mail.scipy.org/pipermail/numpy-discussion/2010-January/047867.html significant bug in numpy-1.4.0] affecting the !MaskedArray implementation.  Therefore cdat_lite shouldn't be used with numpy-1.4.0.
+ '''2009-09-23'''::
+  cdat-lite-5.2 has been released.  Select you favourite method of installation:
+. Use {{{easy_install cdat-lite}}}
+. [http://ndg.nerc.ac.uk/dist/cdat_lite-5.2.tar.gz Download Manually] from the NDG Egg repository
+. Visit the [http://pypi.python.org/pypi/cdat-lite cdat-lite page] on the Python Package Index
+ '''2009-09-15'''::
+  cdat-lite-5.2rc1 is out and [http://pypi.python.org/pypi/cdat-lite available on pypi] or the [http://ndg.nerc.ac.uk/dist NDG Egg repository]
+{{{
+#!rst
+.. contents::
+.. sectnum::
+Cdat-lite is a Python package for managing and analysing climate
+science data.  It is a subset of the Climate Data Analysis Tools
+(CDAT_) developed by PCMDI_ at Lawrence Livermore National Laboratory.
+Cdat-lite aims to compliment CDAT by focussing on it's core data
+management and analysis components and by offering a radically
+different installation system to CDAT.  As a result it is much more
+lightweight (hence the name): CDAT's source distribution is the order
+of 1Gb whereas cdat-lite is under 5Mb.
+Cdat-lite is designed to work with the `CF checker`_ package.
+.. _CDAT: http://www2-pcmdi.llnl.gov/cdat
+.. _PCMDI: http://www2-pcmdi.llnl.gov/
+.. _`CF checker`: http://pypi.python.org/pypi/cfchecker
+cdat-lite versioning
+====================
+Cdat-lite is a project that tracks versions of 2 other projects (CDAT
+and cdunifpp).  From version 6.0rc1 the cdat-lite version will not be
+based directly on the CDAT version.  This is because CDAT updates it's
+version very seldomly and stays as an "alpha" distribution for long
+periods when the parts included in cdat-lite are generally stable.
+Full details of which versions of CDAT and cdunifpp a cdat-lite
+distribution includes is available in the setup.py file and the
+PKG_INFO metadata.
+Installing cdat-lite
+====================
+cdat-lite is distributed as a tarball available from the `cdat-lite
+homepage`_ on the `NERC Data Grid wiki` .  It is also installable
+using the ``easy_install`` tool.  If you are familiar with
+``easy_install`` try this super-quick installation recipe::
+  $ export NETCDF_HOME=/usr/local/netcdf
+  # Required if using a NetCDF4 compiled with HDF5
+  $ export HDF5_HOME=/usr/local/hdf5
+  $ easy_install cdat_lite
+Dependencies
+------------
+To install cdat-lite you will need:
+. `Python 2.5.x`_.  cdat-lite has not been tested on 2.6 but may
+ work (feedback would be gratefully received).  It is unlikely to work on 3.0.
+. `setuptools`_.  cdat-lite will attempt to download and install
+ setuptools if it is missing but it is safer to install it first.
+. `NetCDF-3.x`_ or greater.  cdat-lite should work with any
+ relatively modern NetCDF3 installation on your system provided it is
+ compiled as a shared library.  It will also work with NetCDF4
+ configured in various different ways, including embedded OPeNDAP
+ mode.
+. If you want to run the short test suite you will need nose_
+.. _`Python 2.5.x`: http://www.python.org/download/releases/2.5.4
+.. _`setuptools`: http://pypi.python/org/setuptools
+.. _`NetCDF-3.x`: http://www.unidata.ucar.edu/software/netcdf/
+.. _nose: http://somethingaboutorange.com/mrl/projects/nose/
+Selecting your NetCDF installation
+----------------------------------
+cdat-lite will work with NetCDF3 or NetCDF4 but because it is
+referenced by shared libraries (the python C extension modules) it
+must be compiled as position independent code.  If you have a NetCDF4
+installation you almost certainly are using shared libraries and even
+if you wish to use NetCDF3 it is probably easiest to install NetCDF as
+a shared library (use ``--enable-shared`` in the NetCDF ``configure``
+script).  Alternatively, you can configure NetCDF with::
+  $ ./configure --with-pic ...
+If you are using NetCDF4 you will also need to configure HDF5 with ``--enable-shared`` or ``--with-pic``.
+If you have the command ``nc-config`` in your path cdat-lite will
+detect all library and include dependencies.  Otherwise cdat-lite will
+look for a NetCDF installation in several places.
+If your NetCDF is installed somewhere unusual, or if you want to
+select a specific installation, set the NETCDF_HOME variable.  E.g.::
+  # sh users
+  $ export NETCDF_HOME=/usr/local/netcdf
+  # csh users
+  $ setenv NETCDF_HOME /usr/local/netcdf
+If you are using NetCDF4 cdat-lite will also look for your HDF5
+installation which you can configure in a similar way::
+  # sh users
+  $ export HDF5_HOME=/usr/local/hdf5
+  # csh users
+  $ setenv HDF5_HOME /usr/local/hdf5
+For compatibility with the ``netcdf4-python`` package cdat-lite also accepts ``NETCDF4_DIR`` AND ``HDF5_DIR`` as synonims for these environment variables.
+Note, you don't need these environment variables set to run cdat_lite,
+although the libraries must be findable by your system's dynamic
+linker.  This can be configured by setting ``LD_LIBRARY_PATH`` or using ``ldconfig``.
+Running the installer
+---------------------
+If you have all the dependencies in place you can try using
+``easy_install`` to automatically download and install cdat_lite.  Make sure you have access to the internet, with the appropriate HTTP proxy settings, and do::
+  $ easy_install cdat-lite
+Alternatively you might want to see what you are installing :-).  In
+this case either download the tarball__ or use ``easy_install`` to do it for you::
+  $ easy_install -eb . cdat-lite
+  # The cdat-lite tarball will be downloaded unpacked into you current directory
+Now from the distribution directory run the build and install steps separately::
+  $ python setup.py bdist_egg
+  $ easy_install dist/cdat-lite*.egg
+__ `cdat-lite homepage`_
+.. _`installing locally`:
+Installing as an unprivileged user
+----------------------------------
+If you don't have write access to your python distribution you can use
+the tool virtualenv_ to create a local python environment with
+it's own ``easy_install`` executable which you can then use to install
+cdat-lite.  In combination with ``NETCDF_HOME``, ``HDF5_HOME`` and
+``LD_LIBRARY_PATH`` it should be possible to install all dependencies
+of cdat-lite locally.  See the virtualenv_ for details on
+installation or try this recipe after downloading the virtualenv::
+  # From virtualenv distribution directory
+  $ ./virtualenv.py <virtualenv-path>
+  $ cd <virtualenv-path>
+  $ source bin/activate
+  (venv)$ easy_install cdat-lite
+.. _virtualenv: http://pypi.python.org/pypi/virtualenv
+Platform-specific installation notes
+------------------------------------
+OS X
+''''
+Christopher Lee contributed the following experiences installing on OS X 10.6.7.
+My particular Macbook has an Intel CPU, and the default on the Mac is
+to compile for the architecture x86_64. In order to override this
+(because python is 32 bit, and the netcdf libraries I use are also 32
+bit) I needed to pass in "-arch i386" to the compiler. I also needed
+the little endian flag '-DBYTESWAP' when compiling the netcdf
+interface (inside libcdms). The -DBYTESWAP flag should be included by
+the libcdms configure script, where there is a section for 'darwin'
+(OS X), but it's currently configured without BYTESWAP (line
+). The problem here is that OS X used to run on PowerPC CPUs,
+which don't need the BYTESWAP flag. I'm not sure if this is your
+configure script or if it's from the cdat package.
+I included the -arch i386 and -DBYTESWAP in the setup.py in the
+libcdms section, and the setup works fine.
+After running python setup.py build ; python setup.py install ; I
+still get an error when importing cdms2. This problem is caused by the
+way that libcdms is linked to the netcdf libraries. The 'normal' Mac
+method is to link with absolute paths, but libcdms is linked with
+relative paths (the libraries are references with @rpath). The result
+is that LD_LIBRARY_PATH environment variable is often empty. I'm not
+sure how to fix this in the 'Mac' way with absolute paths, but I added
+my $NETCDF_HOME/lib directory to the variable and cdms2 now imports
+without error.
+Testing the installation
+========================
+cdat-lite ships with a small set of tests designed to verify that it
+has been built successfuly.  These tests require the testing framework
+nose_.  Once cdat-lite is installed just run::
+  $ nosetests cdat_lite
+When run from cdat-lite's distribution directory nosetests will run
+slightly differently, running some tests that are known to fail at the
+moment.  To disable this behaviour do:
+  $ nosetests --config=''
+.. _`cdat-lite homepage`: http://proj.badc.rl.ac.uk/ndg/wiki/CdatLite
+.. _`NERC Data Grid wiki`: http://proj.badc.rl.ac.uk/ndg/wiki
+FAQ
+===
+What is CDAT?
+-------------
+CDAT_ is a large suite of open source tools distributed by PCMDI_ for
+the management and analysis of climate data.  It includes several
+visualisation components and the graphical user interface VCDAT.
+What is the difference between CDAT and cdat-lite?
+--------------------------------------------------
+Differences between CDAT and cdat-lite can be classified as
+differences in scope, i.e. which packages are included, and installation system.
+cdat-lite contains the 'cdms2' package and a few related
+packages.  It does not include the 'vcs' visualisation package or the
+VCDAT graphical user interface.  As of v5.1.1-0.3pre3 the included
+packages are:
+ * cdms2
+ * cdtime
+ * cdutil
+ * genutil
+ * ncml
+ * Properties
+ * regrid2
+ * unidataa
+ * xmgrace
+CDAT bundles virtually all dependencies together in it's source
+distribution -- even Python itself.  This has it's advantages as it
+simplifies satisfying dependencies and avoids version conflicts
+between dependencies.  However, if you want to integrate CDAT's data
+management components into your existing Python architecture CDAT can
+be overkill.
+What has changed between cdat-lite-4.x and cdat-lite-5.x?
+---------------------------------------------------------
+If you are a cdat-lite-4 user (or a CDAT 4 user) you have a big
+migration job on your hands.  CDAT-4 uses the ``Numeric`` package for
+arrays which has been out of date and unmaintained for a long time
+now.  It is known to have problems on 64bit architectures.
+How does cdat-lite track changes to CDAT?
+-----------------------------------------
+cdat-lite tries to release major new versions shortly after new
+versions of CDAT.  Sometimes CDAT-trunk contains important fixes that
+should be applied so that the latest cdat_lite can run ahead of
+official CDAT releases (although sometimes CDAT recommends you build
+from trunk anyway).
+The one exception is the UK Met. Office PP file support which is
+usually updated in cdat_lite before CDAT.  In all cases the exact
+build versions of CDAT and cdunifpp will be stated in the
+distribution's ``setup.py`` file.
+How can I use CMOR2 with cdat-lite?
+-----------------------------------
+We are interested to hear any with experience of using CMOR2 with
+cdat-lite but it should be as simple as downloading the distribution
+and installing it in parallel with::
+  # From the CMOR install directory
+  $ python setup.py install
+How can I use OPeNDAP with cdat-lite?
+-------------------------------------
+OPeNDAP support is an experimental feature of cdat-lite at the moment.
+Unlike CDAT you don't select OPeNDAP explicitly during installation
+but cdat-lite will inherit any OPeNDAP support embedded into the
+NetCDF4 library.  Recent beta releases of NetCDF4 provides a switch to
+transparently use OPeNDAP.
+How do I install cdat-lite as an unprivileged user?
+---------------------------------------------------
+See `installing locally`_
+Which versions of NetCDF does cdat-lite support?
+------------------------------------------------
+TODO
+}}}
+= Page Moved =
+Cdat Lite is now hosted on the [http://proj.badc.rl.ac.uk/cedaservices:CEDA Services wiki] at http://proj.badc.rl.ac.uk/cedaservices/wiki/CdatLite.