Version 16 (modified by domlowe, 13 years ago) (diff) |
---|
NEW! CDAT_LITE 5.0
cdat_lite-5.0 is now available. Below are some notes on what the new version brings or go to the 4.3 documentation
cdat_lite-5.0
Version 5.0 of cdat_lite introduces many changes.
- It is built directly from the CDAT-5.0 codebase and version numbers are completely synchronised. CDAT-5.0 requires Python-2.5 and uses numpy rather than Numeric. See Upgrading for what you need to do to your code.
- It is now only distributed as a tarball. Binary distributions tended to run into library version conflicts on different versions of linux. However easy_install should still be able to compile the package automatically. See Installing. Also you can make your own binary eggs for given OSes.
- You now need to have the netcdf library pre-installed. Automatically building netcdf wasn't portable enough.
Installing
First satisfy a couple of dependencies.
Make sure you have the netcdf libraries installed. If they are in a non-standard location make a note of it (you will be prompted) or set NETCDF_DIR to its installation prefix.
Install numpy. If you aren't bothered about linear algebra optimisation this usually works with easy_install. E.g.
$ easy_install numpy
Now you are ready to install cdat_lite. On many linux systems the C compiler is incorrectly detected therefore it is safest to do:
$ CC=gcc easy_install cdat_lite
Advanced Installation
You might want more control over the installation process but you can still get easy_install to download the tarball for you:
$ easy_install -eb <dir> cdat_lite
This will download and unpack the tarball into <dir> then you can run the setup.py script directly or build a binary egg for installation elsewhere.
Upgrading
The move from Python-2.4 to Python-2.5 and from Numeric to numpy requires very few changes to your code. Most changes that are needed can be done automatically using the convertcdms script.
$ convertcdms Usage: convertcdms.py [options] path Convert scripts that use the Numeric, MA, cdms, and regrid modules to numpy, ma, cdms2, and regrid2. -or- convertcdms.py [options] -r direc Convert all Numeric/MA/cdms Python scripts and C source code in directory 'direc' to numpy/ma/cdms2. Use this form to convert C source code. Arguments: path: A .py file to convert direc: A directory: convert all .py, .c, and .h files Options: -a: Disable 'aggressive' conversion (see Note 1) -c: Clobber the original file(s). The default is to move the original to <name>.orig -h: Print a help message -n: Do not write '# Adapted by...' comment at the top of file. -r: Recursively convert all .py and .c/.h files in directory direc -s direc: Skip the directory. This option may be used more than once. For example, '-s .svn' skips all subversion subdirectories. Notes: 1) The -a option turns off the following translations: - MA.Float => Numeric.Float, similarly for MA.Int, MA.NewAxis - MA.Numeric => Numeric - The 'typecode=' argument in MA functions is changed to 'dtype=' - XX.mask() => XX.mask - XX.mask is None => ((XX.mask is None) or (XX.mask is MV2.nomask)) - A keyword argument 'axis=0' is added to MA.sum, MA.average, MA.product, and MA.repeat 2) By default the following translations are made: - import cdms => import cdms2 as cdms - import regrid => import regrid2 as regrid - import MV => import MV2 as MV - import Numeric => import numpy.oldnumeric as Numeric - import MA => import numpy.oldnumeric.ma as MA - import cdms.MV => import cdms2.MV2 - from cdms import XX => from cdms2 import XX (similarly for regrid, MV, MA, and Numeric) - from cdms.XX import YY => from cdms2.XX import YY (similarly for regrid, MV, MA, and Numeric) - import cdms as XX => import cdms2 as XX (similarly for regrid, MV, MA, and Numeric) - import cdms.XX => import cdms2.XX (similarly for regrid, MV, MA, and Numeric) - import XX, cdms, YY => import XX, cdms2 as cdms, YY (similarly for regrid, MV, MA, and Numeric) - the translations in Note (1) - the translations in numpy.oldnumeric.alter_code1. This module is used for most Numeric and MA-related translations.
A few minor changes may also be required. If you use arrays in logical expressions you may need to change your code. The following would work with Numeric:
>>> import numpy >>> x = numpy.arange(4) >>> if x: ... print 'hello' ... Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() >>>
Installation problems (and solutions)
On a 64-bit machine you may get a message like:
`a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/libnetcdf.a: could not read symbols: Bad value
In which case you need to recompile your NetCDF installation to use the fPIC option as follows:
./configure --with-pic make make install
CDAT-lite 4.3
cdat-lite is a simple repackaging of the I/O layer of the Climate Data Analysis Tools (CDAT) as a Python Egg?. A short article on the reasons behind cdat-lite and some of the design decisions made can be found here on Stephen Pascoe's blog.
Features
- Includes cdscan from the comand-line or from within python using the cdat_lite.scripts.cdscan module.
- Includes the latest version of the British Atmospheric Data Centre's cdunifpp module which provides read access to UK Met. Office PP files.
- Downloads and installs Numeric on demand. If you have Numeric installed already but the package doesn't have an egg-info directory easy-install will re-install Numeric for you unless you use the --no-deps switch.
- Currently focused on providing NetCDF and PP format support. DRS, HDF and DODS (OpeNDAP) are disabled in the default egg build.
Package Overview
- CDAT core packages:
- cdms, cdtime, cdutil, genutil, Properties, regrid, unidata, xmgrace
These top-level packages can be used in the same way you would with a full CDAT installation. E.g.
>>> import cdms >>> f = cdms.open('dataset.nc')
- cdms, cdtime, cdutil, genutil, Properties, regrid, unidata, xmgrace
These top-level packages can be used in the same way you would with a full CDAT installation. E.g.
- cdat_lite support packages: These packages support the cdat_lite
egg installation.
- cdat_lite.clib: libraries and headers used to build CDAT source: libcdms and libnetcdf.
- cdat_lite.scripts: contains a version of cdscan suitable for declaring as a setuptools entry-point. During installation easy-install will create a stub script to execute cdscan on the command line
- cdat_lite.test: contains a test suite.
Installation
Cdat-lite uses easy_install as its installation mechanism. This makes downloading and installing the software a one-step process, once you've installed the setuptools package (itself very easy to install).
Binary eggs and the source tarball of cdat_lite are available from http://ndg.nerc.ac.uk/dist. It is also registered at the Python Cheeseshop so should be discovered automatically by easy_install.
- Start with a python 2.4+ installation with distutils. You cannot build or run the egg without distutils. If your python has been installed by a unix package manager then you may have to install the python-devel package. To check you've got distutils type the following at the Python prompt:
>>> import distutils >>>
- Install setuptools. This includes the easy_install script. The setuptools website has detailed installation instructions but if you have write-permission to your python installation the following should work:
$ wget http://peak.telecommunity.com/dist/ez_setup.py $ python ez_setup.py
- Now you're ready to install cdat_lite. Simply type:
$ easy_install cdat_lite
- Alternatively, if you want to install into a local directory $EGG_DIR, ensure $EGG_DIR exists and is on your $PYTHONPATH then type:
$ easy_install -d $EGG_DIR cdat_lite
- For more installation options see the easy_install documentation
Versioning
cdat-lite versioning is complicated as it is a repackage of CDAT with an updated cdunifpp module. There are therefore 3 version numbers to consider: CDAT, cdunifpp and cdat-lite. Once we start using versions of CDAT from the PCMDI SVN the version string could become very long indeed therefore the following strategy is used:
- Eggs are quoted with the version: <cdat-release>-<cdat-lite-version>
- If a PCMDI SVN version of CDAT is used it is stated in long_description not in <cdat-release>.
- The cdunifpp version is stated in long_description not in the version. Any change to the cdunifpp version naturally triggers a new <cdat-lite-version>.
Using CDMS scripts
cdat_lite includes the libcdms script cdscan within the egg. You have several options to get at it.
The stub script
When you install the egg with easy_install a script stub is created automatically which calls cdscan within the egg. Therefore you should be able to use cdscan as usual. If you've installed the egg in a local directory the script stub will be in that directory.
Import cdscan as a module from within python
>>> import cdat_scripts.cdscan as cdscan >>> cdscan.main(['cdscan', '-x', 'dataset.xml', filename, ...])
The argument to main() must be the equivilent of sys.argv when run from the command line, i.e. the first element should be a pseudo command name.
Import a cdscan entry point using the pkg_resources API.
This is the way scripts are supposed to be found from within eggs and is used by easy_install to automatically create script stubs.
>>> import pkg_resources >>> cdscan = pkg_resources.load_entry_point('cdat_lite', 'console_scripts', 'cdscan') >>> cdscan()
In this case sys.argv must be set to the arguments you want.