Version 4 (modified by spascoe, 13 years ago) (diff)


Python Eggs

StephenPascoe? recently gave a presentation on the Python Eggs and their use within NDG to the SSTD Software Engineering Seminar at RAL. You can  download the .ppt file. If you would like to reproduce the demo download  this tarball and follow the README.

Quick Overview

The following is a little out of date now. We suggest you look at the Eggs documentation links for full details.

Python eggs are designed to be the Python equivilent of Java Jar files -- a means of packaging Python libraries in a single file. Their use is closely related to PythonSetupTools?. The  egg documentation gives you a good summary of their features but it's a little difficult to separate out the propaganda. This page tries to point out the strengths and weaknesses of the technology.

Eggs are zip files with an ".egg" extension. They contain a normal Python package heirarchy (that is package dirs, files, etc.) plus some package metadata in the "EGG-INFO" directory. Python is able to treat any zip file on sys.path as a directory for importing, therefore a python egg can normally be installed just by adding the egg to PYTHONPATH. Also eggs are designed to work well with the PythonSetupTools? package manager that can install/uninstall/upgrade packages over the internet. However, there are complications:

  1. Packages containing C extensions are tied to a particular platform and version of Python. Such packages would have to be distributed as a choice of eggs.
  2. Packages may expect to find data files within it's directory using the __file__ variable and standard filesystem calls. This doesn't work inside a zip file.
  3. If a C extension uses a non-python dynamically linked library the operating system expects to be able to find that library on the file system.

The impact of no. 1. is reduced by using the easy_install script from PythonSetupTools? which will automatically download, build and install python eggs from the internet, however this assumes that a source distribution will build on a platform without manual configuration. Therefore, to take full advantage of eggs with C extensions your script needs to be very well designed.

Eggs tackle no. 2. using a completely new API for accessing files within packages using the pkg_resources module (part of PythonSetupTools?). This requires refactoring your code to avoid assuming you can read package files from the file system and use pkg_resources instead.

Alternatively you can unpack your egg into an egg directory (or tell setuptools to do it when installing). This is good for legacy code that needs the filesystem but means you can't just "drop your egg" into PYTHONPATH.

The PythonSetupTools? documentation describes a way of marking resources to be unpacked on demand (eager_resources) which is supposed to tackle problem 3. but I don't know the details.