Version 1 (modified by astephen, 10 years ago) (diff)


Managing our python distributions (and eggs)

Related pages


Managing source code, versioning and releases is quite a challenge. We are running projects in [ Subversion (SVN)] and we routinely produce distributions and python eggs for internal and external consumption.

This page outlines how we manage some of these things. It shows an example of the development release cycle using the example of updating the COWS stack to be compatible with python 2.6 (see [wiki:CowsFramework/CowsInstallation/CowsPython2.6 full details]).

Release cycle

In SVN, we typically run something like:


This is a standard SVN structure but the process is not automated for the developer. We need to understand how to manage our code within this structure.

A typical release cycle would involve the following steps:

  1. #CreateABranchFromTheTrunk
  2. #MakeChangesInTheBranch
  3. #TagTheTrunkBeforeMergingInTheModifiedBranch
  4. #MergeInTheModifiedBranch
  5. #ResolveConflicts
  6. #TagTheUpdatedTrunk
  7. #CreateASourceDistributionAndEggFromTheTaggedVersion

This cycle ensures that we have a version of the trunk tagged immediately before and after the modified branch has been merged in. This allows us to track changes very clearly in case there are problems.

Example: COWS library upgrade

We had a requirement to update the COWS library to be compatible with Python 2.6 and Pylons 1.0. These were a number of minor changes but they needed testing with other components and so it made sense to fix them in a branch, stabilise the changes, and then merge back in to the trunk.

This example shows the various stages involved...

Create A Branch From The Trunk

First check out the trunk and branches. Then create a new branch:

$ svn co
$ svn copy trunk branches/migrate-py26-pylons10
$ svn commit branches/migrate-py26-pylons10

Don't worry about duplication of volumes. SVN avoids actual duplication of code where it has not changed between different versions.

Make Changes In The Branch

You can then do any amount of editing within the branch. One day, you are happy with the branch and it is time to merge it into the trunk...

Tag The Trunk Before Merging In The Modified Branch

For safety, we tag the trunk before doing the merge:

$ svn copy trunk tags/1.5.1-pre-py2.6
$ svn commit tags/1.5.1-pre-py2.6

Merge In The Modified Branch

Now, the merge process is slightly confusing but makes sense when you have done it a few times. The following web page provides a useful overview of the process:

First, go in to the branch and find out when original copy happened from the trunk:

$ cd branches/migrate-py26-pylons10
$ svn log --stop-on-copy
r7579 | astephen | 2010-10-08 14:56:53 +0100 (Fri, 08 Oct 2010) | 2 lines

Changes for python 2.6 and pylons 1.0 version.

r7492 | astephen | 2010-09-21 14:24:50 +0100 (Tue, 21 Sep 2010) | 3 lines

Replaced use of "routes.url_for" function with the global "pylons.url"
function for compatibility with pylons 1.0.

r7491 | astephen | 2010-09-21 14:22:08 +0100 (Tue, 21 Sep 2010) | 2 lines

Added some whitespace padding.

r7342 | spascoe | 2010-08-18 16:55:16 +0100 (Wed, 18 Aug 2010) | 3 lines

New branch for migration to Python-2.6 and Pylons-1.0.


This shows copy version was 7342.

Now enter the trunk and update it, noting the version number:

$ cd ../../trunk
$ svn up
At revision 7584.

This shows current version is 7584.

Now, whilst in the turnk, run a merge between the two version numbers, with the branch as the main argument:

$ svn merge -r 7342:7584

The current trunk will now have been updated with the changes from the branch. However, any conflicts will still exist in individual files.

Resolve Conflicts

You need to resolve conflicts as you normallyl would in SVN. Either look in the files or accept the version you know is correct. In this case you are confident that the files from the branch are correct:

$ mv
$ mv cows/pylons/templates/catalogue.xml.merge-right.r7584 cows/pylons/templates/catalogue.xml
$ svn resolved cows/pylons/templates/catalogue.xml
$ svn commit

Tag The Updated Trunk

Now tag this current version:

$ svn copy trunk tags/1.6.0-py2.6
$ svn commit

Create A Source Distribution And Egg From The Tagged Version

The only thing left to do is to create a release of the source distribution and/or egg:

$ python sdist

This will create a version of the library tarred and gzipped under:


You can create a python egg using:

$ python bdist_egg

Which will create:


If desired the files under dist/ can be copied to the NDG repository at:

From there, tools such as  Easy Install will be able to find, download and build them.