wiki:CowsFramework/CowsInstallation/MigrationToPython2.6

Version 7 (modified by astephen, 9 years ago) (diff)

--

COWS Migration to Python 2.6 and Pylons 1.0

Introduction

In order to keep the COWS stack (cows + cowsserver + cowsclient) synchronised with our security solution we needed to update the stack to work with Python 2.6. The primary obstacle was updated the Pylons-related code due to a number of minor (but significant) changes introduced between Pylons 0.9.6 and 0.9.7. These changes are explained in the following Pylons documentation page:

 http://wiki.pylonshq.com/pages/viewpage.action?pageId=11174779

This page documents:

  • OUTSTANDING ISSUES WE NEED TO LOOK INTO REGARDING THE MIGRATION to Pylons 1.0
  • changes made in COWS, COWS Server and COWS Client to update to Pylons 1.0.

During this process various dependencies were also updated and documented. A separate page documents the Dependencies and Installation of the COWS stack?.

OUTSTANDING ISSUES WE NEED TO LOOK INTO REGARDING THE MIGRATION to Pylons 1.0

The cows, cowsserver and cowsclient code has been migrated to python 2.6. However, there are a number of controllers and items that still need further testing that may, or may not, have been broken in the pre-migrated version. There are also a few changes that have been made that require validation to ensure the behaviour is appropriate. This section outlines these issues. Once resolved they should be either documented or removed from the list below.

Untested controllers in pylons apps

demo controller in cowsserver broken

The /<dataset>/demo routes point to the demo controller. At present this seems to fail because ajax is not set up right.

Other controllers in cowsserver and cowsclient

There are a number of controllers in both cowsserver and cowsclient. These need to be reviewed so that we:

  • delete any test/unused controllers
  • test any required controllers and explain their usage context

cowsclient get_figure broken

On the cows client there the wmsviz.get_figure() action is broken for the following file formats:

  • SVG
  • EPS

These are called by the web app generated by the wmsviz controller when you click the "Make Figure" button in the right-hand panel.

Basemap problem with Transverse Mercator projection

One of the cowsserver dependencies is [browse:qesdi/geoplot]. Within geoplot, an instance is created of the mpl_toolkit.basemap.Basemap class with the "tmerc" (Transverse Mercator) projection. The actual code is (see:  http://proj.badc.rl.ac.uk/ndg/browser/qesdi/geoplot/trunk/lib/geoplot/grid_builder_national.py#L32):

32	nationalGrid = basemap.Basemap(projection='tmerc',
33	                               lon_0=trueOrigin[0],
34	                               lat_0=trueOrigin[1],
35	                               llcrnrlon=trueOrigin[0], llcrnrlat=trueOrigin[1],
36	                               urcrnrlon=10, urcrnrlat=10,
37	                               resolution=None)

This fails when moving to basemap 1.0 and matplotlib 1.0. When investigating the problem it seems to lie in the definition of the major and minor axes for the ellipsoid. This can be explicitly expressed using the rsphere argument when instantiating Basemap. geoplot (latest trunk - under qesdi) - investigate fix in rsphere usage!!!!

See  http://mapserver.org/errors.html#msprocessprojection-projection-library-error-major-axis-or-radius-0-not-given for some discussion of this issue. According to the matplotlib/basemap documents the rsphere argument can be:

  1. A float/int of the earth's radius (DEFAULT = 6370997m)
  2. A sequence of (major_axis, minor_axis) - I got segmentation faults when I tried making these similar, so I used option 3...
  3. The documentation says Note: sometimes an ellipsoid is specified by the major axis and an inverse flattening parameter (if). The minor axis (b) can be computed from the major axis (a) and the inverse flattening parameter using the formula if = a/(a-b).

It is therefore unclear numbers to use. I have set it to rsphere=(6370997, 1) for now because the following values of rsphere failed:

  • a single integer - complains about "major axis or radius 0 not given"
  • (6370997, 6370997) - segmentation fault
  • (6370997, <something similar to 6370997>) - segmentation fault

After making this change we get plots working but there may be errors in this assumption!

We may need to explore the definition of the transverse mercator projection at sites such as:

Possible alternative to metacarta coastlines

Metacarta web services (used by COWS to serve the coastline maps) have experienced problems recently. An alternative coastline dataset could be:


COWS modifications

The changes made to the cows library are documented in the  diff between the two versions.

The changes all related to the following change:

Replaced use of "routes.url_for" function with the global "pylons.url"
function for compatibility with pylons 1.0.

COWS Server modifications

The COWS server changes are documented in the  diff between the two versions.

These differences covered the following updates relating to Pylons changes:

1. config/middleware.py file changes

config/middleware.py file updated to mirror standard pylons 1.0 version.
Also removed the switch for compatibility with pylons versions before
version 0.9.7.

2. Setting of pylons global "c" variable config attributes (to retain previous behaviour)

Included additional config args introduced in pylons 0.9.7 to allow the
global "c" variable to maintain access to global properties and to not
throw AttributeError exceptions when an attribute is not found. See:

http://wiki.pylonshq.com/pages/viewpage.action?pageId=11174779

3. Modification of config/routing.py syntax and "map" attributes

Updated routes to pylons 1.0 requirement using new Routes syntax.
Also added the lines:

    # We need minimization switched on
    map.minimization = True
    map.explicit = False

To handle enable "implicit" routing, see:
http://routes.groovie.org/manual.html#implicit-defaults-and-route-memory 

4. Replacing "routes.url_for" with "pylons.url" calls

Replaced use of "routes.url_for" function with the global "pylons.url"
function for compatibility with pylons 1.0.

COWS Client modifications

The COWS client changes are documented in the  diff between the two versions.

These differences covered the following updates relating to Pylons changes:

1. config/middleware.py file changes

config/middleware.py file updated to mirror standard pylons 1.0 version.
Also removed the switch for compatibility with pylons versions before
version 0.9.7.

2. Setting of pylons global "c" variable config attributes (to retain previous behaviour)

Included additional config args introduced in pylons 0.9.7 to allow the
global "c" variable to maintain access to global properties and to not
throw AttributeError exceptions when an attribute is not found. See:

http://wiki.pylonshq.com/pages/viewpage.action?pageId=11174779

3. Modification of config/routing.py syntax and "map" attributes

Updated routes to pylons 1.0 requirement using new Routes syntax.
Also added the lines:

    # We need minimization switched on
    map.minimization = True
    map.explicit = False

To handle enable "implicit" routing, see:
http://routes.groovie.org/manual.html#implicit-defaults-and-route-memory 

4. Replacing "routes.url_for" with "pylons.url" calls

Replaced use of "routes.url_for" function with the global "pylons.url"
function for compatibility with pylons 1.0.

5. Escapting "<" character in Javascript within HTML template

Added CDATA section to escape the "<" character in javascript within an
HTML file. This stops Genshi escaping the "<" character into "&lt;" and
interpreting it as HTML instead of Javascript.