Issues for DEWS and NDG

Overall Package and Architecture

The current and planned architecture is outlined in the following UML component diagram. The key components include: securityServices – the package of software and services we expect to deployed at major data providers. In the case of DEWS we expect this package to be deployed in the DMZ at the Met Office, and in the case of NDG2, at all the official project partners. securityClients – the package of software that needs to be deployed to exploit the NDG security infrastructure. In the case of DEWS we expect this to be deployed at ESSC. NDG2 partners will also be deploying this software, and we might expect some other groups to deploy it as well. Key criteria for this package include ease of deployment and quality documentation (for application developers). Note that the login service is most likely deployed as part of the securityServices, but architecturally it need only exploit the smClient software, so it is included in this package for simplicity. clients – will interact with applications which deploy NDG security. In DEWS the two clients of interest are browser users and the BMT. In the NDG, the clients will include command line applications (a la BMT) and browser users.

source:TI12-security/trunk/NDGSecurity/architecture/uml/Component Model.JPG

Users interact with the services, initially via a loginManager:


The Security Services Package

The security services are currently the AttributeAuthority (AA) and the SessionManager (SM), and the latter exploits a MyProxy database. The two outward facing services (AA and SM) have WSDL interfaces. These packages are shown in the adjacent diagram. source:TI12-security/trunk/NDGSecurity/architecture/uml/securityServices.JPG

Four significant items of work on the SecurityServices are expected under the auspices of DEWS:

  1. The services will be upgraded to ZSI-2.0 (or later). (Required as underpinning work). For related problems, see SecurityIssues.
  2. The WSDL interfaces will be rewritten to take advantage of WS-Security and doc-literal encoding. Note that the former may involve some serious modifications to ZSI if we cannot exploit the  pyGridWare infrastructure (see below). (Required for Standardisation)
  3. The connection between the SessionManager and the MyProxy database will be rewritten so that the SessionManager need only deploy python calls, without any extra infrastructure, thus allowing a simpler installation where the SessionManager is not colocated on the same machine as the MyProxy (which is desirable). (Required for Deployability).
  4. The services will be rewritten to be thread safe, and deployed in an appropriate service environment infrastructure to ensure contending service calls are dealt with efficiently. (Required for Scalability)

source:TI12-security/trunk/NDGSecurity/architecture/uml/Deployment Model.JPG

Note that ideally the transition to WSDL+WS-Security and WS-SecureConversation should be done using third-party libraries to minimise coding within the project. The main (only?) candidate for this is to use  pyGridWare, however, we have significant reservations about the number of dependencies for this activity and we are also worried about the performance and future of the project. However, it is likely that some of the performance problems will be mitigated by the planned under the hood changes to ZSI XML parsing. We are also concerned that the resulting code should interact gracefully with the java client needed for application developers (discussed below). In practice this may mean the code has to be reverse engineered to work with the IBM-websphere integration of WSDL and WS-Security.

Security Clients

The Security Clients package is the software required for application developers to secure applications. It is intended that the applications may or may not themselves use any particular web service infrastructure. There are at least three classes of applications that we anticipate:

  • Standalone applications, most likely scripts deployed in a CGI environment that access data using the security infrastructure. Initially we only support python applications, but other programming environments will be supported via a python command line utility (to allow data providers to instantiate their own login services that access their own bespoke architectures
  • Java applications deployed in a container.

The most of important of these from a DEWS perspective is the WebSphere container. But we should also consider the tomcat container as a backup strategy and for international collaboration.


Refactoring Security for DEWS

The smClient Interfaces

The NDG-alpha release architecture works, but needs to be enhanced for DEWS to allow easy integration with the W(C,F)S being built as part of DEWS.

In particular, from an application developer's point of view, applications should be unaware as to the “state” of the security infrastructure. This idea is encoded in the following figure:


Further, we anticipate that the applications we are deploying will be in practice legacy applications, so we want if possible to completely encapsulate the security infrastructure from the applications. This concept is depicted in the inheritance relationship:


Security Software Issues

  1. We currently make heavy use of ServiceProxy, but we think ServiceProxy is not in a useable state in general though. ( e.g.). Not an issue for ZSI 2.0_rc2 and later - ServiceProxy class is not needed.
  2.  pyGridWare Issues
    1. Client Issues. Even the java version of the Globus clients seems to be difficult to configure in a light weight manner: See this  thread.
    2.  Future of WSRF issues.
  3. WS-Security and WSDL. It's not trivial to get this working. We will need to understand WS-policy. Possibly too much for the moment: the standards are available in draft but there is still some catching up to do for the development tools to support it. It will be possible to use WS-Security but more difficult to tie the security policy used into WSDL.
  4. ZSI Performance.
    1. See the  thread in ZSI mailing list entitled “ZSI performance” kicked off by Joshua on the 31st March, 2006.
    2. “Moreover - if I understand your figures right, the ZSI client is quite efficient whereas the server is relatively slow.” ( link) “So it would seem serialization is taking much longer than parsing.”, Joshua
    3. Any big payload should probably go as an attachment, can ZSI do that?
    4. Lots of talk about using celementtree, but no one has done it? Joshua and others have done some work with this but the current system uses pyXML + 4Suite. 4Suite gives some improvement in performance. The main issues with cElementTree is that it doesn't support XPath. Searches on elements are possible but not on attributes. The latter is important for WS-Security as these are used for locating elements for signature/encryption. lxml is an alternative. It's built on libxml2 and so supports XPath but implements a cElementTree style interface. A big task for cElementTree + ZSI + WS-Security would be a cElementTree Canonicalization algorithm - not for the faint hearted :).


  1. Need a Java Interface to the smClient library
  2. Migrate internal web service communication to WS-Security. See T12_Security/WS-Security
    1. but note that WSDL and WS-Security imply the necessity for  WS-PolicyAttachment. ( for example)
    2. Could we use pyGridWare?
      1. Huge list of dependencies? Yes, but noted pyGridWare has an Egg installation interface which should make the process less painful.
      2. What about client dependencies?
      3. But maybe we'd get ibm-websphere integration for free?
  3. Need to package things better for simpler installation
  4. Need to build better and cleaner interfaces for application integration.