Ticket #106 (closed task: fixed)

Opened 13 years ago

Last modified 11 years ago

[DS] Scope out integrating bbFTP client and server with python

Reported by: spascoe Owned by: spascoe
Priority: required Milestone: ALPHA
Component: ndg2 Version:
Keywords: Delivery Cc:

Description (last modified by spascoe) (diff)

There are several things that the DeliveryService client and server will be required to do that are not covered by bbftp:

  1. Client: Create an NDG UserSession? and obtain an attribute certificate. Send the attribute certificate to the server.
  2. Server: parse and validate an attribute certificate to authenticate the user.
  3. Server: Answer the question "Does the user have permission to get/put URI X?" This may require calling a GateKeeper? (see ticket #97).
  4. Client: Get/Put? all data described by CSML instance X. X can be resolved into a series of file URI's on the client side and each file retrieved/sent over bbFTP.

Item 1 will require the Client to be partially written in python. Sending the certificate is covered by bbFTPs authentication extension mechanism. Items 2-4 all require other parts of NDG (Security, MOLES and CSML respectively). Implementing these items will therefore require hooks between C-code (bbFTP) and Python (NDG libraries).

We have the choice of either embedding a python interpreter in the bbFTP executables or embedding bbFTP in a python module. We can make this choice independently for client and server.


This is quite straightforward. We embed the bbFTP client in a Python module. We can then build a high-level delivery client in Python which operates on the CSML level but talks to the bbFTPClient module in terms of files to download.


The problem here is that bbFTPd spawns processes with fork(). These processes do not share memory space, therefore if we were to use the embedding approach each process would have an independent python interpreter. With each interpreter importing Security and probably SOAP libraries (to contact the GateKeeper?) this may lead to very inefficient memory usage.

The traditional way of dealing with this is to communicate between processes using a pipe or socket. This lends itself to keeping the Python part and the C part of the server as separate executables communicating over a socket. A protocol to transmit information about 2 & 3 between each executable would need to be specified.


One problem is that NDG has so far assumed that authorisation will be done at the CSML level (as described in the MOLES security metadata). However, bbFTP expects to receive requests for files. The approach of building CSML level support into the client leaves the server to solve the problem of how to link a file to it's security metadata. The alternative would seem to be to build CSML awareness into the bbFTP server.

Change History

comment:1 Changed 13 years ago by spascoe

  • Status changed from new to assigned

comment:2 Changed 13 years ago by spascoe

One thing to bare in mind is the multitasking model used by bbftp. Bbftpd uses fork() rather than multithreading. This may limit our options for embedding/wrapping with python.

comment:3 Changed 13 years ago by spascoe

  • Description modified (diff)

Scoping of the basic design for the client and server has been put in the description.

I also raise the issue of authorisation at the file level.

comment:4 Changed 13 years ago by spascoe

  • Description modified (diff)

comment:5 Changed 13 years ago by spascoe

Plans for the design of the DeliveryService are evolving as UML at TI05-delivery/trunk/doc/DeliveryService.EAP. I'm hoping the EAP will be sufficiently well developed to close this ticket within a week.

comment:6 Changed 13 years ago by spascoe

  • Summary changed from Scope out integrating bbFTP client and server with python to [DS] Scope out integrating bbFTP client and server with python

Marked as [DS]

comment:7 Changed 13 years ago by spascoe

  • Status changed from assigned to closed
  • Resolution set to fixed

The Enterprise Architect files in svn could be improved but contain a reasonable outline of pybbftp and the design of the overall Delivery Service.

comment:8 Changed 11 years ago by lawrence

  • Keywords Delivery added
  • Component changed from T05_Delivery to ndg2

moved to component ndg2 (obsolete) as part of ndg2 cleanup

Note: See TracTickets for help on using tickets.