wiki:T05_Delivery

PID Outline

Product number
5
Product name
Delivery Service
Product description
Reliable file transfer service.
Product purpose
Given a number of feature instances identified by URIs to data held in cache, move the data to a user defined location in a reliable manner, restarting transfers when necessary, and providing stateful management of delivery progress.
Product Components
  1. Multi-threaded ftp component which is NDG-security aware
  2. Optional checksumming and restarting capability

Introduction

The NDG Delivery service will provide the data transfer functionality for NDG. It is being built upon bbFTP as the file transfer layer and is aware of NDG Security. It is required to:

  1. Move data files from one NDG actor to another.
  2. Allow users to download all files constituting an CSML feature.
  3. Allow data to be transfered from one NDG actor to another, initiated by a third party.
  4. Monitor the progress of an ongoing transfer.

Components

  1. bbFTP private authorisation extensions. BbFTP provides an API for implementing custom authentication mechanisms which it calls "private authentication" but it relies on the UNIX permissions system for authorising individual bbFTP commands. The private authentication API is extended to allow custom, per command, authorisation.
  2. PyBbftp binds the bbFTP client and server to the Python programming language. This passes control of server initiation, client session control, authentication and authorisation to Python code, allowing more sophisticated file transfer applications to built on bbFTP.
  3. DeliveryService-bbFTP. Provides bbFTP functionality with NDG Security.
  4. DeliveryService-HTTP. Provides higher level functions on CSML instances including transfer monitoring and 3rd party control.

Wiki pages that are deprecated: