Changes between Version 4 and Version 5 of ThirdPartyDataTransfer

13/04/06 11:56:37 (14 years ago)



  • ThirdPartyDataTransfer

    v4 v5  
    11= Third Party Data Transfer = 
    3 The issue of whether we need 3rd-party data transfer, and how it could be implemented in the DeliveryService, has been raised in ticket #107. 
     3Information that was here has now moved to Ticket #107. 
    5 Assuming we need some sort of 3rd-party transfer I see 3 options: 
    7  1. Live without dedicated 3rd-party transfer support in the DeliveryService. 
    8     Server to server data transfer can be implemented on a per service basis using 
    9     the DeliveryService in the standard client-server manner. 
    11  2. Implement 3rd-party transfer as a high-level service which calls the low-level data 
    12     transfer (presumably bbFTP) underneath.  Such a service would act as the server to a 
    13     controling client and a client to the other party to the data transfer. 
    15  3. Implement 3rd-party transfer at the xFTP protocol level.  See below for details about this. 
    17 == Option 3. == 
    19 Since GridFTP is supposed to do 3rd party data transfer I decided to take a look at the 
    20 [ GridFTP specification] and was surprised to discover that the FTP protocol already supports 3rd party transfer in theory, it is just rarely implemented. 
    22 Here is the relevant extract from [ RFC 959]: 
    24 {{{ 
    25       When data is to be transferred between two servers, A and B (refer 
    26       to Figure 2), the user-PI, C, sets up control connections with 
    27       both server-PI's.  One of the servers, say A, is then sent a PASV 
    28       command telling him to "listen" on his data port rather than 
    29       initiate a connection when he receives a transfer service command. 
    30       When the user-PI receives an acknowledgment to the PASV command, 
    31       which includes the identity of the host and port being listened 
    32       on, the user-PI then sends A's port, a, to B in a PORT command; a 
    33       reply is returned.  The user-PI may then send the corresponding 
    34       service commands to A and B.  Server B initiates the connection 
    35       and the transfer proceeds.  The command-reply sequence is listed 
    36       below where the messages are vertically synchronous but 
    37       horizontally asynchronous: 
    39          User-PI - Server A                User-PI - Server B 
    40          ------------------                ------------------ 
    42          C->A : Connect                    C->B : Connect 
    43          C->A : PASV 
    44          A->C : 227 Entering Passive Mode. A1,A2,A3,A4,a1,a2 
    45                                            C->B : PORT A1,A2,A3,A4,a1,a2 
    46                                            B->C : 200 Okay 
    47          C->A : STOR                       C->B : RETR 
    48                     B->A : Connect to HOST-A, PORT-a 
    50 }}} 
    52 I.e. One server is put in passive mode and one in active mode.  A good explanation of the difference can be found [ here].  GridFTP states support for 3rd-party transfer in the specification.  In practice this means that both passive and active FTP commands are extended to support striping across multiple data ports. 
    54 On a casual read of the specifications it appears that, provided a server supports active mode, the main issue is with the client.  It must support making control connections to 2 servers simultaniously  and issueing the necessary PASV, PORT, STOR and RETR commands. 
    56 Unlike GridFTP, bbFTP is not an extension of RFC-959.  However, it claims to support both passive and "non-passive" mode.  Therefore, it may be possible to adapt the client to do 3rd party transfer along the lines shown above.  This may be a significant ammount of work and any consideration of it should be balanced against the work required to make GridFTP support NDG Security.