Friday, January 4th 2013 11:25:01 PM PST
Several tools are available on Triton to assist users in moving their data between filesystems and other resources. For files over two gigabytes in size, a great option is bbFTP, which uses multi-threading to maximize throughput over the 1-gigabit network connection. Tests have achieved between 60 and 100 megabytes per second using this method.
Other methods include scp, sftp and bbcp. Links to the bbcp and bbftp documentation are in the Related Links section of this page, and scp and sftp are documented in the Linux man pages.
To use bbFTP, you need both a client and a server.
This server is installed in /opt/bbftp/bin.
Use the -s option to invoke a remote host for bbftpd. You will be prompted for a password on this host (see example below). Please note we have experienced problems using a Korn shell in this context. We recommend tcsh.
If you prefer, you can use a local client on another machine or install the bbFTP client on your desktop. Here are the basic steps needed to do that:
The default target location will be /user/local/bin so if you are unable to write that location or want to preserve an existing copy there, you should change the target for make install.
Additonal versions and server code are available for download from the bbFTP home page.
displays a brief help message
displays version info
bbftp [Options] [-u RemoteUsername] -i ControlFile [RemoteHost]
bbftp [Options] [-u RemoteUsername] -e ControlCommands [RemoteHost]
requests the execution of commands contained in the control file ControlFile or the ControlCommands command string, using RemoteUsername on RemoteHost as the service
See the bbFTP documentation for complete details.
Here is an example of a bbFTP session. In this example, the file HOMA.tar is being copied from GPFS-WAN to the ariel user's /mirage file space on Triton, using the remote bbFTP daemon from tg-login2.
[someuser@login-4-0 TEMP]% /opt/bbftp/bin/bbftp -s -e 'get /gpfs/projects/someuser/KRAKEN/HOMA.tar /mirage/ariel/TEMP/HOMA.tar' -u someuser -p 2 -V tg-login2.sdsc.edu
Password: <someuser's password on tg-login2>
>> COMMAND : get /gpfs/projects/someuser/KRAKEN/HOMA.tar /mirage/ariel/TEMP/HOMA.tar
4299427840 bytes got in 36.7 secs (1.14e+05 Kbytes/sec or 894 Mbits/s)