HPCC Data Transfer
For data transfers to and from the HPCC clusters, please refrain from using scp, sftp, or direct ssh connections and instead use the HPCC Globus Connect data transfer service. Many large research labs and computing centers at other universities also have their own Globus Connect server endpoints that you can use directly within the service.
There are several reasons to prefer this service:
- The Globus Connect service is well connected to the campus network and therefore the outside world with a high-speed connection
- The machine that runs our Globus Connect endpoint is newer, more robust, and has more processors than the cluster login node
- The Globus Connect service eliminates the load of data transfer from the cluster login node, which is used by many people for other functions
- There are Globus Connect endpoint clients available for Linux, Mac or Windows.
It's very easy to set things up for user-driven data transfer using the Globus Connect service. Here are the instructions to get started.
Step 1: Go to https://www.globus.org/app/endpoints/create-gcp, select "Texas Tech University" from the drop down list then click on the "Continue" button, next on the webpage follow the instruction to download and install Globus Connect Personal on your local machine.
Step 2 (optional): When you finish installing Globus Connect Personal, you can change the default settings, for example, you can change the local accessible folder to an ideal location for doing the data transfer in the future, for example, it could be "D:\download" on a Windows machine, or a locally selected folder or directory on a MacOS or Linux one.
Step 3: Start Globus Connect Personal on your local machine, then you can start the data transfer. If you want to transfer data from/to your Lustre storage area on Hrothgar, you currently need to enter "rdms#TTUIVY" as the endpoint for accessing your /home, /lustre/work or /lustre/scratch folder. We plan to add other transfer endpoints in the future and will publish a list.
Once you have done this, in the future you can simply connect again to the service on the web to use it again at http://globus.org.
More information about Globus Connect Personal can be found at https://www.globus.org/globus-connect-personal
If, after reviewing the above information, you would like additional assistance from HPCC staff in moving large or special-case data collections, please send an email with the following information to email@example.com
*Location of files/directories:
Comments: (optionally include any additional details)
(* = Mandatory fields)