Argonne National Laboratory

GM/CA @ APS

User Program

Department of Energy Office of Science
GM/CA @ APS Sponsors:
National Institute of General Medical Sciences (NIGMS) and National Cancer Institute (NCI) of the National Institutes of Health (NIH)
 

User Program - While at the Beamline - Data Management and Backup

Supported media

GM/CA @ APS recommends and supports external USB3, USB2, ESATA, and firewire (IEEE 1394) disks as inexpensive and fast way to backup users data. The table below provides a speed comparison of different connector types for typical 5400rpm HDD external drives (the advantages of USB3 and eSATA for SSD drives are even higher as the speeds may reach 200-300 MB/s):

Connector Bus Speed Real Speed 200GB backup time Restricted by
USB3 500 MB/s 40-50 MB/s 1 hour disk spindle
ESATA 300 MB/s 40-50 MB/s 1 hour disk spindle
Firewire 50 MB/s 24 MB/s 2.5 hours bus
USB2 40 MB/s 20 MB/s 3 hours bus
USB1 1 MB/s 1 MB/s 56 hours bus

At present USB-3 is a recommended choice due to the highest speed, more streamlined mounting procedure compared to the eSATA and availability on the market.

Not all computers may have USB-3 and especially ESATA ports. Please ask your host if in trouble finding the right hub.

Some users have problems distinguishing between the USB-3 and USB-2 ports. Please look at the labeling and the color. The USB-3 ports typically have blue connectors and cables and/or the SS (Super-Speed) marking.

Support for DVD disks is not provided because at the rate of 200GB of data per experiment one would have to burn more than 45 disks.

Supported disk formats

The supported formats on external disks are FAT32, NTFS, exFAT, ext2/ext3/ext4, xfs, and HFS+. We have only Linux computers at the beamlines. Therefore some Windows-specific and MacOSX-specific extensions, for example encrypted drives like WD Passport that require some Windows or MacOSX utility to unlock the encrypted partition, may not work. When choosing format for your hard drive, keep in mind that FAT32 has the best record in terms of portability between different operating systems (Windows, Linux and MacOSX), while HFS+ has the shortest history of support under Linux. Please, try to avoid bringing disks with multiple partitions and disks with already stored data as in some problematic cases your disk may need to be re-partitioned or reformatted.

The HFS+ format is supported on the second-day workstations ws4, ws5, and ws7 only. You will have to temporarily disable HFS+ journaling in order to allow writing to HFS+ drives. Please consult this HFS+ manual for the details.

Firewire and USB disks can be connected to any beamline workstation except for ws2 and ws3 as they may be controling the expertiment . In the second-day area ESATA disks can be connected to ws4 and ws5 and USB-3 disks to most workstations with the SS port label (see marking examples above). Usually disks are mounted automatically upon connecting provided user has logged in before connecting the disk. If the disk is not auto-mounted, go to Applications -> System Tools > Disk Utility, then find your disk in the left-pane list, choose the right partition and click the "Mount Volume" button:

Additional details of mounting external disks under Linux operating system are described elsewhere.

Network transfers

GM/CA @ APS runs 10Gb network and users are welcome to SFTP or FTP out their data to the servers at their home institutions. External access the GM/CA SFTP and Globus GridFTP servers is also possible, but needs to be requested in advance. The access can be allocated for the time of experiment plus one day. That policy is enforced in order to avoid conflict of resources between different user groups. The real rate of remote data transfer will depend on the network connectivity to the users institution. Please check the Network Speed Testing page. Unless there is a bottleneck on user's side, one can expect the SFTP rates of 5-8MB/s and GridFTP rates of 20-25MB/s. For a 200GB data set it may take about 10 and 3 hours respectively. For further up to 5x improvements of Globus speeds, we offer Globus over ScienceDMZ. In order to fully utilize the advantages of ScienceDMZ, a respective node may need to be setup at your Institution. Please request information from your host.

Please be adviced to never using Wi-Fi connnections for network data transfers as it drastically reduces speeds, especially at long distances.

Additional Information

For the latest changes with the backup procedures and detailed technical instructions please refer the Backup Manual (PDF).

 

 

 


GM/CA @ APS is an Office of Science User Facility operated for the U.S. Department of Energy Office of Science by Argonne National Laboratory

UChicago Argonne LLC | Privacy & Security Notice | Contact Us | A-Z Index | Search