The best (and fastest) way to make a copy of experimental data taken during a visit – especially for large data volumes – is copying the data to a USB or eSATA disk whilst it is being acquired, using the designated Data Dispenser hardware available on each beamline.
The copy process ('Clone') should be set up before any data collection is started – please refer to the respective documentation on how to use a beamlines Data Dispenser.
Beamline data dispensers can handle both ext3 or NTFS formatted hard drives.
Should you have your own external hard drive you are free to use it but we may not be able to give support for that.
The Experimental Hall Coordinators (EHCs) have a small number of supported external hard drives for loan (both ext3 and NTFS).
From Diamond workstations
For smaller data volumes, it could also be feasible to copy it to a portable storage medium (DVD, USB flash drive, USB hard drive...) from a beamline workstation.
You will need the following information to find your files:
- FedID and password
- The name of your experiment and visit. (example mt767-1)
From Linux machines your data files will be found in
- /dls/BEAMLINE/data/YEAR/EXPERIMENT-VISIT
and from Windows machines
- \\BEAMLINE-storage\BEAMLINE\data\YEAR\EXPERIMENT-VISIT.
For example, data from the first visit of experiment mt767 in 2009 on B16 will be stored in /dls/b16/data/2009/mt767-1 under Linux or \\b16-storage\b16\data\2009\mt767-1 under Windows.
See also the 'Windows Transfer' and 'Linux Transfer' tabs above for more information.
All beamline workstations have CD/DVD writers. Blank media may be obtained from either the User Office or the Experimental Hall Coordinators (EHCs).
In addition the EHCs have a small number of external hard drives for loan, both Linux/ext3 and NTFS formatted. Should you have your own external hard drive you are free to use it but we may not be able to give support for that.
Note that typically USB drives are formatted as FAT32 by default. This limits the maximum number of files in a directory to fewer than 10-15000, thus it is recommended to reformat the drives as NTFS or ext3 before use; this removes the limitation. If FAT32 is used as the file system, copying might abort with 'disk full' or similar messages even though there is plenty of space left on the external drive.
Copying, especially of large amounts of data, can take some time (see also tab "Transfer Speeds"). Again, it is best done setting up a copy process on the designated Data Dispenser machines before a data collection is started. If copied manually from a workstation, it is probably best be done using rsync (or tar) under Linux (e.g. running rsync after every sample). If Windows is used the command line xcopy is likely to be significantly faster than using the file explorer.
Please note that our Linux workstations currently neither support FireWire nor write access to Windows NTFS formatted hard drives, however there are (free) ext3 drivers for Windows available.
Access from Visitor Network
Windows shares
For simple access to data for Diamond visitors and users, the beamline data directories are exported as readonly Windows shares to the Visitors network. The shares reside on data.diamond.ac.uk and may be accessed by both Windows and Linux equally. Other systems such as Macintosh should also be able to access the shares but we are currently not able to support these.
sftp
data.diamond.ac.uk can also be reached via sftp from the visitors network, i.e. data can also be transferred using sftp. An example Windows client would be WinSCP. IT helpdesk might be able to help you with the installation.
Under GNU/Linux you can simply use the sftp command, or use the SSH features of the Konqueror or Nautilus (sftp:// prefix)
For Mac OSX users the SSH Agent, Fugu or CyberDuck utilities may be useful.
Remote access
Diamond provides an externally accessible ssh/NX gateway server. The server is called 'nx.diamond.ac.uk'.
To transfer data to a remote site, scp, stfp or rsync (over ssh) can be used to connect to this server. Beamline data is then found in /dls/BEAMLINE/data/YEAR/EXPERIMENT-VISIT as described for Linux above.
NX is a remote desktop protocol with very good compression, allowing users to remote destop even over slow remote connections. This gives the user a desktop session on an internal Diamond machine (much like on a beamline workstation), allowing access to data and data analysis software much as if on site.connections. NX client software can be downloaded for Windows, Linux and Mac from http://www.nomachine.com/ for free.
Please note that ssh or NX to nx.diamond.ac.uk are the only available off-site connection methods.