Last Modified: Monday, 16-Mar-2009 14:10:43 EDT

X-Ray Data Handling and Storage

Use of /srv/frames

/srv/frames is for short-term (21 days max) storage of raw frames and intermediate data. For medium- and long-term storage, see the options below, or discuss your needs with a CSB staff member.

Data collected in CSB Core

Data collected in the CSB Core is stored locally on each data collection computer. The data can be accessed from any of the core workstations. The data will be found on /data/mar1, /data/mar2, /data/raxis1, and /data/raxis2. From Core workstations, you can process the data, copy it to /srv/frames, or archive it to DVD.

Data handling after return from a synchrotoron

Make an archive copy

At this writing (4/2005), the best media for archiving your data is DVD. Each DVD disc holds about 250 18-MB data frames. For large datasets, multiple labelled DVDs can be written using the DVD robot. Of course, if you were able to burn your data on to DVDs at the synchrotron, you can skip this step.

If your data is on a firewire/USB disk, you can use one of the Core linux workstations to copy it directly to DVD. (See CD/DVD documentation for details).

If your firewire/USB disk is in a Windows format that cannot be read on a linux workstation, use a Windows workstation to copy the data to /srv/frames. From there, back it up to DVD.

Short-term storage for immediate processing

• You can process data directly from the firewire/usb disk you used to bring data back from the synchrotron.or...
• You can process data directly from the archive DVD containing your data. or...
• You can copy your data from firewire/usb disk or archive DVD to /srv/frames.
• We recommend that you not store your raw data on group disks. Doing so wastes the most expensive storage we have, and jeopardizes the backup integrity of your group disk.

Medium-term storage for reprocessing

If you need more than three weeks to process/reprocess your data, you cannot use /srv/frames. We suggest using one or a combination of the following techniques:

• Process raw data directly from DVD
• Store x-files and other reduced and intermediate data on group disks
• Use a firewire/USB disk to store data and intermediate files for a medium- or long-term project. Connect it to a linux workstation while you are using the data. Make sure to follow proper procedures to avoid losing data. Back up your non-archived data at least once a week.

Data processing/reprocessing from archived storage

Same as for medium-term storage above.

Using IEEE1395/Firewire and USB disks

New documentation is in preparation (4/2005). Old documentation is inadequate but not compeltely useless. If in doubt, consult one of the friendly staff.

Last Modified: Monday, 16-Mar-2009 14:10:43 EDT