Archiving filesystems

This procedure is intended for use by system administrators to create archives of entire filesystems or fractions thereof. Individual users should be able to make use of it, but they will nor be able to use the DVAR wrapper. Steps 1 and 2 require the user to have read/execute access to all files in the requested tree. So in the general case, DVAR must be run by root.

DVAR works by breaking up the target directory tree into compressed tar archives (.tgz files) of several hundred MB each. DVAR then collects the .tgz files into DVD-sized ISO images, ready to burn onto DVDs.

Usage

Log into ds1 as root. Then...

    alias DVAR /srv/local/backup/BNL/CSBarc.01T/DVAR
    usage:
      DVAR -help -- list valid steps
      DVAR [-start N -jid JID] [-end M] root
  	[-start N] -- start with step# N. Must specify JID.
  	[-jid JID] -- Use JID as the job ID.
  	[-end M]   -- go thru step# M.
      
      #DVAR -help
      Step#  Description
        1    Create .spec files to partition root into .tgz archives.
        2    Use .spec to create .tgz archives (takes a long time)
        3    Create .2dvd files which group .tgz archives onto DVDs
        4    Create 0contents listing all files in all .tgz on all DVDs
        5    Using .2dvd, create ISO images from .tgz (takes a long time)
        6    Create PDF label files and XML rimage production files
        7    Print instructions for burning DVDs

At completion of step 6, DVAR will print instructions to initiate the actual burning of the DVDs. Log onto a gigabit core workstation as user (not root). Where $jid is a JobID which DVAR creates:

    alias burnit /srv/local/backup/BNL/2.1.0/burnDVD.py"
    burnit /srv/images/0/root/$jid-??.xml"

Examples

     ds1# alias DVAR /srv/local/backup/BNL/CSBarc.01T/DVAR
     ds1# DVAR /snap/people/day.0
     ds1# DVAR -end 4 /snap/ljr/0/day.0 
     ds1# DVAR -start 5 -jid ljr_0_20060817_1325 /snap/ljr/0/day.0

Notes and caveats

Design documents are in /srv/local/backup/BNL/CSBarc/0readme.*
DVAR defines temporary directories $wdir and $tdir as being subdirectories of /bck/bck/DVARtemp/. Only root can write there. /etc/cron.daily/tmpwatch on ds1 cleans files more than 48 hours old.
A listing of all files in every disk in the archive is in the file 0contents.JobID. A copy is on every disk of the archive, and in $wdir, which DVAR assigns to /bck/bck/DVARtemp/wdir/.
After running DVAR, it is still necessary actually to burn the DVDs. This can be done by root from ds1, or by any user from a java-enabled workstation. To monitor progress, burnit must be able to open a window on the initiating workstation.
DVAR can be run overnight. The actual burning procedure can be run the next day when it can be monitored. However, make sure running DVAR does not overlap with the nightly snapshot backup.
Neither the DVAR wrapper, nor the individual steps, are robust.
I believe that all files in the requested tree are archived (unless they are larger than a DVD). But it is possible that there will be errors in directory ownership or permission if the entire archive is restored. Also, files whose full pathname exceeds 255 characters may cause problems.
When archiving /snap filesystems, be sure to specify only day.0. Otherwise, multiple copies will be archived.

Last Modified: Thursday, 14-Sep-2006 14:55:48 EDT

CSB Home | Search | Table of Contents | General Information

Center for Structural Biology (www.csb.yale.edu) at Yale University (www.yale.edu)
Contact: webadmin(at)mail^csb^yale^edu