research > lab > docs > scratch disks

CSCS Lab Scratch Disks

CSCS has a number of "scratch" disks which can be used for storing larger amounts of data. These scratch spaces are mirrored daily to protect the data in case there is a disk failure.

For example, if you are using drone to run experiments which generate large amounts of data, you can have drone place the output (usually the expdata/ directory) on one of the scratch areas. Keep the original program, control files, log/ directory, and so on, on a disk that is being backed up. Then if something should happen to the scratch disk, you can still regenerate the experiment.

You can find out what scratch disks are available and how much is available on each by running the script /common/scripts/show-scratch.pl as shown here:

ghiandi-jmbuck$ show-scratch.pl
 
The current CSCS scratch areas -- please try to use
scratch areas with the most available space, and then
if possible, run your experiments on that machine
(so the data can be written directly to the scratch disk
instead of being sent over the network).
 
Mounted on           1k-blocks      Used Available Use%
/scratch/prateria0   115341864  35410672  74072072  33%
/scratch/ghiandi0     76893224  11226956  61760324  16%
/scratch/posare0      76893224  11498048  61489232  16%
/scratch/lupo0        76915744  26591220  46417352  37%
/scratch/lupo1        38442480     47140  36442572   1%
/scratch/lastrada0    76893224  38776408  34210872  54%
/scratch/pulcina0     76893224  41952188  31035092  58%
/scratch/pesce0       38464340  12063352  24447084  34%
/scratch/battistini0  76893224  52774816  20212464  73%
 
See the CSCS Lab documentation pages for more information:
   /lab/scratch-disks.html

The mountpoint of the scratch areas are on left. The number to the right of the mountpoint is the number of kilobytes on the disk, the next number is the number of kilobytes used, the next number is the number of kilobytes of free space, and the next number is the percentage of disk that is used.

Choose a scratch area that has the most space available. You can make a directory for yourself with a command like

mkdir /scratch/pulcina0/yourLogin

where you replace "yourLogin" with your login. Then you can write and read data, create subdirectories, and so on, in that directory, but no one else can erase it or write in that area.

If you want to see how much scratch space is being used by a particular user, you can run the script /commons/scripts/show-scratch.pl with the -u option. This generates an output like this:

ghiandi-jmbuck$ show-scratch.pl -u jmbuck
 
Listing all scratch dirs owned by user: jmbuck
 
Size ( MB )         Last modification   Path
1966                Jul 19 13:48        /scratch/prateria0/jmbuck
 
See the CSCS Lab documentation pages for more information:
   /lab/scratch-disks.html

The number of megabytes contained in the directories are listed on the left. To the right of that is the date that the directory was last modified. To the right of that is the path to the directory.

If you would like to list the scratch space along with the actual location of the mounted drive (machine name along with the mountpoint on that machine) then you can run show-scratch.pl with the -v option (verbose mode). This generates an output like this:

ghiandi-jmbuck$ show-scratch.pl -v
 
The current CSCS scratch areas -- please try to use
scratch areas with the most available space, and then
if possible, run your experiments on that machine
(so the data can be written directly to the scratch disk
instead of being sent over the network).
 
Filesystem           1k-blocks      Used Available Use% Mounted on
alveare:/1a/data/scratch2
                      76893224  41952188  31035092  58% /scratch/pulcina0
battistini:/1a/data/scratch
                      76893224  52774816  20212464  73% /scratch/battistini0
ghiandi:/1a/data/scratch
                      76893224  11226956  61760324  16% /scratch/ghiandi0
lastrada:/1a/data/scratch
                      76893224  38776408  34210872  54% /scratch/lastrada0
lupo:/1a/data/scratch
                      76915744  26591220  46417352  37% /scratch/lupo0
lupo:/2a/data/scratch
                      38442480     47140  36442572   1% /scratch/lupo1
pesce:/1a/data/scratch
                      38464340  12063352  24447084  34% /scratch/pesce0
posare:/1a/data/scratch
                      76893224  11498048  61489232  16% /scratch/posare0
prateria:/1a/data/scratch
                     115341864  35410672  74072072  33% /scratch/prateria0
 
See the CSCS Lab documentation pages for more information:
   /lab/scratch-disks.html

The machine name and the mountpoint on the machine is listed on the left. The rest of the output is the same as the output without the -v option.

Since even our scratch areas are not infinite, please do remove files you do not need as soon as possible. If you need long term storage, for possible future use, talk to us about the use of DLT tapes, Jaz or Zip disks, or burning CDs to storing larger amounts of data that is seldom used.

To retrieve data from the scratch mirrors, simply go to /scratch/scratchdiskname-mirror where scratchdiskname is the name of the scratch disk whose mirror you want to access. For example, to access /scratch/pesce0's mirror just change directory to /scratch/pesce0-mirror

For more information about using the scratch areas for storing simulation output (e.g., from drone experiments) see our page on how to run drone experiments

Updated September 1, 2005