Shared Storage

A job can run on any worker, so users need the same files on the controller and on every worker. OneSlurm can mount two NFS exports in every VM of the Cluster.

ParameterMount pointExample
ONEAPP_SLURM_NFS_HOME/home10.125.0.1:/srv/nfs/slurm/home
ONEAPP_SLURM_NFS_SCRATCH/scratch10.125.0.1:/srv/nfs/slurm/scratch

Both are optional. An empty value skips the mount.

Before You Start

The controller and worker images already include the NFS client.

  • Create the NFS server and the exports outside OneSlurm.
  • The controller and all the workers must reach the NFS server.
  • Do not attach another disk at /home or /scratch when you set the matching input.

How It Works

The net-12-mount-nfs script runs in every VM when the network is configured.

ItemValue
Format of the inputhost:/export
Filesystem typenfs4
Mount optionssec=sys,_netdev
PersistenceAdds a line to /etc/fstab after a successful mount, only once

If a mount point is already mounted, the script skips it.

Check the Mounts

On the controller.

findmnt /home
TARGET SOURCE                       FSTYPE OPTIONS
/home  192.168.100.222:/export/home nfs4   rw,relatime,vers=4.2,...,sec=sys,...

On a worker, through Slurm.

srun -N1 -n1 findmnt /home
srun -N1 -n1 findmnt /scratch

User Directories

  • /home holds the home directory of each user. Create it after the user exists in LDAP, owned by the UID and GID of the user. See Identity Management.
  • /scratch has no structure. Create a directory for each user, for example /scratch/alice, owned by the user.
  • Create these directories on the NFS server. With the default root_squash export option, root on the Cluster VMs cannot write to the exports.