PALMA3

Overview

palma3 is the login node to a newer part of the PALMA system. It has various queues/partitions for different purposes:

• u0dawin: A queue for general purpose. It is usable for everyone, even without being a member of the groups that have submittet a proposal for PALMA. It replaces the old ZIVHPC cluster
• k20gpu: Four nodes equipped with 3 K20 nVidia Tesla accelerators each
• normal: 44 nodes with 32 Broadwell CPU cores each. (Not fully installed yet)
• zivsmp: A SMP machine with 512 GB RAM. The old login node of ZIVHPC. (At the moment still part of ZIVHPC.)
• phi: 2 Nodes with 4 Intel Xeon Phi accelerators each. (A part of ZIVHPC at the moment).

The batch system

The batch system on PALMA3 is SLURM, but there is a wrapper for PBS installed, so most of your skripts should still be able to work. If you want to switch to SLURM, this document might help you: https://slurm.schedmd.com/rosetta.pdf

When using PBS skript, there are some differences to PALMA:

• The first line of the submit script has to be #!/bin/bash
• A queue is called partition in terms of SLURM. These terms will be used synonymous here.
• The variable $PBS_O_WORKDIR will not be set. Instead you will start in the directory in which the script resides.
• For using the "module add" command, you will have to source some scripts first: "source /etc/profile.d/modules.sh; source /etc/profile.d/modules_local.sh"

Submit a job

Show running jobs

• squeue
• qstat
• showq

Show information about the queues

scontrol show partition

Running interactive jobs with SLURM

Use for example the following command:

srun -p u0dawin -N 1 --ntasks-per-node=8 --pty bash

This starts a job in the u0dawin queue/partition on one node with eight cores.

-- Holger Angenent - 2016-08-22