Apptainer¶

What is Apptainer?¶

Apptainer (formerly Singularity) is a container platform designed for HPC environments. It allows users to package their software, dependencies, and environment into a portable .sif image file that runs consistently across different systems — without requiring root privileges at runtime.

Why use Apptainer on PERUN?

Reproducibility — pin exact versions of CUDA, Python, and libraries
Portability — build once, run anywhere with a compatible driver
No module conflicts — your environment lives entirely inside the container
GPU support — native NVIDIA GPU passthrough via --nv flag

Core Concepts¶

Term	Description
`.sif`	Singularity Image Format — the container file you run
`pull`	Download and convert a Docker image to `.sif`
`exec`	Run a command inside the container
`build`	Create a new `.sif` from a definition file
`--nv`	Pass NVIDIA GPU drivers from host into container
`--no-home`	Exclude home directory; current working directory is still accessible
`--containall`	Full isolation — nothing from host is bound unless explicitly specified

Checking the GPU Driver¶

Before pulling an image, check what CUDA version the GPU nodes support. The container's CUDA version must be less than or equal to the host driver version.

Check CUDA driver on a GPU node

srun --partition=GPU --gres=gpu:1 nvidia-smi

The top-right corner of the output shows the maximum supported CUDA version.

Pulling an Image¶

Images are pulled from Docker Hub and converted to .sif format. This is a one-time operation.

Pull an image

apptainer pull my_image.sif docker://repository/image:tag

Finding images

Browse all available containers at: https://hub.docker.com/search

Throughout this documentation we will use PyTorch as an example, however the same approach applies to any image and any type of workload.

Running a Container¶

Basic GPU execution¶

Run a script inside a container with GPU access

apptainer exec --nv my_image.sif python3 my_script.py

Bind flags explained¶

Flag	Binds `$HOME`	Binds `$PWD`	Use case
(none)	✅	✅	Full access, development
`--no-home`	❌	✅	Clean runtime, still sees working dir
`--containall`	❌	❌	Full isolation, bind everything manually

PWD on SLURM

$PWD inside a SLURM job is the directory from which sbatch was submitted. Your script and data files should be in that directory, or use #SBATCH --chdir=/path/to/dir to set it explicitly.

Using --containall with manual bind

When using --containall, nothing from the host is visible inside the container. Use --bind to explicitly mount directories:

apptainer exec --nv --containall \
    --bind /path/on/host:/path/in/container \
    my_image.sif python3 my_script.py

Multiple directories can be bound by chaining --bind flags:

--bind /data:/data --bind $PWD:/mnt/work

Customizing an Image¶

If the base image does not include everything you need, you can extend it by writing a definition file and building a new .sif on top of it.

Definition file¶

A definition file specifies the base image and any additional setup steps. The %post section is a standard shell script — you can use pip, apt-get, or any other shell commands here.

my_custom.def

Bootstrap: localimage
From: my_image.sif

%post
    pip install <package>
    apt-get update
    apt-get install -y <package>

Building on PERUN¶

Build with fakeroot (no root required)

apptainer build --fakeroot my_custom.sif my_custom.def

Fakeroot

--fakeroot allows building containers on HPC without real root privileges, including running apt-get inside %post.

Building locally

You can also build images on your own machine with sudo and then transfer the .sif to PERUN:

sudo apptainer build my_custom.sif my_custom.def

Environment Variables¶

Some tools write cache or config files to $HOME, which is unavailable when using --no-home. Use --env to redirect these to a writable location.

Redirect a cache directory

apptainer exec --nv --no-home \
    --env SOME_CACHE_DIR=/tmp/cache \
    my_image.sif python3 my_script.py

Full Example: PyTorch + Matplotlib on GPU¶

This end-to-end example demonstrates everything covered above: pulling an image, extending it with additional packages, writing a test script, and running it as a SLURM job. Everything here can be run directly on PERUN — no local Apptainer installation required.

1. Pull the base image¶

apptainer pull my_image.sif \
    docker://pytorch/pytorch:2.6.0-cuda12.4-cudnn9-runtime

2. Extend the image with matplotlib¶

The base PyTorch image does not include matplotlib, so we build a custom image on top of it.

my_custom.def

Bootstrap: localimage
From: my_image.sif

%post
    pip install matplotlib
    apt-get update
    apt-get install -y curl

apptainer build --fakeroot my_custom.sif my_custom.def

3. Write the test script¶

test_gpu.py

import torch
import matplotlib
matplotlib.use('Agg')  # non-interactive backend, required on HPC (no display)
import matplotlib.pyplot as plt

print(f"PyTorch version: {torch.__version__}")
print(f"CUDA version: {torch.version.cuda}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU name: {torch.cuda.get_device_name(0)}")

x = torch.rand(3, 3).cuda()
print(f"Tensor on GPU: {x}")

fig, ax = plt.subplots()
ax.plot([1, 2, 3], [4, 5, 6])
ax.set_title("Matplotlib test")
plt.savefig("matplotlib_test.png")
print("Matplotlib test passed, saved matplotlib_test.png")

4. Write the job script¶

apptainer_gpu.sh

#!/bin/bash
#SBATCH --job-name=apptainer_gpu
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --gres=gpu:1
#SBATCH --partition=GPU
#SBATCH --output=apptainer_%j.out

apptainer exec --no-home my_custom.sif curl --version

apptainer exec --nv --no-home \
    --env MPLCONFIGDIR=/tmp/matplotlib_cache \
    my_custom.sif python3 test_gpu.py

sbatch apptainer_gpu.sh

5. Expected output¶

Example output

The following output is specific to this example. Your actual output will vary depending on your script and hardware, but the key indicators — CUDA available: True and a visible GPU name — should appear similarly for any working GPU container.

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.20
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd
PyTorch version: 2.6.0+cu124
CUDA version: 12.4
CUDA available: True
GPU name: NVIDIA H200
Tensor on GPU: tensor([[0.8106, 0.3767, 0.5260],
        [0.0261, 0.7204, 0.8969],
        [0.8814, 0.2480, 0.8198]], device='cuda:0')
Matplotlib test passed, saved matplotlib_test.png

A matplotlib_test.png file will also be saved to your working directory.

More Information¶

Documentation

Official Apptainer documentation:

https://apptainer.org/docs/

Local Installation

If you need to build images locally or install Apptainer on your own machine, refer to the official installation guide:

https://apptainer.org/docs/admin/main/installation.html