Skip to content

Communication Network

The communication infrastructure of the HPC PERUN system is designed to provide extremely fast, reliable, and low-latency data transfer between computing nodes, storage systems, and user environments.

The network architecture consists of two interconnected segments — the InfiniBand computing network and the universal data & management network.
Together, they form the communication backbone of the entire supercomputing ecosystem.

Communication Network – Cisco Nexus Switches

1. InfiniBand Network – High-Performance Computing Architecture

The core of the computational environment is built on a non-blocking Fat-Tree InfiniBand network architecture, using:

  • NVIDIA InfiniBand NDR Quantum-2 64-port OSFP switches
  • Bandwidth: 400 Gb/s per link

This topology ensures a fully connected structure between all compute nodes without throughput limitations.

Key properties:

  • Extremely low latency
  • High aggregate bandwidth
  • No oversubscription between nodes
  • Efficient scaling for parallel workloads
  • Balanced system-wide communication even at full load

Thanks to this design, the InfiniBand network supports the demanding requirements of HPC applications, including large-scale simulations, AI training, multi-GPU workloads, and distributed parallel computing.

2. Universal Data Network

The second major part of the communication infrastructure is the data and management network, responsible for:

  • user access
  • system management and orchestration
  • connectivity to storage systems
  • integration with the university and external research networks

It consists of high-performance and redundant switching devices, including:

  • Cisco Nexus 9364D-GX2A
  • Cisco Nexus 9348GC-FX3

This network provides:

  • high availability and redundancy
  • modern data-transfer standards
  • automated management and monitoring
  • stable connectivity for user services and system infrastructure

The universal data network ensures a reliable environment for system operation, user interaction, and data flow outside the high-speed InfiniBand computation core.

Together, the InfiniBand network and the universal data network form the robust backbone of the PERUN supercomputing system, enabling:

  • high-performance parallel computation
  • efficient and stable data transfer
  • uninterrupted communication across all components of the HPC ecosystem.