# What are NVLink and NVSwitch and how do they work? NVLink and NVSwitch help solve a key challenge in modern AI and HPC systems by enabling fast communication between multiple GPUs. Their high data throughput supports tightly connected GPU architectures and makes it possible to build scalable, high-performance GPU clusters. ### Key Takeaways The key facts about NVLink and NVSwitch at a glance: - NVLink is NVIDIA’s proprietary point-to-point interconnect for direct data exchange between GPUs, using separate transmit and receive paths for full-duplex communication. - NVSwitch extends this principle into a switch fabric and enables all-to-all communication between every connected GPU. - Bandwidth per GPU rose from 160 GB/s with Pascal to 1.8 TB/s with Blackwell. - Compared with PCIe Gen5 x16 at 126 GB/s bidirectionally, NVLink offers several times the throughput and more predictable latencies. - Over NVLink, GPUs access each other’s memory directly using loads, stores and atomics, without a detour through host memory. - In DGX and HGX systems, scaling depends not only on bandwidth but above all on the topology. - At rack level, multiple GPU units can be combined into a single large NVLink domain. ## What is NVLink? NVLink is NVIDIA’s proprietary **high-speed wired interconnect** for direct data exchange between GPUs and, in some configurations, CPUs. For GPU-to-GPU communication, it avoids the need to route data through the PCIe bus, helping to reduce bottlenecks and increase throughput in HPC, AI training and memory-intensive parallel workloads across large clusters and accelerated servers. Technically, NVLink uses direct point-to-point connections with separate transmit and receive paths, enabling **full-duplex** communication so data can be sent and received at the same time. NVIDIA also describes NVLink as a low-latency shared-memory interconnect that allows connected GPUs to access each other’s memory regions quickly. When two GPUs are physically connected via NVLink, CUDA peer-to-peer transfers can use NVLink instead of PCIe, the standard expansion bus commonly used in servers. ## How has NVLink evolved and what are its technical specifications? NVLink has evolved steadily since the Pascal generation. Early developments focused on increasing bandwidth per link, while later generations primarily expanded the number of links available per GPU and, with it, the total bidirectional bandwidth. In the [Blackwell](https://www.ionos.com/digitalguide/server/know-how/nvidia-blackwell/) generation, this reaches 1.8 TB/s per GPU, compared with 900 GB/s for [Hopper](https://www.ionos.com/digitalguide/server/know-how/hopper-gpu/) and 600 GB/s for A100.
Generation Typical architecture Max. bandwidth per link (bidirectional) SerDes lanes per direction Max. links per GPU Max. bandwidth per GPU (bidirectional)
NVLink 1 Pascal 40 GB/s 8 4 160 GB/s
NVLink 2 Volta 50 GB/s 8 6 300 GB/s
NVLink 3 Ampere 50 GB/s 4 12 600 GB/s
NVLink 4 Hopper 50 GB/s 2 18 900 GB/s
NVLink 5 Blackwell 100 GB/s 2 18 1.8 TB/s
NVLink 6 Rubin 100 GB/s not specified 36 3.6 TB/s
Note The NVLink 6 specifications for the Rubin platform are preliminary and may still change. - **Max. bandwidth per link (bidirectional)** indicates the total amount of data a single NVLink can transfer simultaneously in both directions, combining send and receive traffic. - **SerDes lanes per direction** shows how many physical signal lanes are used in each direction. SerDes stands for serializer/deserializer, a technology that converts parallel data streams into high-speed serial signals and back again. - **Max. links per GPU** describes the maximum number of individual NVLink connections a GPU can support at the same time. A higher number allows the GPU to be connected more closely with other GPUs in a multi-GPU system. - **Max. bandwidth per GPU (bidirectional)** refers to the combined theoretical data rate of all NVLink connections on a GPU in both directions. This value is particularly important when scaling AI and HPC systems. ## How does NVSwitch support multi-GPU systems? NVLink is fundamentally a point-to-point technology. NVSwitch extends this principle into a full switch fabric. NVIDIA describes NVSwitch as a physical switching component that **connects multiple NVLinks** and enables all-to-all communication between GPUs at full NVLink speed. In NVSwitch-based HGX and DGX systems, this creates a unified GPU fabric that can function as a tightly connected computing environment for many workloads. This is particularly important for AI and [high-performance computing](https://www.ionos.com/digitalguide/server/know-how/hpc-high-performance-computing/). When training large language models or running scientific simulations, workloads are distributed across multiple GPUs. Intermediate results, model parameters and memory contents must therefore be exchanged continuously between them. NVSwitch enables this by creating a **tightly coupled GPU cluster** that can, for certain workloads, operate almost like a single large computing system. For modern DGX and HGX systems – NVIDIA’s complete AI servers and the underlying multi-GPU platforms used in data centers – NVSwitch is therefore a key component. This architecture allows multi-[GPU servers](https://www.ionos.com/digitalguide/server/know-how/gpu-server/) to scale efficiently while maintaining high performance for demanding workloads such as large-model and LLM training. ## How do NVLink and PCIe differ for AI workloads? PCIe remains the standard interface for connecting GPUs to CPUs, NICs and other I/O components. However, for intensive GPU-to-GPU communication, it is considerably less suitable than NVLink. The main advantages of NVLink over a PCIe-centered design include: - **Higher throughput per GPU**: The H100 provides 900 GB/s of bidirectional bandwidth across 18 links, while Blackwell reaches 1.8 TB/s. By comparison, PCIe Gen5 x16 offers 126 GB/s bidirectionally. - **Lower and more predictable latencies**: NVLink uses dedicated point-to-point connections with full-duplex transmission and avoids many of the detours through host and I/O structures required for GPU-to-GPU transfers. This is particularly beneficial for synchronization-intensive workloads such as distributed LLM training. - **Direct peer access to remote GPU memory**: NVIDIA describes NVLink as a shared-memory interconnect. Peer memory can be accessed using loads, stores and atomics rather than transferring data through host memory first. - **Fewer CPU detours**: CUDA can route transfers between NVLink-connected endpoints via NVLink instead of PCIe. This reduces overhead and places less load on the host path. - **Better conditions for memory pooling and unified memory**: NVLink does not automatically combine the HBM of multiple GPUs into a single physical memory pool. However, it enables faster access to peer memory, improving the conditions for multi-GPU workloads that require larger amounts of memory and for unified-memory-like access across multiple accelerators. - **Scaling via switch fabric rather than a pure I/O tree**: NVSwitch connects multiple NVLinks to support all-to-all communication at the fabric level. PCIe is not designed for this type of GPU interconnect architecture. ## How do topologies affect scaling in DGX and HGX systems? In DGX and HGX systems – complete NVIDIA [AI servers](https://www.ionos.com/digitalguide/server/know-how/ai-server/) and the underlying multi-GPU platforms – the advantages of NVLink and NVSwitch become particularly apparent. Performance depends not only on the bandwidth of individual connections, but also on the topology, meaning how GPUs are interconnected within a system or across multiple systems. **Overview of typical topologies:** - **Direct GPU connections**: In early NVLink systems, GPUs were connected through fixed point-to-point links. This significantly accelerated data exchange, although not every GPU could communicate with every other GPU equally efficiently. - **NVSwitch fabric within a server**: NVSwitch creates a shared interconnect fabric that allows multiple GPUs to exchange data simultaneously with fewer intermediate steps. This is particularly important for training workloads that require continuous synchronization between GPUs. - **Large NVLink domains at rack level**: In newer systems, this principle extends beyond individual servers. Multiple GPU units can be combined into a larger NVLink domain, allowing very large AI models to be processed across tightly connected accelerators. In practice, this development is particularly important for scaling. The more efficiently GPUs are interconnected, the easier it becomes to distribute large models and workloads across multiple accelerators. This is a markdown version of: [https://www.ionos.com/digitalguide/server/know-how/nvlink-and-nvswitch/](https://www.ionos.com/digitalguide/server/know-how/nvlink-and-nvswitch/) for AI/LLM consumption.