What are NVLink and NVSwitch and how do they work?

NVLink and NVSwitch help solve a key challenge in modern AI and HPC systems by enabling fast com­mu­ni­ca­tion between multiple GPUs. Their high data through­put supports tightly connected GPU ar­chi­tec­tures and makes it possible to build scalable, high-per­for­mance GPU clusters.

What is NVLink?

NVLink is NVIDIA’s pro­pri­etary high-speed wired in­ter­con­nect for direct data exchange between GPUs and, in some con­fig­u­ra­tions, CPUs. For GPU-to-GPU com­mu­ni­ca­tion, it avoids the need to route data through the PCIe bus, helping to reduce bot­tle­necks and increase through­put in HPC, AI training and memory-intensive parallel workloads across large clusters and ac­cel­er­at­ed servers.

Tech­ni­cal­ly, NVLink uses direct point-to-point con­nec­tions with separate transmit and receive paths, enabling full-duplex com­mu­ni­ca­tion so data can be sent and received at the same time. NVIDIA also describes NVLink as a low-latency shared-memory in­ter­con­nect that allows connected GPUs to access each other’s memory regions quickly. When two GPUs are phys­i­cal­ly connected via NVLink, CUDA peer-to-peer transfers can use NVLink instead of PCIe, the standard expansion bus commonly used in servers.

How has NVLink evolved and what are its technical spec­i­fi­ca­tions?

NVLink has evolved steadily since the Pascal gen­er­a­tion. Early de­vel­op­ments focused on in­creas­ing bandwidth per link, while later gen­er­a­tions primarily expanded the number of links available per GPU and, with it, the total bidi­rec­tion­al bandwidth. In the Blackwell gen­er­a­tion, this reaches 1.8 TB/s per GPU, compared with 900 GB/s for Hopper and 600 GB/s for A100.

• Max. bandwidth per link (bidi­rec­tion­al) indicates the total amount of data a single NVLink can transfer si­mul­ta­ne­ous­ly in both di­rec­tions, combining send and receive traffic.
• SerDes lanes per direction shows how many physical signal lanes are used in each direction. SerDes stands for se­ri­al­iz­er/de­se­ri­al­iz­er, a tech­nol­o­gy that converts parallel data streams into high-speed serial signals and back again.
• Max. links per GPU describes the maximum number of in­di­vid­ual NVLink con­nec­tions a GPU can support at the same time. A higher number allows the GPU to be connected more closely with other GPUs in a multi-GPU system.
• Max. bandwidth per GPU (bidi­rec­tion­al) refers to the combined the­o­ret­i­cal data rate of all NVLink con­nec­tions on a GPU in both di­rec­tions. This value is par­tic­u­lar­ly important when scaling AI and HPC systems.

How does NVSwitch support multi-GPU systems?

NVLink is fun­da­men­tal­ly a point-to-point tech­nol­o­gy. NVSwitch extends this principle into a full switch fabric. NVIDIA describes NVSwitch as a physical switching component that connects multiple NVLinks and enables all-to-all com­mu­ni­ca­tion between GPUs at full NVLink speed. In NVSwitch-based HGX and DGX systems, this creates a unified GPU fabric that can function as a tightly connected computing en­vi­ron­ment for many workloads.

This is par­tic­u­lar­ly important for AI and high-per­for­mance computing. When training large language models or running sci­en­tif­ic sim­u­la­tions, workloads are dis­trib­uted across multiple GPUs. In­ter­me­di­ate results, model pa­ra­me­ters and memory contents must therefore be exchanged con­tin­u­ous­ly between them. NVSwitch enables this by creating a tightly coupled GPU cluster that can, for certain workloads, operate almost like a single large computing system.

For modern DGX and HGX systems – NVIDIA’s complete AI servers and the un­der­ly­ing multi-GPU platforms used in data centers – NVSwitch is therefore a key component. This ar­chi­tec­ture allows multi-GPU servers to scale ef­fi­cient­ly while main­tain­ing high per­for­mance for demanding workloads such as large-model and LLM training.

How do NVLink and PCIe differ for AI workloads?

PCIe remains the standard interface for con­nect­ing GPUs to CPUs, NICs and other I/O com­po­nents. However, for intensive GPU-to-GPU com­mu­ni­ca­tion, it is con­sid­er­ably less suitable than NVLink.

The main ad­van­tages of NVLink over a PCIe-centered design include:

• Higher through­put per GPU: The H100 provides 900 GB/s of bidi­rec­tion­al bandwidth across 18 links, while Blackwell reaches 1.8 TB/s. By com­par­i­son, PCIe Gen5 x16 offers 126 GB/s bidi­rec­tion­al­ly.
• Lower and more pre­dictable latencies: NVLink uses dedicated point-to-point con­nec­tions with full-duplex trans­mis­sion and avoids many of the detours through host and I/O struc­tures required for GPU-to-GPU transfers. This is par­tic­u­lar­ly ben­e­fi­cial for syn­chro­niza­tion-intensive workloads such as dis­trib­uted LLM training.
• Direct peer access to remote GPU memory: NVIDIA describes NVLink as a shared-memory in­ter­con­nect. Peer memory can be accessed using loads, stores and atomics rather than trans­fer­ring data through host memory first.
• Fewer CPU detours: CUDA can route transfers between NVLink-connected endpoints via NVLink instead of PCIe. This reduces overhead and places less load on the host path.
• Better con­di­tions for memory pooling and unified memory: NVLink does not au­to­mat­i­cal­ly combine the HBM of multiple GPUs into a single physical memory pool. However, it enables faster access to peer memory, improving the con­di­tions for multi-GPU workloads that require larger amounts of memory and for unified-memory-like access across multiple ac­cel­er­a­tors.
• Scaling via switch fabric rather than a pure I/O tree: NVSwitch connects multiple NVLinks to support all-to-all com­mu­ni­ca­tion at the fabric level. PCIe is not designed for this type of GPU in­ter­con­nect ar­chi­tec­ture.

How do topolo­gies affect scaling in DGX and HGX systems?

In DGX and HGX systems – complete NVIDIA AI servers and the un­der­ly­ing multi-GPU platforms – the ad­van­tages of NVLink and NVSwitch become par­tic­u­lar­ly apparent. Per­for­mance depends not only on the bandwidth of in­di­vid­ual con­nec­tions, but also on the topology, meaning how GPUs are in­ter­con­nect­ed within a system or across multiple systems.

Overview of typical topolo­gies:

• Direct GPU con­nec­tions: In early NVLink systems, GPUs were connected through fixed point-to-point links. This sig­nif­i­cant­ly ac­cel­er­at­ed data exchange, although not every GPU could com­mu­ni­cate with every other GPU equally ef­fi­cient­ly.
• NVSwitch fabric within a server: NVSwitch creates a shared in­ter­con­nect fabric that allows multiple GPUs to exchange data si­mul­ta­ne­ous­ly with fewer in­ter­me­di­ate steps. This is par­tic­u­lar­ly important for training workloads that require con­tin­u­ous syn­chro­niza­tion between GPUs.
• Large NVLink domains at rack level: In newer systems, this principle extends beyond in­di­vid­ual servers. Multiple GPU units can be combined into a larger NVLink domain, allowing very large AI models to be processed across tightly connected ac­cel­er­a­tors.

In practice, this de­vel­op­ment is par­tic­u­lar­ly important for scaling. The more ef­fi­cient­ly GPUs are in­ter­con­nect­ed, the easier it becomes to dis­trib­ute large models and workloads across multiple ac­cel­er­a­tors.