NVIDIA HGX 8-GPU SXM5 baseboards still define the high end of the 2026 AI server market, with Blackwell B200 parts delivering 180 GB HBM3e per GPU at 8 TB/s and 1.8 TB/s of NVLink 5 bandwidth per device [S1]. Eight of those GPUs are wired all-to-all through NVSwitch rather than PCIe, so the platform behaves as one large accelerator mesh instead of a cluster of add-in cards [S1].
That baseboard architecture is now the design template competing accelerators are being measured against. AMD Instinct MI325X OAM modules pack 256 GB HBM3e and 6.0 TB/s of memory bandwidth at a 1,000 W TBP on the CDNA 3 architecture, with Infinity Fabric as the scale-out interconnect [S1]. Dell's PowerEdge XE9680 accepts either 8x HGX H100/H200 SXM or 8x MI300X/Gaudi 3 OAM parts in a single 6U chassis, while the 2U XE9640 takes four SXM or OAM GPUs for smaller training and inference nodes [S1].
Training versus inference: the box is no longer one-size-fits-all
Training jobs run synchronised across an 8-GPU NVLink mesh, want maximum HBM capacity, FP8/FP4 throughput, and a 400/800 GbE or InfiniBand east-west fabric to handle all-reduce, and run near-continuously at full load, which is the design case for liquid cooling and dedicated colocation halls [S1]. Inference is a different workload: token throughput tracks HBM bandwidth rather than raw FP8 throughput, so lower-power PCIe cards like the L40S (48 GB GDDR6, 864 GB/s, 350 W, no NVLink) often beat SXM parts on cost-per-token [S1].
The split has practical consequences for rack power and cooling. A fully populated HGX H200 sled sits at roughly 5.6 kW just on the GPUs (8x 700 W), and a B200 sled reaches 8 kW, which is why the pressure transmitter and flow meter loops on the coolant distribution unit become part of the platform spec rather than a building-side afterthought. Training racks are typically liquid-cooled, while PCIe-based inference racks can still run on air and standard data-hall power feeds [S1].
GPU spec comparison: NVIDIA, AMD and the rest in one table
Every number in the table below comes from manufacturer datasheets, so it does not go stale with quarterly model refreshes [S1]. HBM3e is the differentiator at the high end: the H200 SXM at 141 GB and 4.8 TB/s was the 2024-2025 step, the B200 SXM at 180 GB and 8 TB/s is the 2025-2026 step, and AMD's MI325X at 256 GB and 6.0 TB/s is the highest-capacity single-GPU part shipping in 2026 [S1].
The 2x jump in NVLink bandwidth, from 900 GB/s on Hopper to 1.8 TB/s on Blackwell via 18 links at 100 GB/s each, is what lets a single 8-GPU B200 node handle model-parallel training jobs that previously needed multi-node InfiniBand collectives [S1]. H100 PCIe cards bridge only in pairs and top out at 600 GB/s on the dual-GPU NVL card, so they are not a substitute for an SXM mesh on large training jobs [S1].
NVLink 5 versus Infinity Fabric versus Gaudi: the fabric war
Per-GPU interconnect bandwidth is now the single biggest specification gap between competing platforms, and the table above is what buyers should be reading before looking at TFLOPS marketing slides. NVIDIA NVLink 5 at 1.8 TB/s per GPU is 2x NVLink 4 (900 GB/s) and 3x NVLink 3 on A100 (600 GB/s) [S1]. AMD's Infinity Fabric on MI325X is the PCIe Gen5 x16 path into the host, and AMD's strength is the highest single-GPU HBM3e capacity in the market at 256 GB, which matters for long-context inference where the model weights fit on one device [S1].
Intel's Gaudi 3 takes the OAM slot in the same Dell XE9680 sled as MI300X, trading peak interconnect for Ethernet-native scale-out. None of these three platforms are drop-in compatible: HGX baseboards carry NVSwitch silicon and SXM sockets, OAM sleds carry different mezzanine cards, and the host CPU topology (typically dual-socket Xeon or EPYC with 1-2 TB of DDR5) is the same envelope, but the GPU-to-GPU mesh is fundamentally different [S1]. The serial server management plane and out-of-band monitoring are where the platforms look most alike, since they all use standard BMC stacks for power, thermal and inventory telemetry.
PCBs, CCL and the supply chain underneath the box
AI server platforms are reshaping the upstream PCB and copper-clad laminate (CCL) market in 2026, and that is where the real supply bottlenecks sit. AI server mainboards now need large-format, high-layer-count PCBs above 40 layers with ultra-low-loss dielectrics to carry 112 Gb/s SerDes and PCIe Gen5/Gen6 signalling [S2]. The global CCL market was US$16.02 billion in 2025 and is forecast to hit US$21.5 billion in 2026, a 34.2% year-on-year jump driven by AI specification upgrades [S2].
Taiwanese suppliers held a combined 37.4% global CCL share in 2025, with Elite Material Co. leading the world at 18.9% [S2]. Japanese suppliers still own the upstream IC substrate materials and glass fabrics, which is the structural gap in the supply chain. To reduce that exposure, Taiwanese makers are pushing Low Dk2 glass fabric, quartz fabric and PTFE formulations for next-generation AI boards [S2]. If you are sizing a new AI server program, the CCL and substrate lead-time, not the GPU allocation, is usually the binding constraint on shipment dates in 2026. This is also where PLC and industrial valve demand spikes at the substrate fab level, since the wet processes for low-Dk glass and quartz CCL are much more chemical-handling-intensive than older FR-4 lines.
Selection criteria: training cluster, inference farm, or hybrid
Pick the platform by workload, not by brand. For frontier-model training above the 70B-parameter scale, the 8-GPU SXM mesh is effectively mandatory: HGX B200 at 8 TB/s HBM per GPU and 1.8 TB/s NVLink 5 is the only shipping platform that holds the full NVLink mesh and HBM3e capacity in one box [S1]. For mixed training and inference at lower parameter counts, HGX H200 or HGX H100 sleds still give the best software support, with the XE9680 6U form factor accepting either H100/H200 SXM or MI300X/Gaudi 3 OAM parts in the same chassis [S1].
For inference-heavy fleets, PCIe cards like the L40S (48 GB GDDR6, 864 GB/s, 350 W, no NVLink) deliver the lowest cost per token on standard 2U servers, since they avoid the SXM power and cooling envelope entirely [S1]. The AMD MI325X at 256 GB HBM3e and 6.0 TB/s is the most attractive option for long-context single-GPU inference, since the model weights can stay on one device. Air cooling still works in 2U and most 4U sleds; liquid cooling becomes mandatory once a sled crosses roughly 6 kW of GPU load, which is the B200 and GB200 envelope. The pressure sensor and flow instrumentation on the cold plate loop is therefore part of the standard BOM for any 2026 B200 or GB200 deployment.
Who the 2026 AI server market is for, and who should wait
The 2026 AI server market is for hyperscalers, sovereign-AI programmes and large enterprises training or serving models in the multi-billion-parameter range, where NVLink 5 bandwidth, 180+ GB HBM3e per GPU, and liquid-cooling-ready rack densities of 50-100 kW pay back inside one model cycle [S1][S2]. It is also for system integrators building on the Dell XE9680 / XE9640 platform, since those sleds accept NVIDIA, AMD or Intel GPUs in the same chassis and let a single deployment mix accelerator vendors by slot [S1].
It is not for small on-premises inference shops, edge sites, or any deployment where total per-node power stays below 4 kW, because the SXM platforms are physically and economically oversized for those jobs and PCIe inference cards are a better match. It is also not a good fit for buyers who cannot secure the 40+ layer ultra-low-loss CCL supply, since lead times on those materials are the dominant schedule risk in 2026 [S2]. A practical decision rule: if your per-GPU HBM requirement is above 80 GB and your job is all-reduce-heavy training, you need an SXM/OAM 8-GPU sled; if your per-GPU HBM requirement is below 80 GB and latency-per-token is what you are billed on, stay on PCIe inference hardware and skip the liquid-cooling capex entirely [S1].
Three signals to watch over the next two quarters: NVIDIA GB200 NVL72 rack-scale shipments and the per-rack power draw they settle on, AMD MI400 series announcements that would close the NVLink 5 interconnect gap, and the second-half 2026 CCL capacity additions out of Taiwan and Japan that determine whether 40+ layer PCB lead times finally ease from the current allocation regime [S1][S2]. NVIDIA Blackwell vs AMD MI400 vs Intel Gaudi 4: 2026 AI Accelerator Spec Breakdown and PCB Demand 2026-2030: AI Server Pull, HDI Migration, and Capacity Bottlenecks are the natural next reads for buyers tracking those signals.