What Are the Key Hardware Sections of an AI Server?
To truly grasp the intricate composition of an AI server, disassembling its hardware provides invaluable insight into its printed circuit board (PCB) architecture. Using the NVIDIA DGX A100 as a primary reference, given its detailed documentation, and acknowledging the similar design principles found in the DGX H100, an AI server can be functionally segmented into five core hardware areas. This breakdown highlights where the majority of the system's PCB value resides and how it contributes to specialized AI processing capabilities.
The initial sections encountered are the front fan module, comprising eight fans typical of an 8U server layout, followed by the drives and a dedicated front control board. The DGX A100, for instance, integrates eight 3.84 TB drives, accumulating to 30 TB of internal storage, managed by this control board responsible for system oversight and external communications. The most defining sections, however, are found towards the rear: the GPU board assembly, the CPU motherboard assembly, and the power modules that sustain the entire operation.
Analyzing the GPU Board Assembly: Core of AI Processing
The GPU board assembly represents the technological heart of an AI server, distinctly differentiating it from conventional server architectures. This crucial section is primarily responsible for the intensive computational demands of artificial intelligence workloads and contains the most valuable and specialized PCBs within the system.
GPU Substrates: Foundation for Advanced Packaging
At the base of each NVIDIA A100 GPU and its accompanying DRAM lies an advanced substrate, essential for their 2.5D/3D packaging. These FCBGA carriers typically measure between 70x70 mm and 100x100 mm and feature 14 to 16 layers. For a DGX A100, which houses eight GPUs, eight such substrates are required. Industry analysis suggests a value of approximately $100 per substrate, translating to around RMB 650 each, making the total system value for GPU substrates approximately RMB 5,200.
NVSwitch Carriers: Enabling High-Speed Interconnect
NVSwitch modules facilitate the high-bandwidth NVLink communication among GPUs. The NVSwitch carriers, while structurally similar to GPU substrates, have less complex processing requirements but are engineered for ultra-fast data transfer. With an estimated value of $30 per unit (RMB 195) and six units within an A100-based system, these carriers contribute roughly RMB 1,170 to the overall PCB value.
OAM (GPU Accelerator Cards): Housing the Processing Power
The OCP Accelerator Module (OAM) serves as the host for the GPU chip, acting as the GPU accelerator card itself. Each GPU in a DGX A100 corresponds to one OAM, meaning eight are present in the system. High-end configurations, such as the SXM OAM version, demand a 20-layer PCB made from Ultra Low Loss CCL material with multi-stage HDI, costing around RMB 12,000 per square meter for its roughly 0.03 square meter area. More standard PCIe OAMs utilize 14-layer PCBs with a mix of Ultra Low Loss and high Tg FR4 CCL, incorporating single-stage HDI, priced at about RMB 7,000 per square meter. Based on the DGX A100’s high-end setup, each OAM is valued at an estimated RMB 2,880.
UBB (Unit Baseboard): The Central GPU Platform
The Unit Baseboard (UBB) is a singular, large PCB that integrates the entire GPU platform. Based on the chassis dimensions and supply chain insights, the UBB’s area is approximately 0.30 square meters. It is a 26-layer through-hole PCB, constructed with Ultra Low Loss CCL, estimated at RMB 10,000 per square meter, bringing its per-system value to approximately RMB 3,000.
In summary, the DGX A100’s GPU assembly, encompassing GPU substrates, NVSwitch carriers, OAMs, and the UBB, accounts for about 0.624 square meters of PCB area. The estimated total PCB value for this section is approximately RMB 12,250, with substrate-level components making up 52% (RMB 6,370) and PCB-level products forming the remaining 48% (RMB 5,880).
Exploring the CPU Motherboard Assembly: System Control and Memory
The CPU motherboard assembly, while less specialized than the GPU section for AI tasks, forms the backbone of the server's general computing and control functions. This assembly includes the central processing unit and its associated PCBs, alongside system memory and network interfaces.
CPU Substrates and Mainboard
Similar to their GPU counterparts in complexity, CPU substrates provide the foundation for the server's processors. Assuming an approximate value of $100 per substrate and two CPUs in a DGX configuration, the total per-system value for CPU substrates is estimated at RMB 1,300. The CPU motherboard itself, a critical component, houses the CPUs, PCIe switch chips, TPM modules, and various daughter cards. For the DGX A100, which incorporates 64-core AMD processors and PCIe 4.0, the motherboard typically features 10 to 12 layers, utilizes Low Loss CCL, and is a through-hole design. With an estimated area of 0.38 square meters and a market price of around RMB 3,000 per square meter, the CPU motherboard contributes roughly RMB 1,140 to the system's PCB value.
Functional Daughter Boards: Memory, Networking, and Storage
This category includes an array of essential components that extend the server’s capabilities, such as system memory cards, Network Interface Cards (NICs), expansion cards, and the system drive PCB. These boards generally consist of 8 to 10 layers, use Mid Loss CCL, and are priced at about RMB 1,500 per square meter.
● CPU Memory Cards: The DGX A100 integrates 32 CPU memory modules, providing 2 TB of RAM. Each server memory card typically occupies an area of about 0.004 square meters.
● NICs: Featuring Mellanox ConnectX series NICs (e.g., X-7 and X-6), the standard DGX A100 configuration includes ten NICs (eight single-port 200 Gb/s InfiniBand and two dual-port 200 Gb/s Ethernet). Each ConnectX-7 NIC PCB has an area of approximately 0.012 square meters.
● Riser/Expansion Card: A single riser card, with an estimated area of 0.01 square meters, accommodates specific board placements.
● System Drive PCB: The DGX A100 utilizes two 1.92 TB M.2 NVMe system drives mounted on a single PCB, estimated at 0.01 square meters.
Collectively, these functional daughter boards accumulate to about 0.27 square meters, contributing an estimated RMB 405 to the per-system PCB value. The DGX A100’s CPU motherboard assembly, totaling approximately 0.662 square meters of PCB, has an estimated per-system value of RMB 2,845. This value is distributed with substrates accounting for 46%, the main motherboard 40%, and functional daughter boards 14%.
Auxiliary Components: Power, Drives, and Control Boards
Beyond the core CPU and GPU assemblies, other essential accessory modules ensure the AI server's operational stability and functionality. These components, while individually less significant in terms of PCB value compared to the main processing units, are indispensable for a complete and reliable system.
Power Supplies and Storage Drives
The DGX A100 incorporates six power supply units. Based on a reference 2,200 W server PSU, each power supply PCB measures approximately 0.019 square meters. Additionally, the system houses eight hard drives, with each 3.5" drive containing a PCB area estimated at 0.008 square meters. These boards, along with the front control board, generally feature 6 to 10 layers using FR4/Mid Loss CCL and are valued between RMB 1,000 to RMB 1,500 per square meter. The front control board, positioned between the eight drives, has an estimated area of 0.01 square meters.
The aggregate PCB value for these auxiliary components, including the power supplies, drives, and the front control board, is estimated to be around RMB 226 per system.
Overall PCB Footprint and Value in an AI Server
A comprehensive look at the internal architecture of an AI server like the NVIDIA DGX A100 reveals a complex interplay of various PCB types, each contributing differently to the system's overall functionality and cost. The estimated total PCB area for a DGX A100 system reaches approximately 1.474 square meters, with an estimated total PCB value of RMB 15,321.
Value Distribution Across Components
The GPU assembly stands out as the most significant contributor to the total PCB value, accounting for approximately RMB 12,250 (80%). This highlights its role as the primary driver of an AI server’s specialized capabilities. The CPU motherboard assembly follows, contributing about RMB 2,845 (19%), underscoring its foundational importance for general computing and system management. The remaining accessories, including power supplies and drives, make up a smaller but still essential portion, at about RMB 226 (1%).
PCB Classification by Type
When categorized by board type, substrate-level products—those integral to advanced packaging for GPUs and CPUs—represent a substantial RMB 7,670 (50.1%) of the total PCB value. The remaining RMB 7,651 (49.9%) comes from PCB-level products, which encompass the various mainboards, accelerator cards, and functional daughter boards throughout the system. This detailed analysis underscores the sophisticated engineering and high material value embedded within modern AI server hardware.
