AI thermal management after direct-to-chip liquid: why a liquid-cooled AI rack still runs fans, what stays in air, and where solid-state airflow fits in.
NVIDIA's own hardware documentation for the DGX GB200 rack contains a sentence that almost every liquid cooling article skips. After describing how coolant runs up and down the rack through manifolds and into cold plates bonded to the CPUs and GPUs, it adds: "The rest of the components like networking and storage devices are air cooled, which is pushed through the system by the fans." [3] The power shelf in the same rack, it notes, "uses six air-cooled 5.5kW PSUs". [3]
The most aggressively liquid-cooled rack in mainstream production is, in part, still an air-cooled rack. That is the subject of this article.
AI thermal management is the practice of removing heat from accelerator-dense computing hardware, where a single rack can draw well over a hundred kilowatts, cold plates carry the processors, and a residual air path still has to cool everything the cold plates never touch.
It is worth separating from classical data centre cooling. That was a facility question: chilled water, economiser hours, containment, PUE. AI thermal management is a facility and board-level question at once, and the board-level half is under-served.
The Uptime Institute Global Data Center Survey 2025 reports a weighted average annual PUE of 1.54, "the sixth consecutive year that this headline figure has virtually stood still". [1] Its density distribution shows how far most estates sit from AI conditions: 30 per cent of respondents report a modal rack density of 4 to 5 kW, and only 3 per cent report 30 kW or above. [1]
Against that, NVIDIA describes the GB200 NVL72 as requiring 120 kW of cooling capacity per rack, [2] and Supermicro's NVL72 SuperCluster datasheet lists operating power of 125 kW to 135 kW. [4] That is roughly an order of magnitude above the band most halls were designed and staffed for.
The problem is not that silicon suddenly runs hotter. Junction temperature limits have barely moved. It is heat flux at the package and total load in the cabinet. Move 130 kW out of a 42U enclosure and the air mass flow required, at any sensible temperature rise, exceeds what the rack geometry can pass.
Air cooling is a mass flow problem. The heat a stream of air can carry is Q = m x cp x dT: mass flow rate times specific heat capacity times the temperature rise across the equipment. Air has a specific heat capacity near 1.0 kJ/kg·K and a density around 1.2 kg/m³ at typical inlet conditions. Water carries roughly four times the heat per unit mass and is about 800 times denser. That ratio is the whole argument for liquid.
You cannot buy much back from the temperature side. ASHRAE TC 9.9 sets the recommended inlet envelope at 18 to 27 °C for all equipment classes, with Class A4 allowable up to 45 °C. [5] Widening the envelope buys economiser hours and lowers facility power, but it does not increase the heat a given volume of air can remove. It shrinks the dT headroom you had.
That leaves mass flow, which means fans, and fans scale badly. Fan power rises roughly with the cube of flow rate while heat removed rises linearly, so doubling airflow to double the cooling costs about eight times the fan power. Somewhere around 30 to 50 kW per rack, the fan power, acoustic penalty and pressure drop stop being worth paying. This is a physical ceiling on high density rack cooling, not an engineering backlog, and the same argument at board level is worked through in ionic wind versus fans for electronics cooling.
Data centre cooling energy consumption is the visible symptom: the IEA puts the share of cooling systems in total data centre consumption at "about 7% for efficient hyperscale data centres to over 30% for less-efficient enterprise data centres". [6] Averages also hide the failures that take hardware offline. Hot spot cooling in a data centre is local: recirculation behind a poorly blanked cabinet, a stagnant pocket behind a power shelf, a transceiver cage starved by its neighbours. A rack can pass its inlet temperature audit and still cook one module.
Direct-to-chip liquid cooling, also called DLC or cold-plate cooling, brings a coolant loop to a metal plate on the die package and returns it to a coolant distribution unit that hands the heat to the building loop. It works because it shortens the thermal path: instead of die to lid to heatsink to air to room, you get die to lid to plate to liquid.
This is the right answer for a high-TDP accelerator, and it should be said without qualification. Nothing solid-state, nothing plasma, nothing in this article displaces a cold plate on a modern GPU. NVIDIA implemented "direct liquid cooling techniques" with a blind-mate manifold for the NVL72 and contributed the designs to the Open Compute Project, [2] about as clear a signal as the industry gives that direct-to-chip is the default above 100 kW rather than an exotic option.
Single-phase loops circulate water or a water-glycol mix and dominate deployment; two-phase loops boil a dielectric at the plate for higher flux at a near-isothermal plate temperature, at the cost of fluid handling complexity. Around them sit immersion, thermally excellent but held back by servicing, fluid compatibility and regulatory pressure on certain fluorinated fluids, and rear-door heat exchangers, the pragmatic retrofit for existing halls whose ceiling is set by server fan mass flow, because the heat still has to leave the chassis in air first.
Here is the part that gets skipped: liquid cooling the accelerator does not remove every fan from the chassis.
A peer-reviewed testbed study of direct-to-chip cold plate cooling measured a heat capture ratio by the fluid of 94 per cent on a single 53 kW rack and 93 per cent across a 128 kW multi-rack test, with the balance removed in air. [7] Those are strong numbers, taken with cold plates on the high-power heaters, and they are the optimistic end: the more components sit outside a plate, the larger the residual. Supermicro advertises "custom-designed cold plates optimized for various GPUs, CPUs and memory modules", [4] an admission that memory sits on the boundary and that vendors draw that boundary differently.
The residual concentrates in the components hardest to reach.
| Component | Usual cooling in an AI rack | What limits it |
|---|---|---|
| GPU or accelerator die | Direct-to-chip cold plate | Coolant supply temperature, plate thermal resistance |
| CPU | Direct-to-chip cold plate | Same loop, shared budget |
| DIMMs and memory modules | Air in most designs, plates in some [4] | Narrow inter-DIMM channels, low local velocity |
| Voltage regulators | Air | Board layout, position in the airflow shadow |
| NICs and network devices | Air, "pushed through the system by the fans" [3] | Faceplate density, no room for a heatsink |
| Optical modules | Air | Cage clearance, case temperature budget falls as speed rises |
| Power shelf PSUs | Air, six air-cooled 5.5 kW units in the DGX GB200 [3] | PSU inlet temperature, rear recirculation |
Every row in the air column is served by a fan, and an AI cooling fan has the same failure modes as any other fan. It fails by bearing wear. It ingests dust and gets less effective before it gets loud. It draws power against the same budget the cold plate loop was meant to protect. And it cannot reach the places where the resistance actually sits.
That last point is a boundary layer problem. In a transceiver cage with millimetre clearances, in a stagnant pocket behind a power shelf, or in a sealed enclosure with no through-flow, the limiting resistance is the thin, slow layer of air clinging to the surface. Pushing more bulk air down the aisle does not thin it, there is no room for a rotating machine to do so locally, and no liquid loop will ever be plumbed to a QSFP cage.
A dielectric barrier discharge plasma actuator is two electrodes separated by a thin dielectric, one exposed and one encapsulated. A high-voltage AC drive breaks down a shallow layer of air at the exposed electrode; ions accelerate along the surface and drag neutral air with them, forming a wall jet of a few metres per second. The device is roughly 200 micrometres thick, has no blades and no bearings, and the discharge is non-thermal, so it adds no meaningful heat to what it cools. The mechanism is covered in how DBD plasma actuators work.
The published measurements are about local enhancement, not bulk flow. Go and colleagues superimposed an ionic wind on an existing 0.3 m/s bulk airflow and measured roughly a factor-of-two increase in the average heat transfer coefficient and about 20 K of additional cooling, drawing under 100 mW. [8] That is a boundary-layer result in exactly the low-flow regime the residual air path of a liquid-cooled rack lives in.
The placement is therefore narrow: not on the accelerator, where the cold plate belongs, but in the volumes the cold plate never covered and the chassis fan cannot serve. Optical transceiver cages are the clearest case, treated in ionic wind cooling for data centre optical transceivers; sealed enclosures in edge AI hardware are the other. The physics sits in our guide to ionic wind and solid-state cooling, and the board-level version in fanless GPU cooling.
Microfluidic cooling etches coolant channels into the silicon itself, or into a manifold bonded to the die, so liquid reaches the heat source without passing through a lid, a thermal interface and a plate. Microsoft has reported a lab-scale in-chip system that removed heat "up to three times better than cold plates" depending on workload and configuration, and cut the maximum temperature rise of the silicon inside a GPU by 65 per cent, while stating it remains a prototype and that the company is still working with fabrication and silicon partners to bring it into production. [9] In the literature, van Erp and colleagues demonstrated monolithically integrated manifold microchannels cooling heat fluxes above 1.7 kW/cm² using 0.57 W/cm² of pumping power. [10]
The framing matters. Microfluidic cooling competes with the cold plate. It is a candidate replacement for the liquid layer, not for air movement. If it succeeds at scale, the DIMMs, regulators, NICs, optics and power shelf are exactly where they were, and the residual air-cooled fraction grows as a share of the total rather than shrinking. Solid-state airflow and on-die microfluidics address different rows of the table above.
Rack density is the primary selector, retrofit constraint the tiebreaker.
| Rack density | What to do | The question people forget |
|---|---|---|
| Under 15 kW | Conventional air with disciplined containment | Nothing else is justified |
| 15 to 30 kW | Optimised air, higher inlet under A3/A4, blanking discipline | Fan power as a share of IT load |
| 30 to 60 kW | Rear-door heat exchanger for existing halls, direct-to-chip if the refresh allows | Server fans still set the ceiling |
| 60 to 100 kW | Direct-to-chip liquid, single-phase first | Size the residual air load explicitly |
| Above 100 kW | Direct-to-chip is assumed | CDU redundancy, facility water temperature, and the air-cooled remainder |
| Any density, sealed or constrained volume | Neither fans nor liquid loops apply | This is the solid-state airflow case |
Ionic wind is a poor bulk air mover. Electrical-to-fluid energy conversion sits around 1 to 2 per cent, [8] and the flow it produces is metres per second, not tens of metres per second. If the problem needs high volumetric flow, a fan is still the correct component. If the problem is a kilowatt-class accelerator, a cold plate is still the correct component, and no amount of surface airflow changes that.
The honest constraints are a kilovolt-class AC drive supply, some ozone generation in air that is waveform and duty cycle dependent and manageable by design, and dielectric ageing under continuous electrical stress. We cover those in ionic wind reliability and ozone explained and compare alternatives in the solid-state cooling landscape.
Direct-to-chip liquid has settled the high-density accelerator question. It has not settled the low-flow, sealed and boundary-layer-limited parts of the same rack, and those are growing as a share of the problem.
YPlasma builds solid-state DBD plasma actuators that move air with no blades and no moving parts, for sealed enclosures, transceiver cages and the residual air paths of liquid-cooled hardware. Book a technical demo.
No. Direct-to-chip cold plates cover the CPUs and GPUs. NVIDIA's DGX GB200 documentation states that the remaining components, such as networking and storage devices, are air cooled by fans, and that the power shelf uses six air-cooled 5.5 kW power supplies. A liquid-cooled rack is a hybrid rack.
Use rack density as the trigger. Below 15 kW, conventional air with good containment is enough. Between 30 and 60 kW, a rear-door heat exchanger is the usual retrofit. Above 60 kW, direct-to-chip liquid becomes the practical answer, and above 100 kW it is assumed.
One peer-reviewed testbed measured a heat capture ratio of 94 per cent on a 53 kW rack and 93 per cent across a 128 kW multi-rack test, with the balance leaving in air. The real figure depends on how many components sit under a cold plate, and vendors draw that line differently.
No. Microfluidic cooling puts coolant channels into or immediately onto the silicon, so it competes with the cold plate layer. It does nothing for DIMMs, voltage regulators, NICs, optical modules or the power shelf, which stay in the air path.
In the residual air path, not on the accelerator. They target the boundary layer in constrained, low-flow volumes such as transceiver cages, rear-of-rack pockets and sealed enclosures, where a rotating fan does not fit and a liquid loop is not permitted.