Home › Guides › Hardware, power and cost

Cooling GPU Servers: Air vs Liquid, With the Heat Math Worked Out

Updated October 21, 2026 · By the Xingyu Zhitong editors

Every watt a GPU server draws ends up as heat in the room. That sounds obvious, but its consequences are not: a single eight-GPU node can dissipate as much heat as several household space heaters, and a rack of them can exceed what a conventional office or small server room can remove. Cooling is therefore a hard design constraint, not an afterthought. This article does the heat arithmetic, compares air and liquid approaches, and lists what to check before a GPU server arrives.

Step 1: Know the heat load

Electrical power in equals heat out, almost exactly, because the useful work of computation leaves the building as information rather than energy. NVIDIA lists the H100 SXM at up to 700 W, configurable, and the H200 SXM likewise; the PCIe-form-factor NVL variants run at lower limits, 350 to 400 W for H100 NVL and up to 600 W for H200 NVL. For a whole system, NVIDIA lists about 14.3 kW maximum for the DGX B200 with eight Blackwell GPUs. Use the system maximum from the vendor's data sheet for sizing, and measured averages for operating cost; the reasoning behind that split is in PUE and electricity cost.

Useful conversions, which are standard unit relations:

• 1 watt = 3.412 BTU per hour.
• 1 "ton" of cooling = 12,000 BTU per hour = about 3.517 kW.

Example, using an assumed 10 kW node: 10,000 W × 3.412 = 34,120 BTU/h, or 2.84 tons of cooling capacity, and that is for one server. Four such servers in a rack is 40 kW, more than ten tons, concentrated in a footprint of under one square meter.

Step 2: How much air does it take?

Air carries heat in proportion to its flow rate and the temperature rise it undergoes passing through the equipment. A common engineering approximation at sea level is:

airflow (CFM) ≈ 3.16 × power (W) ÷ temperature rise (°F)

For the 10 kW example with a 20 °F (about 11 °C) rise from inlet to exhaust: 3.16 × 10,000 ÷ 20 = 1,580 CFM, roughly 2,680 cubic meters per hour, for one server. If the allowed rise is only 10 °F, the airflow doubles. The numbers explain why GPU servers have powerful fans that are loud and consume real power (fan energy is part of the IT load and therefore of your bill). They also explain why mixing hot exhaust back into server inlets is so damaging: the inlet temperature goes up, the allowed rise goes down, fans speed up, and the cycle worsens. Hot-aisle/cold-aisle layouts, blanking panels in empty rack slots, and containment exist to keep supply and exhaust air separate.

Facilities also have limits at the room level. The air conditioning system must remove the total heat, and the supply air must physically reach the racks. Floor-tile airflow, ducts and the capacity of each cooling unit set a ceiling on kW per rack that is often well below a dense GPU configuration. When a facility quotes a maximum density per rack, believe it; exceeding it causes hot spots even if the overall room capacity looks adequate.

Operating temperature guidance for IT equipment is published by ASHRAE's technical committee on mission-critical facilities; the exact recommended and allowable ranges change between editions, so check the current edition and, more importantly, the specific equipment's data sheet, which states the inlet temperature range the vendor supports.

Step 3: How much coolant does it take?

Water carries far more heat per unit volume than air. With a specific heat of about 4.186 kJ/(kg·K), the flow needed is mass flow = power ÷ (4,186 × temperature rise). For 10 kW and a 10 K coolant temperature rise: 10,000 ÷ (4,186 × 10) = 0.239 kg/s, about 14.3 liters per minute. That is a garden-hose-scale flow for a load that needed over 1,500 CFM of air. Real coolants may be water-glycol mixtures with somewhat lower specific heat, and system designs specify flow rates per cold plate; use the vendor's figures. The comparison shows why liquid is attractive: the same heat moves through a few small tubes instead of a large volume of air.

Cooling architectures

1. Room air cooling (CRAC/CRAH units). The conventional approach. Works well to moderate densities; cheap to operate at small scale; limited by airflow physics at high density.
2. Rear-door heat exchangers. A liquid-cooled radiator replaces the rack's rear door and removes much of the exhaust heat before it enters the room. Servers stay air-cooled internally, so this is a retrofit-friendly step up in density.
3. Direct-to-chip liquid cooling. Cold plates sit on the GPUs and CPUs and carry heat into a coolant loop; fans still cool the remaining components such as memory, drives and power supplies, with far less airflow. NVIDIA and server vendors offer liquid-cooled variants of their GPU platforms; availability depends on the product, so check the vendor's data sheet for the exact configuration.
4. Immersion cooling. Servers sit in a dielectric fluid bath. It removes the fans entirely but changes maintenance, warranty and hardware compatibility, so it is a specialized choice.

Air versus liquid: practical trade-offs

Liquid cooling is not automatically cheaper to run. Its benefit depends on whether the heat can be rejected efficiently outdoors, for example with dry coolers in a mild climate, rather than through chillers. It also does not remove the need to supply power; see power supply sizing for the single-machine version of that question.

Worked comparison of airflow demand

Suppose you plan 4 nodes at 10 kW each (an assumption). Air at a 20 °F rise needs 4 × 1,580 = 6,320 CFM through the rack, and the room must remove 40 kW or 11.4 tons. With direct-to-chip liquid cooling capturing, say, 70 percent of node heat in the liquid loop (an assumed split; the real fraction depends on the product), the coolant carries 28 kW and the air must remove 12 kW, cutting the airflow need to roughly 30 percent of the all-air case, about 1,900 CFM. The liquid loop at a 10 K rise would flow about 28,000 ÷ (4,186 × 10) × 60 ≈ 40 liters per minute for the rack. These are planning figures, not design values.

Reading throttle behavior

Cooling problems rarely announce themselves as errors. They appear as lower clocks. NVIDIA's monitoring tools report clock throttle reasons, including thermal slowdown and power capping, and a job whose throughput drifts down over the first ten minutes and then plateaus is a typical signature of a thermally limited system. Compare the same benchmark at the start of the day, when the room is cool, and in the afternoon, when it is not. Differences of ten percent or more mean the cooling envelope is already being used up.

Also watch for uneven behavior across GPUs in one chassis. In many air-cooled servers the GPUs at the back of the airflow path receive pre-warmed air and run hotter than those at the front. For tensor-parallel work, where every GPU must finish each layer before the next begins (see tensor versus pipeline parallelism), the slowest and hottest GPU sets the pace for all of them, so one throttled card drags down the whole node. Fixing airflow for the worst GPU is therefore worth more than improving the average.

Small and home setups

The same physics applies to workstations. A dual-GPU tower with two 450 W cards plus a CPU may dump over a kilowatt into a room, and the room needs ventilation. For tower GPUs, case airflow (front-to-back, with positive pressure through filtered intakes), spacing cards so the intake of one is not starved by the other, and blower-style or open-air coolers chosen to match case layout matter. If an enclosure heats up in a closet, thermal throttling lowers performance, which on an LLM workload means lower tokens per second without any visible error. The same problem occurs at the edge, as discussed in Jetson power and battery runtime.

Pre-delivery checklist

1. Get the vendor's maximum power, maximum inlet temperature, required airflow in CFM or coolant flow, and weight.
2. Confirm the room or colocation space supports that kW per rack with its cooling, not only total kW.
3. Check electrical circuits and power distribution units for continuous load; involve a licensed electrician.
4. Plan hot/cold separation, blanking panels and cable management that does not block airflow.
5. For liquid cooling, plan leak detection, coolant specification, filtration, maintenance access and who is responsible for the facility loop.
6. Monitor: GPU temperatures, inlet temperature, fan speeds and throttle reasons via the vendor's tools. A GPU running at its thermal limit under steady load is telling you cooling is inadequate.
7. Re-run your cost estimate with realistic overhead using the HGX electricity cost calculator.

Cooling designs, water systems and electrical installations carry safety and code requirements that vary by location. This article is general information; use qualified mechanical and electrical engineers for real installations. The decision between renting and owning (see renting versus buying) often turns on whether you can host the heat at all.

Sources

• NVIDIA H100 product page
• NVIDIA H200 product page
• NVIDIA DGX B200 page
• The Green Grid
• Uptime Institute Global Data Center Survey 2024
• ASHRAE (thermal guidelines for data processing environments)

Figures are estimates based on published model configurations and vendor specifications. Real memory use, speed and power draw depend on the software stack, drivers, settings and workload, so test on your own hardware before you buy. Spotted a mistake? Email huhuwen09@gmail.com with the page address and we will check and correct it.