Home › Guides › Hardware, power and cost

Renting Cloud GPUs vs Buying Your Own: A Break-Even Method

Updated October 21, 2026 · By the Xingyu Zhitong editors

"Renting is cheaper until it is not" is the usual summary of the cloud versus on-premises GPU decision, and it is true but useless without numbers. The break-even point depends on a handful of inputs, and the most important one is not any price. It is utilization: the fraction of hours in which the hardware does paid or useful work. This article gives a method you can apply with your own quotes, and a worked example in which every price is a labeled assumption you should replace.

A note on prices before we start: GPU rental rates and hardware street prices change quickly and differ by provider, region, commitment term and availability. This article therefore never states a current market price as fact. All dollar amounts below are "example assumptions" chosen to make the arithmetic easy to follow.

Step 1: Count the cost of owning

Ownership cost has more lines than the sticker price of the server. A complete list:

• Capital cost: the GPU server or workstation, plus networking, storage, rack hardware and spares.
• Electricity: IT load multiplied by facility overhead (PUE, explained in our PUE guide), times hours, times your tariff.
• Space, power and cooling provisioning: colocation fees, or the cost of upgrading your own room. High-density GPU nodes may need circuits and cooling that office space lacks (see air versus liquid cooling).
• Support and maintenance: vendor support contracts, replacement parts, and staff time for firmware, drivers and failures.
• Cost of capital and depreciation: money spent on hardware cannot be spent elsewhere, and GPUs lose resale value as newer generations ship.
• Idle time: hours when the machine is on but not productive still cost power and depreciation.

On the rental side, the list is shorter but has its own traps: hourly rate, storage and snapshot fees, data egress charges, minimum commitments, and the time your team spends re-provisioning environments when instances are reclaimed.

Step 2: Convert ownership into a cost per hour

Take the total cost of ownership over the period you expect to keep the hardware, subtract any resale value you can honestly expect, and divide by the hours the hardware is actually used. The key formula:

effective cost per used hour = (capital + operating costs over the period − resale value) / (hours in period × utilization)

The denominator is what surprises people. A node that is powered on for three years has about 26,280 hours. If it does useful work 30 percent of the time, you are paying for 26,280 hours but only using about 7,884.

Worked example (all numbers are example assumptions)

Consider a hypothetical eight-GPU server with these assumptions:

• Purchase price including networking and spares: $300,000 (example assumption).
• Average IT power draw under load: 10 kW (example assumption; NVIDIA lists about 14.3 kW maximum for a full DGX B200, and node configurations vary).
• PUE of the hosting facility: 1.4 (example assumption).
• Electricity price: $0.12 per kWh (example assumption).
• Support, space and staff overhead: $30,000 per year (example assumption).
• Planning horizon: 3 years; resale value at the end: $60,000 (example assumption).

Electricity per year at continuous load: 10 kW × 1.4 × 8,760 h = 122,640 kWh; at $0.12 that is $14,717 per year, or about $44,150 over three years. Support and overhead over three years: $90,000. Total three-year cost: 300,000 + 44,150 + 90,000 − 60,000 = $374,150.

Hours in three years: 26,280. Per hour of wall-clock time, ownership costs 374,150 / 26,280 ≈ $14.24 for the entire node, or about $1.78 per GPU-hour if utilization were 100 percent. At 50 percent utilization the cost per used GPU-hour doubles to about $3.56; at 25 percent it quadruples to about $7.12. (Electricity falls somewhat when idle, which this simple model ignores; the effect is small compared with depreciation and overhead.)

Now assume a hypothetical rental price of $3.00 per GPU-hour (again an example assumption, not a quote). Under these inputs, ownership wins above roughly 59 percent utilization (1.78 / 3.00) and loses below it. Change the rental rate to $2.00 and the break-even moves to 89 percent; change it to $5.00 and it falls to 36 percent. The method matters more than the answer: your threshold will be different, and you can compute it with the cloud versus on-prem ROI calculator.

Step 3: Estimate utilization honestly

Most failed purchases fail here. Utilization is not "the hours someone has a job queued". It is hours of useful compute divided by hours available. Common patterns:

• Steady inference serving with predictable traffic can keep GPUs busy around the clock, but remember peak-to-average ratios. If traffic peaks at three times its average, you either provision for the peak (and idle in the troughs) or accept degraded latency.
• Research and experimentation is bursty. Weeks of heavy training are followed by weeks of analysis. Averaged over a year, utilization is often far below what the team expects.
• Training runs with deadlines favor renting, since you can scale to many GPUs for a short time and release them.

Look at your last three months of actual usage, if you have it, rather than your intentions. If you have no history, rent first; the rental bills become the dataset that tells you whether to buy.

Step 4: Adjust for things the formula leaves out

1. Technology risk. Each hardware generation shifts what is possible. Memory capacity jumps (see A100, H100, H200 and B200 compared) can make an older card unable to hold the model you want in two years. Rented capacity lets you move to new hardware when it arrives; owned hardware does not.
2. Lead time and availability. Buying can take weeks to months; renting takes minutes when capacity exists, but popular GPU types may be unavailable at peak times.
3. Data gravity and egress. If your data already lives in a cloud, moving it costs time and money in either direction. If it lives on-premises, pulling it to the cloud for every run has a cost too.
4. Staff and risk. Operating GPU clusters (firmware, drivers, failed GPUs, network tuning) is a skill. If nobody on the team has time, the support line in your model should be higher.
5. Commitment discounts. Many providers offer lower rates for reserved capacity. A one- or three-year reservation behaves like a partial purchase, with a lower hourly rate in exchange for a utilization guarantee. Include it as a third column in your comparison.

Small-scale version: a single workstation GPU

The same logic applies at a smaller scale. A consumer GPU such as the RTX 4090 (24 GB, 450 W board power per NVIDIA) is attractive for experimentation with models up to roughly the 7B to 30B range with quantization; see running 7B-class models on one GPU. The capital cost is lower, there are no hosting fees if it sits under your desk, and electricity is a few cents per hour. But the card cannot hold large models (see running 70B models locally) and lacks the data-center features that make multi-GPU scaling efficient (see NVLink versus PCIe). A common pragmatic pattern is to own one workstation GPU for daily development and rent data-center GPUs for the occasional large job.

Why utilization estimates are usually too high

Teams tend to forecast utilization from the best week they remember. Typical gaps between forecast and reality include: waiting for data to be prepared, failed runs that must be repeated, holidays, the gap between finishing one project and starting the next, and the simple fact that people work during the day while GPUs could run overnight only if jobs are queued in advance. A scheduler and a queue of low-priority jobs can fill nights and weekends, which raises utilization cheaply, but only if someone maintains the queue. Include that effort as a cost, or assume lower utilization.

Utilization also has a quality dimension. A GPU showing 100 percent "utilization" in monitoring tools may be mostly waiting on memory or on another GPU. Efficiency of the workload (see tokens per second) matters as much as the busy percentage when you convert hours into useful output.

A sensitivity checklist before you sign

• Recompute break-even with utilization 20 points lower than your estimate. If the decision flips, the purchase is fragile.
• Recompute with rental rates 30 percent lower, since rental pricing has historically trended down as supply grows. That is an observation about the past, not a forecast, so test both directions.
• Recompute with resale value at zero. If ownership still wins, the downside is bounded.
• Check that the facility can actually deliver the power and cooling. A GPU server that cannot run at full load because of circuit or thermal limits changes every number above.
• Ask for a written quote for support and spares rather than guessing.

The short version

Compute your effective cost per used GPU-hour from the full ownership list, divide by honest utilization, and compare it to rental quotes you actually hold, including egress and storage. If the break-even utilization is higher than what you have measured, rent. If it is lower with a comfortable margin, and you have the power, cooling and people to operate it, buying starts to make sense. In between, reservations or a hybrid approach usually beat either extreme.

This article is general planning information, not financial advice. Tax treatment of hardware and leasing varies by jurisdiction; ask an accountant for the specifics of your situation.

Sources

• NVIDIA DGX B200 page (system power figure)
• NVIDIA GeForce RTX 4090 page
• U.S. EIA Electric Power Monthly, average price of electricity by sector and state
• Uptime Institute Global Data Center Survey 2024

Figures are estimates based on published model configurations and vendor specifications. Real memory use, speed and power draw depend on the software stack, drivers, settings and workload, so test on your own hardware before you buy. Spotted a mistake? Email huhuwen09@gmail.com with the page address and we will check and correct it.