The spreadsheet says cloud is three times more expensive than owning your GPUs. The spreadsheet is lying — on both sides. Cloud quotes omit egress, idle waste, and the premium baked into every GPU-hour. Ownership quotes omit staffing, refresh cycles, and the depreciation cliff that arrives with each new chip generation. In 2026, with AI infrastructure budgets under real scrutiny for the first time, the build-versus-colo-versus-cloud decision deserves honest arithmetic. Here it is.
The 8-GPU math, three ways
Take the standard building block of AI infrastructure: a single 8×H100 SXM5 node, the workhorse of training and large-scale inference. A DGX-class node costs $250,000–$350,000 to buy. Now run it three ways over three years at full utilization.
GPU cloud on-demand — AWS, Azure, GCP list pricing — runs roughly $20,000 per month for this class of capacity. Three-year total: about $720,000. Committed 3-year reserved pricing cuts that to roughly $12,000 per month, or $432,000 over the term. Colocation with owned hardware: $280,000 in CapEx plus about $3,500 per month in colo and power, totaling roughly $406,000 over three years — and the hardware retains an estimated $60,000–$80,000 in residual value, bringing the effective cost to approximately $326,000–$346,000.
The headline: at 100% utilization, colocation saves about 52% versus on-demand cloud and roughly 20% versus 3-year reserved. Even at 70% utilization, owned hardware in colo still beats on-demand. The crossover — where cloud becomes cheaper — sits around 40–50% utilization. Below that, the flexibility premium is worth paying; above it, you are renting your own inefficiency.
Stretch the horizon to five years, the standard enterprise TCO window, and the gap widens further. Lenovo's 2026 generative-AI TCO analysis, comparing on-premises infrastructure against hyperscaler list rates as of July 2026, puts an 8×H100 configuration at $4,306,416 for cloud on-demand over five years, $2,362,811 for 3-year reserved, and $871,912 for on-premises — before staffing. Even after adding a staffing delta of $300,000–$900,000 over the period, on-premises wins decisively for sustained 24/7 workloads. Note the methodology's honesty: Lenovo excluded storage, egress fees, and support plans from the cloud side — the most charitable possible cloud comparison — and on-prem still won by multiples.
A smaller-scale illustration makes the same point. A 2×H100 inference server — $50,000 of hardware amortized over 36 months ($1,389/month), power at $0.12/kWh ($121/month), a colo rack allocation ($900/month), and 0.15 FTE of engineer time ($1,875/month) — totals roughly $4,285 per month all-in. The cloud equivalents: CoreWeave reserved around $3,285/month for the same GPUs, AWS on-demand an order of magnitude more (with the caveat that AWS's smallest H100 instance carries 8 GPUs — itself a pricing-structure lesson). For steady inference, the owned box wins; for bursty experimentation, the API wins. The workload decides, not the philosophy.
The costs both sides leave out
Cloud's hidden ledger starts with egress. AWS charges $0.09 per GB for data transfer out beyond the free tier; at enterprise AI scale — training datasets in, model checkpoints out, inference traffic flowing — egress routinely adds 15–30% to the total AI bill, and it is the most commonly omitted line in cloud comparisons. Then the GPU premium: cloud providers charge 2–3x wholesale GPU rates on every GPU-hour, indefinitely — you never build equity, and the meter never stops. Then idle waste: reserved capacity you do not use is the most expensive kind. ICONIQ's 2026 State of AI report found inference costs averaging 23% of revenue at scaling-stage AI companies — a line item large enough to decide the infrastructure question on its own.
Ownership's hidden ledger starts with people. H100 clusters demand specialist skills — InfiniBand networking, liquid-cooling operations, firmware lifecycle management — commanding 30–50% salary premiums over general ML engineers. The realistic staffing delta for on-premises inference is 0.5–1.5 FTE ($60,000–$180,000 per year at $120,000 fully loaded); omitting it is the single most common error in on-prem business cases. No in-house capability? Budget $15,000–$40,000 per month for a managed service provider for a cluster of meaningful size. Then the refresh cycle: GPU servers have a 3–5 year economic lifespan, and refresh adds roughly 20–30% to the five-year on-prem cost. Then the operational realities the spreadsheet never captures: 3–6 month hardware lead times, NVIDIA enterprise licensing at about $2,500 per GPU per year, 10% annual maintenance on hardware cost, facility readiness at roughly $12,000 per H100 node.
Colocation sits between the two, and its economics are the most transparent: you buy the hardware once and rent the power, space, and cooling monthly. The colo operator absorbs the facility CapEx — the generators, the chillers, the building — and you keep the compute equity. For the utilization band between roughly 50% and 90%, it is usually the cheapest honest answer.
The decision rule that actually works
Forget tribal affiliation. The decision reduces to four questions. First, utilization: sustained above 50%? Own the hardware. Spiky or experimental? Rent the cloud. Second, duration: a 3–5 year horizon favors ownership; comparing over 12 months always favors cloud and always misleads. Third, staffing: do you have — or can you buy — the operational capability? If not, price the managed service into the colo column before comparing. Fourth, data gravity and compliance: GDPR, HIPAA, SOC 2, and sector AI-governance frameworks often settle the question before cost is considered, and sub-10ms inference latency requirements reliably favor colocated bare metal over multi-tenant cloud instances.
The sophisticated answer, and the one most enterprises are converging on, is hybrid by workload: own the base load in colocation, burst to cloud for peaks, and keep training where the data lives. Cloud is a pricing model. Colocation is a real-estate decision. Owning hardware is a commitment. Match the instrument to the utilization curve — and re-run the math every renewal cycle, because the only constant in this market is that last year's answer has expired.
The company that becomes the pricing authority for AI infrastructure will own the most valuable search traffic in the sector. Datacenterprices.com is available for acquisition by that company — inquiries to info@primedomaincapital.com.