Why this is a break-even question and not a price comparison
A managed API charges you per token. A GPU charges you per hour whether or not a token is moving. Those are different shapes of cost, so there is no single answer to "which is cheaper" — there is only a volume above which the fixed bill wins and below which it loses. This calculator finds that volume and then tells you where you sit relative to it.
The comparison people usually make is a price comparison: they divide a GPU's hourly rate by its peak throughput, get a cost per million tokens that looks spectacular next to an API rate, and conclude that self-hosting is several times cheaper. The arithmetic is right and the conclusion is usually wrong, because peak throughput is a benchmark number and your deployment does not run at peak. It runs at whatever fraction of the month your traffic actually keeps it busy.
That fraction — utilisation — is the term that decides the answer, and it is the one most often left out. Divide by 35% instead of 100% and your effective cost per million tokens triples. A deployment that looked five times cheaper than the API is now marginally cheaper, and after storage, load balancing, monitoring and the engineer who gets paged at three in the morning, it may not be cheaper at all.
So this calculator asks for utilisation explicitly and refuses to let you skip it. It also asks for platform and operations cost, because a GPU hourly rate is not a total cost of ownership, and comparing a raw GPU rate against a fully-managed API rate compares two different products.
The three formulas behind the answer
Monthly self-hosted cost. The hourly rate times the number of GPUs times 730 hours, plus whatever else you pay to run the platform. 730 is the average month: 8,760 hours in a year divided by 12. Use it rather than 720, which assumes every month is 30 days.
Self-hosted = h × g × 730 + O
Monthly capacity. Sustained throughput in tokens per second, times 3,600 seconds an hour, times 730 hours, times utilisation. Throughput here means what the deployment delivers under your real batching and your real prompt lengths, not the number in a vendor benchmark run at batch 256 with a 128-token prompt. If you do not have a measured figure, estimate one with the throughput and latency calculator and then discount it.
Capacity = T × 3,600 × 730 × u
Break-even volume. Set the two bills equal. The API bill at volume N is N × R ÷ 1,000,000. Solve for the N that makes it match the fixed bill:
N* = (h × g × 730 + O) ÷ (R ÷ 1,000,000)
Two properties of that expression matter. It does not contain throughput or utilisation at all — break-even in tokens depends only on the fixed bill and the API price. Throughput and utilisation enter one step later, as the constraint that decides whether your hardware can actually serve N* tokens. That is the check most spreadsheets miss: a break-even volume your deployment physically cannot reach is not a break-even at all, and the calculator warns when you are in that regime.
The other property is that break-even scales linearly with the fixed bill. Add a second replica for redundancy and you double the volume you need to justify it. Redundancy is a real requirement, but it is bought out of the same budget as the comparison.
Worked example: two GPUs at $2.50 an hour against a $3.00 API
You are serving an open-weights model on two rented GPUs at $2.50 per GPU-hour. Measured on your own traffic the pair sustains 2,500 output tokens per second. Your traffic is business-hours heavy, so you hold about 35% utilisation across the month. The managed API you would otherwise use blends to $3.00 per million tokens, and you currently serve 500 million tokens a month.
- Fixed monthly bill. $2.50 × 2 GPUs × 730 hours = $3,650.
- Monthly capacity. 2,500 × 3,600 = 9,000,000 tokens per hour. × 730 hours = 6,570,000,000 at full occupancy. × 0.35 = 2,299,500,000 tokens a month.
- Effective cost per million. $3,650 ÷ 2,299.5 million = $1.5873 per million tokens. Against $3.00 on the API, that is a little under half the unit price — but only if you fill the capacity.
- API cost per token. $3.00 ÷ 1,000,000 = $0.000003.
- Break-even volume. $3,650 ÷ $0.000003 = 1,216,666,667 tokens a month, about 1.22 billion.
- Your position. At 500 million tokens the API costs 500,000,000 × $0.000003 = $1,500, against $3,650 for the GPUs. Self-hosting costs $2,150 more per month at today's volume.
You need roughly 2.4 times your current volume before the hardware pays for itself (1,216.7 ÷ 500 = 2.43). The capacity check passes — 1.22 billion is comfortably under the 2.30 billion the deployment can serve at 35% — so the break-even is reachable rather than theoretical. If your utilisation were 15% instead of 35%, capacity would be 6,570 × 0.15 = 985.5 million, which is below the 1,216.7 million break-even, and no amount of demand would make this configuration beat the API without adding hardware or raising occupancy.
How to read the result
Compare your volume with the break-even, then check capacity. Being above break-even is necessary; being able to serve the traffic is also necessary. The calculator flags the case where break-even sits above capacity, which means the configuration cannot win on cost at any demand level. The fix is always one of three things: more GPUs, higher sustained throughput per GPU, or higher utilisation.
Treat the effective cost per million as the honest unit price. It is the number to put next to an API rate, because it already carries the utilisation penalty. A raw peak-throughput figure is a hardware specification, not a price.
Ask how much headroom the answer has. If self-hosting wins by 10%, it does not really win: the API side of the comparison has fallen steadily as models get cheaper, while your GPU rate is fixed by a contract or a purchase. A margin that thin is consumed by one price change or one month of low traffic. If it wins by three or four times, the decision is robust to both.
Remember what the comparison excludes on both sides. Self-hosting excludes the API's rate limiting, autoscaling, safety tooling, uptime commitments and the fact that nobody on your team has to be on call for it. The API excludes your ability to run a fine-tuned or gated model, to keep data inside your own network, and to control latency at the tail. Those are the reasons teams self-host that have nothing to do with the number on this page, and they are frequently the real ones.
Two adjacent levers usually beat the hosting decision outright on effort-to-saving. Caching a long static prefix costs an afternoon; size it with the prompt caching savings calculator. Cutting context length reduces both the API bill and the memory pressure on a self-hosted deployment; the KV cache size calculator shows how much VRAM a long context is consuming, which is often what caps your batch size and therefore your throughput.
Effective self-hosted cost per million tokens by utilisation
| Utilisation | Tokens served per month | Effective cost per 1M | Versus a $3.00 API rate |
|---|---|---|---|
| 10% | 657 M | $5.5556 | 1.85× dearer |
| 25% | 1,642.5 M | $2.2222 | 0.74× the API |
| 35% | 2,299.5 M | $1.5873 | 0.53× the API |
| 50% | 3,285 M | $1.1111 | 0.37× the API |
| 75% | 4,927.5 M | $0.7407 | 0.25× the API |
| 100% | 6,570 M | $0.5556 | 0.19× the API |
The unit cost is inversely proportional to utilisation, so halving occupancy doubles the price per million. The 10% row is the one that turns a confident spreadsheet into a more expensive bill than the API it replaced.
Utilisation is not the same as uptime
A deployment can be up 100% of the month and utilised 20% of it. Uptime measures whether the service responds; utilisation measures whether the GPUs are generating tokens. Interactive traffic follows working hours and time zones, which means a deployment sized for the peak hour is idle for much of the night, and the hourly meter runs through all of it. Measure utilisation as tokens actually generated divided by tokens the hardware could have generated in the same period. If you are estimating rather than measuring, and your traffic is a single-region business-hours product, estimate low.
Assumptions and limits of this comparison
- The GPUs are held continuously. A persistent serving deployment pays for 730 hours a month. If you can genuinely scale to zero overnight, model that as a lower GPU count or lower hours, not as higher utilisation.
- Throughput is a single sustained number. Real throughput varies with prompt length, batch size and how much of the context is cached. The figure you enter should come from your own load test at your own prompt distribution.
- Input and output tokens are treated alike. Prefill is compute-bound and decode is memory-bound, so a prompt-heavy workload behaves quite differently from a generation-heavy one. If your ratio is extreme, model the two phases separately.
- The models are assumed comparable. A smaller open model at half the cost is only a saving if it does the job. Establish quality first, then compare prices.
- No commitment discounts are modelled. Reserved or committed-use pricing lowers the hourly rate and raises the switching cost. Enter your contracted rate rather than the on-demand one if you have signed for capacity.
- Nothing here prices the engineering. Building and running an inference platform consumes people. Put a realistic monthly figure in the operations field, because leaving it at zero systematically tilts the answer.
What to check before and after this calculation
Before you compare costs, size the deployment properly. A model that does not fit in the VRAM you have does not have a throughput number at all, and one that fits only without a KV cache will collapse at your real context length. The GPU VRAM requirement calculator gives the weights and activation footprint, and the KV cache calculator gives the term that grows with context and batch. Both feed the GPU count you enter here.
Quantisation changes the comparison materially, because it lets a model fit on fewer or cheaper GPUs and raises tokens per second on a memory-bound decode. The quantisation memory savings calculator shows the footprint at each precision; a model that drops from two GPUs to one halves the fixed bill and therefore halves the break-even volume.
After the calculation, run the sensitivity rather than the point estimate. Recompute with utilisation at half your assumption, with the API rate 30% lower than today, and with one extra GPU for redundancy. If self-hosting still wins under all three, the decision is sound. If it only wins under your best case, you are betting on your best case.
Finally, keep the unit economics in view. A cost per million tokens is an infrastructure number; what a chief financial officer asks about is cost per user. Feed the effective rate from this page into the cost per user per month calculator to see whether the hosting decision actually moves gross margin, or whether it moves a line that was never the constraint.