Build vs buy: the honest math on running your own AI

Every week a CTO asks me some version of the same question: is it cheaper to run our own model or keep paying per token? The honest answer is that it depends on numbers most spreadsheets get wrong in both directions. This post walks through the real ones: hardware, power, people, break-even volumes, and the costs that never show up until something leaks.

What the API really costs

Per-token pricing looks tiny until you multiply it by real volume. One vendor-side analysis found a retrieval pipeline costing about 2,275 dollars per month on a frontier closed API ran about 168 dollars per month on an optimized open-weight model, a 93 percent reduction (Accrets). The same class of analysis claims up to an 18x per-token cost advantage for owned hardware at sustained high utilization. Treat vendor numbers as the optimistic end, but the direction is real.

What self-hosting really costs

The hardware specifics, and what each tier actually runs, are in the hardware and serving stack in detail.

Where break-even actually sits

Published estimates disagree, and it is worth understanding why. Against premium APIs, break-even lands around 5 to 10 million tokens per month; against budget APIs it moves to 50 to 100 million (AI Pricing Master). Conservative independent analyses put DIY break-even far higher once you price staff honestly (Zander), while vendor TCO studies claim payback in under four months at high utilization (Lenovo).

The spread comes from three assumptions: utilization (an idle GPU earns nothing), staff cost (the biggest hidden line), and model class (a 8B model and a 400B MoE have very different economics). Run the numbers with your real volumes, not the volumes in the pitch deck.

The line the spreadsheet always misses

Breaches involving shadow AI cost about 670,000 dollars more than the average breach, putting them around 4.63 million dollars per incident (Verax, citing IBM's Cost of a Data Breach 2025). Meanwhile 77 percent of employees paste data into generative AI tools, and 82 percent of that goes through personal accounts (Technology Radius). GDPR fines reach 20 million euros or 4 percent of global turnover, and since August 2, 2026 the EU AI Act has live market surveillance on top.

This is why I think data location decides this question more often than price does. Once you put a realistic number on one leaked dataset, the per-token comparison stops being the decision.

The third option nobody prices: buy the box

Build versus buy is a false binary. The third option is an appliance: someone else does hardware selection, serving, updates, and evaluations, and the data still never leaves your building. That is where Kaldryn One sits.

And to be honest about when an appliance is the wrong choice: very low volumes, purely public data, or a team that genuinely wants to own the stack as a core competence. In those cases, use an API or build it yourself. The point is to decide with numbers.

My rule of thumb

1. Under a few million tokens a month and public data: use an API.
2. Steady volume and sensitive data: run it yourself or buy an appliance.
3. No in-house ML operations capacity: appliance over DIY, every time.
4. Hard latency or offline requirements: on-premises wins by default.
5. Regulated sector: price the breach and the fine, then reread rule 2.

Do the token math with your real volumes. Price the engineer hours honestly. Then put a number on the thing the spreadsheet ignores. If you want a second pair of eyes on that calculation, my inbox is open.