Why own the box

Renting inference is easy, elastic, and someone else’s problem when it breaks. So why put a machine on your desk and take on the maintenance?

The usual answer is cost, and cost is the weakest of the real arguments. Here are the ones that hold up, and the ones that do not.

The meter changes what you build

This is the argument that gets the least attention and matters the most.

When every call costs money, you price each experiment before you run it. You batch things you would rather iterate on. You skip the sweep across twelve prompt variants because eleven of them will be wasted spend. You think twice before pointing a model at a directory of ten thousand files just to see what falls out. None of that shows up on an invoice, because the invoice only records what you did run.

A box you already own removes the meter from the thinking. The marginal cost of the next experiment is electricity, and electricity is cheap enough to ignore at this scale. That changes which experiments you attempt, and the ones you attempt when nobody is counting are frequently the ones that teach you something.

It is the same reason people write more when they stop paying by the page.

Some data cannot leave

Client files under NDA. Patient data. A vendor’s drawings that arrived with a confidentiality clause. Source code for something you have not shipped.

For this category the cloud question is not about price or quality, it is a policy question with a binary answer, and the answer is often no. A local box turns “we cannot use a model for this” into “we can,” which is a larger capability gain than any benchmark difference between model families.

The related benefit is that you stop having to reason about it. There is no data-handling addendum to read, no retention window to verify, no question about whether this quarter’s terms changed. The bytes never left the room.

The model you validated against stays put

Hosted models get deprecated, silently updated, and retired on someone else’s schedule. If you built a pipeline whose prompts were tuned against a specific model, you are renting the ground you are standing on.

A local weights file does not change unless you change it. When you validate something, the validation keeps meaning what it meant. For anything you intend to still be running in three years, this is worth more than a few points on a leaderboard.

What this box is actually good at

The unified memory design is what makes the local case practical here rather than aspirational. A single 128 GB coherent pool holds models that no single consumer card can, so the question shifts from “will it fit” to “how fast does it run,” which is a much better question to be arguing about.

The honest shape of it: this hardware holds large models and runs them at moderate speed. For iteration, prototyping, private data, and personal-scale serving, that is the right trade. See serving a local LLM for what running it looks like in practice.

The costs nobody puts in the brochure

You are now the ops team. Driver updates, disk pressure, a service that died at 3am, a model that will not load after a CUDA bump. Renting includes an on-call rotation you never see. Owning does not.

Capital is committed up front. Rented inference scales down to zero when you stop using it. A box does not. If your usage is genuinely bursty and occasional, renting is cheaper and it is not close.

The hardware ages against a moving target. Model architectures shift, quantization formats improve, and a box bought today runs today’s models well and next year’s models adequately. This is survivable, but it is real, and it is the reason to buy for the work you have now rather than the work you imagine.

Bandwidth, not capacity, is the ceiling. Token generation is memory-bandwidth-bound. Holding a very large model is not the same as serving it quickly to several people at once.

Computing your own breakeven

Breakeven arithmetic gets quoted with confident numbers that are stale within a quarter, so here is the method instead of the answer.

Add the purchase price to the electricity it will draw over the period you care about, at your utility rate. Divide by the number of months. That is your monthly cost of owning, and it does not vary with use.

Against it, put what you would otherwise spend renting: your actual token volume at current prices, not your imagined volume. Pull the real number from a month of invoices if you have them.

Two adjustments matter more than the raw comparison. First, add the value of the work you currently do not do because the meter is running, which is the hardest term to estimate and often the largest. Second, add the value of the work you cannot do at all for policy reasons, which for some people is the entire justification on its own.

If those two terms are near zero for you, rent. That is a legitimate answer and the honest one for a lot of workloads.

Where the line falls

Own the box when your data has rules attached, when you want to stop pricing your own curiosity, or when you need something that will still behave identically in three years.

Rent when your usage is spiky, when you need frontier-scale models you cannot host, or when you would rather someone else carry the pager.

Most people who own one end up doing both, which is the sensible outcome: the box handles the private, the iterative, and the routine, and the rented frontier models handle the occasional problem that genuinely needs them.