Private LLM: What Self-Hosting Costs, and When It Is Worth It

TL;DR: A private LLM keeps prompts out of anyone else's training data. Three routes exist: self-host an open model, rent a dedicated GPU, or use a platform's private mode. Perspective AI is the third, from $14.99/mo, with no self-hosting, and that one subscription keeps GPT, Claude and Gemini beside the private mode rather than replacing them with a single local model.

Most pages about private LLMs skip the part that decides the question, which is arithmetic. Whether route one is available to you depends on how many gigabytes of video memory your card has against how many the model needs, and whether route two is sensible depends on an hourly rate multiplied by the hours you leave the machine on. Both numbers are below, computed rather than asserted. Route three is the one this site sells, and its shape matters for the arithmetic: a private mode attached to one subscription that already carries GPT, Claude, Gemini and the open-weight families, rather than a single self-hosted model replacing all of them. This page belongs to our model access section, and it carries a disclosure up front: Perspective AI is route three, we do not sell route one or two, and the readers those routes suit should take them.

What Is A Private LLM?

A private LLM runs so your prompts are not logged, retained, or used to train anyone else's model. Three routes deliver that: weights on hardware you control, a rented dedicated deployment, or a hosted platform storing no transcript.

Those three differ in cost by two orders of magnitude and in guarantee by rather less, which is the tension the rest of this page works through.

If You Came Here To Self-Host, The Honest Answer First

A large share of people searching for a private LLM want to run one themselves, and a subscription product is not that. Perspective AI has no self-hosted build, no on-premise deployment, and no container you can point at your own GPU. If your requirement is that the weights sit on a machine you can unplug, stop reading about platforms and go straight to the next two sections, which are the parts of this page written for you.

Saying so costs nothing and saves you an hour. The reason we can write the self-hosting sections credibly anyway is that the numbers are public and reproducible, and none of them depend on our product being the answer.

The VRAM Arithmetic That Decides Route One

The formula is one line: parameters multiplied by bits per weight, divided by eight, gives bytes of weights. Everything else is overhead on top.

Add roughly ten to twenty per cent for the KV cache and the context window, more if you want long contexts or concurrent users. Now compare against the hardware: the top consumer card most people can buy carries 24 GB. An 8B model at 4-bit fits comfortably. A 32B at 4-bit fits with room for context. A 70B at 4-bit needs about 35 GB and therefore does not fit at all, which is the single fact that ends most self-hosting plans, because 70B-class is where open weights start feeling competitive with hosted frontier models for general work. Which weights are worth the trouble is the subject of the open-source model rankings, and what genuinely runs on a laptop is covered in on-device AI.

Route One: Own The Hardware

Buying means a 48 GB workstation card or better, plus a host machine that can feed it, plus the electricity to run both. The capability is real and the privacy property is the strongest available: the prompt never crosses a network you do not own, and you can prove it with a firewall rule rather than a policy document.

Two costs are habitually underestimated. The first is that you are buying peak capacity to serve average load, so the card sits idle most of the day while the depreciation clock runs. The second is operational: quantizing, serving, updating and monitoring a model is a part-time job, and it is a job that does not exist in either other route. Buy when the requirement is regulatory or contractual, when someone will own the operations, and when you are confident the model choice will not change every quarter.

Route Two: Rent The GPU By The Hour

Renting removes the capital outlay and keeps the isolation, and it makes the economics legible in a way that owning does not. These are Runpod Secure Cloud on-demand list rates, read on 18 August 2026, with the monthly figures computed at 730 hours for an always-on instance and 240 hours for eight hours a day.

The structural point is in the last two columns rather than the first. A rented GPU bills identically whether you send it one prompt in a month or a million, because you are paying for reserved capacity and not for work done. That is the cost shape self-hosting actually has, and it is the opposite of the per-token shape people expect from having used an API. Rates move, and the sensible habit is to re-read the provider's own pricing page on the day you plan rather than trusting any table, this one included. If you turn the instance off between sessions the 240-hour column applies, at the price of a cold start and a stack somebody has to babysit.

Route Three: One Subscription With A Private Mode Attached

The third route gives up the strongest guarantee and buys back everything else. You do not control the hardware, so the property on offer is narrower: not "the prompt never leaves my machine" but "no transcript is stored on the vendor's servers." Whether that is sufficient depends entirely on whether your risk is the inference environment or the durable record, and for most individuals and small teams it is the record.

Perspective AI is this route. Its published documentation describes private-mode conversations as stored only in your browser, encrypted locally per account, and never written to platform storage, with each request relayed statelessly so the server meters the credits used without retaining the prompt or reply. Because history lives on your device, clearing it removes it and there is no server copy to ask anyone to delete. The claim-by-claim sourcing is in our private mode breakdown, and the boundary is stated there too: your prompt still reaches the model that answers it, in this product as in every hosted one.

What route three adds against the first two is breadth and a flat bill. One subscription reaches GPT, Claude, Gemini, Grok and DeepSeek together, at $14.99/mo on Starter or $49.99/mo on Pro for a larger credit allowance. The model catalog lists what is actually available, and how the subscription plans in this market are priced puts that next to the single-vendor alternatives. Against a rented A100 sitting idle at $1,014.70 a month, the comparison is not close for anyone whose actual constraint was the transcript rather than the silicon.

Private GPT Is The Same Question With A Different Word

Private GPT usually means a self-hosted, ChatGPT-shaped interface running an open-weight model instead of a vendor-hosted one. It is not a separate category and everything above applies unchanged: the VRAM table decides whether it runs, and the rental table decides whether it is affordable. One caution worth carrying: several open-source projects use variations of that name, so a search for private GPT will mix the general concept with specific repositories that have their own hardware requirements and their own maintenance status. Check the last commit date before you build a workflow on one.

Which Route Fits Which Reader

Take route one when a rule, a contract, or a regulator says the data cannot leave your infrastructure, and when somebody will own the operations. The cost is capital plus a part-time job, and the compensation is the only guarantee on this page you can test by pulling a cable.

Take route two when the same requirement holds but you would rather rent than buy, or when you are still deciding which model to standardise on. Watch the idle meter, because it is the failure mode that turns a cheap-sounding hourly rate into a four-figure invoice.

Take route three when what you actually wanted was for the conversation not to persist somewhere you cannot see, and when you would rather have every model available than one model you administer. That is most people asking this question, which is why the search results are so full of infrastructure pages that do not fit them. If the underlying worry is who keeps what, the retention clocks compared across vendors and who trains on your conversations answer it with published numbers, and the consumer meaning of private AI sorts out the vocabulary that got you here.

FAQ