Run AI on your own hardware.
The reference for engineers running AI locally. VRAM math, hardware tradeoffs, model benchmarks, and the agent ecosystem — all in one place. Built to help you understand, not just install.
Pick your path. Or take all of them.
Seven tools, one mission — help you decide which model, which hardware, and which framework actually fits what you're building.
Local AI means running language models on hardware you own instead of calling a company's API. This section explains what that actually means, the honest reasons people choose it, and the tradeoffs to weigh before you commit.
What is local AI, and why run models locally?
Open-weight models you download and run on your own hardware, the real reasons people choose them, and where the limits actually are.
Most people meet AI through a hosted service. You open a website or an app, type into a box, and a company's servers do the work and send back an answer. Local AI inverts that arrangement. Instead of renting access to a model that lives in someone else's data center, you download the model's weights (the trained numerical parameters that define how it behaves) and run them on your own computer.
The models that make this possible are usually called open-weight models. Their creators publish the trained parameters under a license that lets you download, run, and often modify them. That is not quite the same as open-source software in the strict sense, because the training data and full training code are not always released, but the finished weights are yours to run. Once the file is on your disk, no account, subscription, or internet connection is required to use it. A local runtime program loads those weights, feeds it your prompt, and streams back the response entirely on your machine.
In practice a local setup has three parts: the model file (commonly distributed in quantized formats like GGUF, which compress the weights so they fit in less memory, as covered in the quantization guide), a runtime or inference engine that executes it, and the hardware it runs on. The model catalog on this site tracks open-weight models and the memory each one needs, so you can see what is actually downloadable and runnable rather than only what is hosted behind an API.
Why people run locally
Your prompts and files never leave the machine. There is no account to create, no usage history tied to your identity, and nothing handed to a third party. For sensitive documents, proprietary code, or personal notes, the data simply stays where it already is.
Hosted APIs bill per token, so cost scales with how much you use them. A local model carries an up-front hardware and electricity cost but no metered charge per request. Once it runs, you can generate freely without watching a billing meter. Gauge request volume with the token simulator.
A downloaded model works with no internet at all: on a plane, in an air-gapped lab, or during an outage. It also cannot be rate-limited or discontinued out from under you. The version you saved keeps working exactly as it did the day you downloaded it.
You choose the exact model version, settings, and system prompt, and they do not change unless you change them. A hosted endpoint can be updated silently, while a local model gives you a fixed, reproducible target. That matters for research, testing, and anything that needs to behave the same way twice.
There are no request quotas or throttling tiers. The only ceiling is your own hardware. You can batch, loop, and automate freely, which makes local models a natural fit for agents and pipelines that would otherwise burn through an API quota.
You are not locked to one vendor's single model. You can keep several specialized models on disk, switch between them, and adapt open weights to your own domain. The recommender helps match a model to what you are building.
The honest tradeoffs
Running locally is not free of cost. It trades a recurring bill for an up-front one, and it asks more of you. You become the operator: you pick the model, manage updates, and troubleshoot when something does not load. The largest, most capable open models are enormous, and the very best hosted systems are often bigger still, so a model that fits comfortably on a laptop will not match a frontier cloud model on every task.
The single biggest constraint is GPU VRAM, the dedicated memory on your graphics card. A model has to fit in memory to run quickly, and it is that memory budget, far more than raw compute, that decides which models you can run and how fast they respond. This is why quantization matters so much: compressing the weights to fewer bits per parameter shrinks the memory footprint, letting a larger model fit on a smaller card in exchange for a modest and usually acceptable drop in quality.
As a rough rule of thumb, a 7-to-8-billion-parameter model quantized to 4-bit needs on the order of 5 to 6 GB of VRAM, and the requirement climbs with parameter count from there, while longer context windows add more on top. Treat these as approximations, not guarantees. For the memory a specific model and quantization need on specific hardware, use the stack builder to check what fits, or the workstation builder to plan a machine around the models you want to run. If you have no dedicated GPU, small models can still run on the CPU and system RAM. They are just slower.
Common questions
Do I need a GPU?
Not strictly. Small models run on a CPU using ordinary system RAM. They will simply generate text more slowly. A dedicated GPU with its own VRAM is what makes larger models both possible and fast, because the model can live in fast graphics memory instead of being shuttled through slower system memory. Apple Silicon Macs sit in the middle: their unified memory lets the GPU draw on a large shared pool. If you are choosing hardware, the workstation builder shows what different setups can run.
Is running models locally free?
The software and the open-weight models are typically free to download and use. What is not free is the hardware you run them on and the electricity they draw. The meaningful difference from a hosted API is that there is no per-token or per-request charge: once you own the machine, generating more text does not cost more per use. It is a shift from a recurring metered bill to a mostly up-front cost.
How much VRAM do I need?
It depends on the model's size and how heavily it is quantized. As an approximate guide, a 7-to-8-billion-parameter model at 4-bit quantization needs roughly 5 to 6 GB of VRAM, and the requirement grows with parameter count and context length. Rather than rely on a single number, check a specific model and quantization against your hardware with the stack builder, and read the quantization guide to understand the memory-versus-quality trade.
Is a local model as good as ChatGPT?
For many everyday tasks, such as drafting, summarizing, coding help, and question answering, a good open-weight model running locally can be very capable. On the hardest reasoning and the broadest general knowledge, the largest hosted frontier systems still tend to lead, partly because they are far bigger than what fits on typical personal hardware. The honest framing is that local models have closed much of the gap for common work while trading some peak capability for privacy, cost control, and independence. The model index and community data can help you gauge which open models are strongest for your use case.
Where do I start?
Begin with the getting-started guide, which walks through installing a runtime and downloading your first model. If you are not sure which model to pick, the recommender suggests options from your use case and hardware, and the stack builder confirms what will actually fit before you download anything.
We don't list specs. We help you understand the tradeoffs, compare the options, and decide what to actually run.
Ready to run AI locally?
No accounts required. Free reference material for engineers running AI on their own hardware.