How to run an AI model on your own computer
Running a model locally means your data never leaves your computer, it works offline and there are no monthly fees. You only need a computer that’s powerful enough and about 15 minutes.
01 · Check your hardware
What matters most is your graphics card’s memory (VRAM). On Windows: Task Manager → Performance → GPU → “Dedicated GPU memory”. On a Mac with an M-series chip, about three quarters of your main memory can be used.
Rule of thumb for the usual compressed version (Q4): 0.6 GB per billion parameters plus 1.5 GB. A 7B model needs about 6 GB, so it fits an 8 GB card. Our finder checks this for you when you choose “local”.
02 · The easy way: LM Studio
LM Studio is a free desktop app for Windows, macOS and Linux with a normal chat window – no command line needed.
- Download and install LM Studio from lmstudio.ai.
- Open the search (magnifying glass) and type the model name. The models come from Hugging Face.
- Pick a version marked Q4_K_M – a good balance of size and quality. LM Studio shows whether it fits your memory.
- Click Download, then load the model and start chatting.
03 · For tinkerers: Ollama
Ollama runs models from the command line and offers a local API that other programs can use.
- Install Ollama from ollama.com.
- Open a terminal and start a model from the Ollama library, for example:
ollama run <model-name>. Our model pages show the exact command where we’ve checked it. - Models from Hugging Face in GGUF format can be started directly:
ollama run hf.co/<user>/<repository>:Q4_K_M
04 · Finding models on Hugging Face
Hugging Face is the largest library of open models. Tips:
- Filter by GGUF – that’s the format LM Studio and Ollama use.
- Read the model card: it states the licence. Some models may only be used privately or for research.
- Prefer files from the original maker or well-known uploaders.
05 · What to expect
Local models are usually somewhat weaker than the largest cloud models, but often good enough for writing, summarising and translating. The first answer takes a few seconds while the model loads. Without a graphics card it works too – just much slower.