Local Agents
Run your own AI on your own hardware: setup, model selection, and absolute privacy.
1 Why and on what
Local AI: why keep a neural net on your own machine
Local AI in plain words: data privacy, free after install, works offline and with no limits. And honest about the downsides - weaker than the cloud and you need spare memory.
Hardware for Local AI: How Much VRAM You Actually Need
What hardware runs local AI models: the one factor that matters is your graphics card's video memory (VRAM), RAM as backup, Apple Silicon with unified memory. Ballpark figures for 7B, 13B and 70B, plus a rescue plan for a weak machine.
2 Install and run
Ollama: run a local AI model with a single command
Ollama is the easiest way to run a local AI model: installation, the pull and run commands, an offline chat in your terminal. We break down the quiet server under the hood in plain words.
LM Studio: local LLMs without the terminal, all point-and-click
LM Studio - a GUI for local LLMs: search and download models from Hugging Face, chat just like ChatGPT, and run a local server with an OpenAI-compatible API. No terminal.
How to Pick a Local AI Model for Your Hardware
How to pick a local AI model: we break down families (Llama, Qwen, Mistral, Gemma), parameter counts from 7B to 70B, and Q4/Q5/Q8 quantization to fit your computer's VRAM.
3 Put it to work
Connecting a local model to an agent: base URL and key
A local model serves an OpenAI-compatible address on localhost. Here's how to wire it into your editor, an extension, and your own script using a base URL and a dummy key.
Local or cloud AI agent: what to pick and when
Local or cloud AI agent - which one should you pick? Privacy and zero cost vs brains and speed. When your own machine is enough, when you need the cloud, and how to build a hybrid.