Ollama

Verdict

Ollama is the least painful way to run a local model, and it works on ordinary hardware — we got 23 tokens per second on a CPU-only desktop, which is roughly reading speed. If you want to try local AI without buying a GPU, start here.

Who should not use it: anyone who needs instant answers. Our test prompt took 86 seconds. A hosted model answers the same question in two or three. If you are building something interactive, or you are impatient, local inference on a CPU will frustrate you.

Tested on

There is no discrete graphics card in this machine. Inference ran on the CPU. That is the point of the test — almost every published benchmark assumes an NVIDIA card, and most people do not have one.

Measured performance

The prompt was: “Explain what a vector database is, in three short paragraphs.”

The thing nobody warns you about

We asked for three short paragraphs. Ollama returned 1,985 tokens.

qwen3.5:2b is a reasoning model, and it thinks out loud. Before answering it produced an eight-step “Thinking Process”, drafted the paragraphs three times, second-guessed itself — “Wait, I need to make sure I don’t add unnecessary formatting”, “Actually, I can merge some ideas to make it punchier” — and ran a final check that it had in fact written three paragraphs.

The answer itself is around 260 tokens. Roughly 85% of the compute went to deliberation nobody asked for — about 75 seconds of visible thinking for 9 seconds of answer.

This is not a flaw in Ollama. It is a property of the model, and it is invisible until you run it. If you pick a reasoning model for a simple task, you pay for the reasoning. Choose accordingly.

What it actually costs

Free to install and run locally. No account required for local models.

Ollama’s launcher now also offers :cloud models. In our install, two of the three recommended models were cloud-hosted and one was marked “Upgrade required” — so there is a paid tier, and it is surfaced prominently in a tool whose premise is local execution. Worth knowing before you assume everything on the menu runs on your machine.

The real cost is hardware and disk. Models are large: a 2B model is ~2.7 GB, and the 26B Gemma 4 offered in the launcher is around 19 GB.

Pricing checked 15 August 2026.

Limits and failure modes

• CPU-only inference is slow but usable. 23 tokens/s is about reading speed. Fine for drafting, painful for anything interactive.
• Model names are not guessable. gemma4:4b does not exist — Gemma 4’s small variants are e2b and e4b. We also had llama3.2:3b fail. Check the tags before pulling.
• Reasoning models are expensive on simple prompts — see above.
• Disk fills quickly. Use ollama rm to clear models you have finished with.
• No GPU acceleration on Intel integrated graphics in this configuration.

Alternatives

• LM Studio — the same idea with a graphical interface. Better starting point if you dislike the terminal.
• llama.cpp — the engine underneath much of this. More control, more work.
• Open WebUI — a browser front-end that sits on top of Ollama.
• Jan — offline-first desktop app, no terminal at all.

Install

Download from ollama.com/download and run the installer. Then:

ollama --version
ollama pull qwen3.5:2b
ollama run qwen3.5:2b --verbose

The --verbose flag prints the timing block after each response. Without it you get an answer and no data.

Full walkthrough, including the pulls that failed: Install Ollama and run your first local model

Last verified: 15 August 2026 · Ollama 0.32.13