How to Choose a Local LLM: Reading Ollama Model Tags (2026 Guide)

Downloading a local model takes one command. Getting real value out of it takes something else: knowing which model to pick, and why one is right for your task while another wastes your time. This is the part most guides skip. They hand you an install command and a list of models, and leave you to figure out that a 4B model and an 8B model of the same family can behave completely differently.

The tool here is Ollama, which runs open models locally from the command line. But the tool is the easy half. The half worth learning is how to read what a model tells you about itself before you ever run it.

Why the Model Matters More Than the Tool

Every open model was built by a different team, on different data, with different goals. Meta, Google, Alibaba, Microsoft, Mistral, DeepSeek, and IBM are not shipping slightly different versions of the same thing. They are shipping genuinely different models with different strengths, licenses, context limits, and behavior.

Two models with the same parameter count can score worlds apart on your actual task. A model tuned for coding will frustrate you at creative writing. A reasoning model will give better answers on a hard math problem and waste seconds on a trivial one. Picking well is the difference between local AI that feels useful and local AI that feels like a toy.

The good news is that models are not black boxes. Each one publishes its identity in a set of tags. Learn to read them and you can predict how a model will behave before downloading a single byte.

The Types of Models You Will Meet

Before comparing individual models, it helps to know that they come in distinct types. A model built to write code and a model built to search your documents are not competitors. They do different jobs. Recognizing the type is the first filter when you pick one.

Two of these trip people up, so they are worth a sentence each.

Base versus Instruct. A base (or completion) model was trained only to predict the next word. It was never taught to follow instructions. Ask one a question and it might just keep writing more questions, because it is completing a pattern, not answering you. The instruct version of the same model, the one you almost always want, was further trained to respond. On Ollama most listed models are already instruct-tuned, but when you see a base or text tag, that is why it behaves differently.

Embedding models are not chat models. An embedding model gives you no answer at all. It turns a piece of text into a list of numbers that captures its meaning, so that similar texts land near each other. This is the engine behind "search my own documents" and retrieval-augmented generation (RAG). You pair an embedding model with a chat model: the embedding model finds the relevant text, the chat model answers using it.

The lesson is the same as with tags: match the model type to the job before you argue about which one is best. A coding model will not help you search PDFs, and an embedding model will never write you a paragraph.

Installing Ollama

macOS. The cleanest path is Homebrew:

brew install ollama
brew upgrade ollama    # update later

You can also download the .dmg from ollama.com for a normal app install.

Windows. Download OllamaSetup.exe from ollama.com and run it. It installs into your user account with no administrator rights, runs in the background automatically, and serves its API on http://localhost:11434. To store models on another drive, add a user environment variable OLLAMA_MODELS pointing at a path like D:\ollama\models, then restart Ollama from the system tray.

Linux. One command installs Ollama and registers it as a systemd service:

curl -fsSL https://ollama.com/install.sh | sh

Running Ollama as a Background Service

You want Ollama running quietly and starting on boot. After the service is up, every command below behaves the same on every platform.

macOS (Homebrew):

brew services start ollama      # start now and on every login
brew services restart ollama    # apply config changes
brew services info ollama       # status, PID, plist path

Prefer this over running ollama serve by hand. The manual server does not start on boot and manages model storage less reliably.

Windows. The installer already runs Ollama in the background for your user. For a true system service that starts before login, use the standalone ollama-windows-amd64.zip build with a service wrapper like NSSM:

nssm install Ollama "C:\ollama\ollama.exe" serve
nssm start Ollama

Linux:

sudo systemctl enable ollama    # start on boot
sudo systemctl start ollama

Confirm it is running on any platform:

curl http://localhost:11434
# Ollama is running

The Commands You Actually Use

A handful of commands cover almost everything.

ollama run llama3.2:3b                 # start a chat
ollama run llama3.2:3b "Explain TCP"   # one-shot answer, no chat
ollama pull gemma3:4b                  # download or update a model
ollama ls                              # what is installed, with size and quant
ollama ps                              # what is loaded in memory right now
ollama stop gemma3:4b                  # unload from memory, keep on disk
ollama rm gemma3:4b                    # delete from disk
ollama show qwen2.5:7b                 # parameters, template, license

Inside a chat, /show info describes the loaded model, /clear wipes history, and /bye exits. Add --verbose to run to see tokens per second.

Reading a Model's Identity: The Tags That Actually Matter

This is the core skill. When you look at a model on ollama.com/library, it exposes several pieces of information. Each one tells you something concrete about how the model will behave.

Parameter size. The number before the b, as in 7b or 8b, is how many billion parameters the model has. More parameters usually means better answers and higher memory use. But size is not everything. A well-trained 4B model often beats an older 7B one. Size sets the floor for memory, not the ceiling for quality.

Quantization. This is how much the model's weights are compressed. It appears in tags like q4_K_M or q8_0. Lower precision means a smaller download and less memory, at a small cost in accuracy. q4 (4-bit) is the sweet spot for most people and is what Ollama gives you by default. q8 (8-bit) is closer to full quality at roughly double the size. fp16 is full precision, largest and heaviest. If a model is too big for your machine, a heavier quantization is the first lever to pull.

Context length. This is how much text the model can consider at once, measured in tokens (a token is roughly three quarters of a word). Small models range from a few thousand tokens to well over a hundred thousand. This number decides whether you can feed the model a whole document or just a paragraph. If you plan to summarize long files, context length matters more than raw parameter count.

Capability tags. This is the part people miss, and it is the most useful. Models on Ollama carry capability labels:

• thinking means the model has a reasoning mode. It works through a problem internally before answering. Better on logic and math, slower on everything.
• tools means the model supports function calling. It can decide to call code you give it. This is what you need to build an agent.
• vision means the model accepts images, not just text.
• embedding means the model turns text into vectors for search, not chat.

Before you pull anything, that tag line tells you whether a model reasons, sees images, calls tools, or is meant for search. It is the fastest read on whether a model fits your job.

Thinking Models vs Standard Models

This distinction deserves its own section, because it explains a behavior that confuses a lot of people. Consider qwen2.5:7b next to qwen3:8b.

qwen2.5:7b is a standard instruction model. You ask, it answers directly. Fast and predictable.

qwen3:8b is a reasoning model with a thinking mode. Before its final answer, it generates a hidden chain of internal reasoning. That extra step makes it noticeably stronger on math, logic, and multi-step problems. It also makes it slower and more verbose, because it is literally producing more tokens before it responds. An 8B reasoning model can feel slower than a 7B standard one, and the reason is the thinking pass, not the extra billion parameters.

The practical rule: use a standard model for quick everyday tasks, and reach for a reasoning model when the problem genuinely needs step-by-step work. With Qwen3 you can turn thinking off for a single prompt by adding /no_think, which gives you both in one model. The same split shows up across makers: DeepSeek-R1 is a dedicated reasoning family, while Llama 3.2, Gemma 3, and Mistral are standard models.

Why Every Model Family Is Different

Parameter size and tags describe a single model. The family it comes from describes its character. A quick orientation to the makers worth knowing:

Meta (Llama) is the family that opened the field. Broad, reliable, well-documented, with strong community support and long context on its larger models. Its license carries conditions on very large-scale commercial use, worth checking if you deploy widely.

Google (Gemma) is built for efficiency and on-device use. The small Gemma 3 models are excellent instruction followers and several handle images. Permissive licensing makes them easy to build on.

Alibaba (Qwen) ships fast and covers the widest range, from tiny edge models to large flagships, with strong coding and multilingual support across 100-plus languages. Qwen3 brings both thinking and tool use in one model.

Microsoft (Phi) proves small can be smart. The Phi models are text-only but punch far above their size on math and logic, which makes them a strong pick on constrained hardware.

Mistral is lean and consistent, with permissive Apache 2.0 licensing and well-rounded models that are fast on modest machines.

DeepSeek is the reasoning specialist. Its R1 family shows its work and competes with far larger models on hard problems, under permissive licensing.

IBM (Granite) is aimed squarely at business and tool use, tuned for the kind of structured, agentic tasks enterprises actually run.

You do not need to memorize this. You need to know that the differences are real, and that trying a task on two or three families from different makers will teach you more than reading any spec sheet.

Small Models Worth Knowing

These run on almost any recent machine. The rough memory rule for 4-bit versions: budget about the parameter count in GB, plus a little. A 3B model wants roughly 3 to 4 GB free, a 7B to 8B model wants 6 to 8 GB.

A solid starting set: llama3.2:3b for speed, qwen2.5:7b for quality, and qwen3:8b to feel what reasoning adds.

ollama pull llama3.2:3b
ollama pull qwen2.5:7b
ollama pull qwen3:8b

How to Research a Model Yourself

The point of all this is to make you independent. Three sources tell you what any model really is.

The Ollama library (ollama.com/library) is the fast read: sizes, quantizations, context length, and the capability tags. Start here to see at a glance whether a model reasons, sees, or calls tools.

The Hugging Face model card is the deep read. This is where the makers publish the real detail: training data, benchmark scores, context length, intended use, and known limitations. Almost every Ollama model links back to its Hugging Face page. When you want to truly understand a model, this is the primary source.

Independent benchmarks and leaderboards let you compare models head to head on coding, math, or reasoning. Treat the numbers as a guide, not gospel, and always confirm with your own test prompts on the work you actually do. A model that tops a leaderboard can still lose on your specific task.

When the model you want is not in Ollama's library, you pull it straight from Hugging Face. The companion guide, The Hugging Face CLI, walks through downloading any model, choosing its quantization, and running it in Ollama or with MLX on a Mac.

Tuning Performance

Ollama reads its configuration from environment variables. The ones that matter most:

On macOS, set these with launchctl setenv and run brew services restart ollama. On Windows, add them as user environment variables. On Linux, use systemctl edit ollama.

Talking to Ollama over HTTP

The service exposes a REST API on http://localhost:11434. This is what every app and agent talks to.

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Give me three uses for a local LLM.",
  "stream": false
}'

The main endpoints are /api/generate for one-shot completions, /api/chat for conversations and tool use, /api/embeddings for vectors, and /api/tags to list installed models.

Building a Tiny Agent

The moment local AI clicks is when a model does a real task. Two small, complete examples in Python. Install the client first:

pip install ollama

A commit message writer. Reads your staged git changes, sends the diff to a local model, prints a commit message. Nothing leaves your machine.

import subprocess
import ollama

diff = subprocess.run(
    ["git", "diff", "--staged"],
    capture_output=True, text=True
).stdout

if not diff.strip():
    print("No staged changes. Run 'git add' first.")
    raise SystemExit

response = ollama.chat(
    model="qwen2.5:7b",
    messages=[
        {"role": "system", "content":
         "You write git commit messages. One line, imperative mood, "
         "under 70 characters. Output only the message."},
        {"role": "user", "content": f"Write a commit message for this diff:\n\n{diff}"},
    ],
)

print(response["message"]["content"].strip())

A real agent with a tool. An agent is a model that can decide to call a function, get a result, and use it. Here it can look up the time, and decides on its own whether the question needs the tool. This is where the tools capability tag matters: the model must support it.

import ollama
from datetime import datetime

def get_current_time() -> str:
    """Return the current local time as HH:MM:SS."""
    return datetime.now().strftime("%H:%M:%S")

messages = [{"role": "user", "content": "What time is it right now?"}]

first = ollama.chat(model="qwen3:8b", messages=messages, tools=[get_current_time])
messages.append(first["message"])

for call in first["message"].get("tool_calls", []):
    if call["function"]["name"] == "get_current_time":
        messages.append({
            "role": "tool",
            "name": "get_current_time",
            "content": get_current_time(),
        })

final = ollama.chat(model="qwen3:8b", messages=messages, tools=[get_current_time])
print(final["message"]["content"])

That loop, the model asks for a tool, you run it, you feed the result back, is the core of every agent, from this toy to the largest production systems. Everything else is more tools and better prompts. Tool calling needs a model whose tag line includes tools, such as Qwen3, Llama 3.2, or Granite.

Where to Start

Pull llama3.2:3b and chat with it. Pull qwen2.5:7b and feel the jump in quality. Run a reasoning model on a hard problem and watch it work. Then open the model page for anything you download and read its tags, because that habit, not any single model, is what makes local AI genuinely useful.

References

• Ollama Official Documentation↗
• Ollama Model Library↗
• Ollama Python Library↗
• Hugging Face Models Hub↗
• The Hugging Face CLI: Download, Manage, and Run Any Model