Which Model?

working memory

Context window

Everything you paste — prompt, attachments, prior chat — has to fit. Bigger window means more code or more pages of a contract before you start chunking.

Use bigger windows for whole repos, long contracts, or many attached PDFs. Use smaller ones when the prompt is short and latency matters.

thinks before it answers

Reasoning model

A second class of model that runs an internal "scratchpad" before responding. Slower and pricier per query, but lands harder problems — math, multi-step logic, code refactors.

Best for hard math, planning, and codebase refactors. Overkill for rewriting a paragraph or classifying a support ticket.

handles more than text

Multimodal

Reads images, audio, video, or PDFs as input — not just text. Native multimodal models trained on all four feel sharper than ones bolted onto text-only bases.

Choose this when the task includes screenshots, charts, PDFs, audio, or video. Text-only work does not need the extra surface area.

the unit you pay by

Tokens

Sub-words the model reads and writes — roughly 0.75 word per token. Pricing is "$X per million input tokens / $Y per million output." Output is usually 3–5× more expensive than input.

Long inputs and long outputs both cost money. Summaries are cheap; repeated analysis loops over giant files are where bills grow.

fast vs. flagship

Tier (Fast · Mid · Flagship)

Each lab ships a tiered family. Fast tiers are 5–20× cheaper and answer in <1s — built for production pipelines. Flagship tiers cost more, think harder, and only make sense when quality matters more than latency.

Start with the cheapest tier that can do the job, then move up only when you see quality failures you can name.

can you run it yourself

Open vs. closed weights

Open-weight models publish the trained parameters — you can self-host, fine-tune, or run offline. Closed-weight models live behind an API only. Open trades some quality for full control and zero per-call cost.

Use open weights for privacy, offline use, fine-tuning, or predictable unit economics. Use hosted APIs when managed quality matters more.

can it call other things

Tool use / agents

Whether the model can call APIs, run code, search the web, or read files mid-conversation — instead of just generating text. The foundation under every "agent" headline.

Tool access is what turns a chat answer into a workflow: search, fetch, calculate, write, verify, repeat.

how recent its world is

Knowledge cutoff

The date training data ends. Anything after — a release, a news event, a library version — the model has to be told or look up. Models with web search smooth this over; pure-LLM answers don't.

For current events, prices, docs, or laws, use a model with retrieval or bring the sources yourself. Guessing from memory is the trap.