Open-weight models now come from OpenAI, Google, NVIDIA, Meta, Mistral and several Chinese labs, and the best of them handle most business work well. Picking one is a four-way trade: quality on your task, the memory it needs, the license, and where it came from. This page is the shortlist I start every engagement from. It changes every few months, and I keep it current.
How to read the tables
- Total / active parameters. Most strong models are now mixture-of-experts: all the parameters must fit in memory, but only the active ones are used per token, which sets the speed.
- Memory is approximate at 4-bit quantization, before room for context. Add 10 to 30%.
- Speed is single-user generation from published measurements, on the hardware named. Many users at once on a GPU server multiplies total throughput many times over.
- License matters. "Open" ranges from Apache 2.0 and MIT, which let you do almost anything, to custom licenses with attribution rules or revenue thresholds.
General assistant and search over your documents
The everyday workhorse: answering questions, drafting, summarizing and retrieval-augmented generation (RAG) over company documents.
| Model | Origin | Total / active | Memory (4-bit) | License | Measured speed |
|---|---|---|---|---|---|
| gpt-oss-120b | OpenAI, US | 117B / 5.1B | about 65GB | Apache 2.0 | 35 to 56 tok/s on a DGX Spark or Strix Halo |
| Nemotron 3 Super | NVIDIA, US | 121B / 12.7B | about 68GB | NVIDIA open model license | |
| Gemma 4 31B | Google, US | 31B dense | about 18GB | Apache 2.0 | |
| Qwen3.8-Flash-Next | Alibaba, China | 125B / 6B | about 100GB | Qwen community license | 54 to 75 tok/s on a Mac Studio M5 Ultra |
My default is gpt-oss-120b. It's US-made, Apache-licensed, fits in a single 128GB box and handles tools well. Gemma 4 is the pick when a smaller footprint or strong multilingual support matters: it covers 140 languages.
Coding and agents
Agents read huge amounts of context and call tools in loops, so reliability at tool calling and long context matter more than raw chat quality.
| Model | Origin | Total / active | Memory (4-bit) | License | Measured speed |
|---|---|---|---|---|---|
| DeepSeek V4 Flash | DeepSeek, China | 284B / 13B, 1M context | about 160GB | MIT | about 100 tok/s on 8x RTX PRO 6000; 40 to 53 on an M5 Ultra |
| GLM-5.3-Flash | Z.ai, China | 320B / 18B, 1M context | about 180GB | MIT | |
| Mistral Medium 3.5 | Mistral, France | 128B dense | about 72GB | Modified MIT, revenue threshold | 23 tok/s on 4x M5 Ultra |
| Qwen3.8-27B | Alibaba, China | 27B dense | about 16GB | Apache 2.0 | 51 tok/s on 4x M5 Ultra |
| Nemotron 3 Nano | NVIDIA, US | 32B / 3.6B | about 18GB | NVIDIA open model license |
The strongest open coding and agent models right now are Chinese-origin, which is a real issue for some clients (see below). For a US-only shortlist, gpt-oss-120b and Nemotron 3 are capable agent models, and Mistral Medium 3.5 is the European option.
Long documents: contracts, records, filings
Long-document work needs a big context window and, above all, fast prompt processing. A 200-page contract is around 100,000 tokens; the model reads all of them before it writes a word.
- Context: DeepSeek V4 Flash and GLM-5.3-Flash take up to 1M tokens, Qwen3.8 about 262K, Gemma 4 31B about 256K.
- Hardware: this is where NVIDIA matters. Reading a 32,000-token prompt took 44 to 50 seconds on a Mac Studio M5 Ultra and about 17 seconds on two DGX Sparks in one comparison, and a GPU server is far faster again.
- Design: for document review at scale, I usually split documents and process them in parallel across many requests, rather than relying on one giant context. It's faster, cheaper and easier to verify.
Content and SEO writing
Drafting product descriptions, location pages, meta descriptions and articles at volume is one of the best uses of a private model: the work is high-volume, the cost per token is zero, and the output always gets human review anyway.
There's no meaningful benchmark for "good marketing copy", so I judge models on samples in your brand voice. Mistral Medium 3.5, gpt-oss-120b and Gemma 4 31B all write well; the bigger difference comes from good prompts, examples of your best existing content and a review step. Running this privately also keeps unreleased product and campaign plans off third-party servers.
The supporting cast
The main model gets the attention, but real systems need several small models alongside it. All of these run on a single GPU or share the main box.
| Job | Models | Notes |
|---|---|---|
| OCR and scanned documents | olmOCR 2 (AllenAI, US, Apache), PaddleOCR-VL (Apache), DeepSeek-OCR 2 | Turns scans and PDFs into clean text before the main model sees them |
| Embeddings | Qwen3-Embedding (0.6B to 8B, Apache), BGE-M3 (MIT) | Powers document search; choose by language mix and size |
| Reranking | Qwen3-Reranker, BGE reranker | Re-sorts search results so the right passage reaches the model |
| Transcription | Whisper large-v3-turbo (MIT), NVIDIA Parakeet and Canary | Depositions, calls, dictation; Parakeet runs thousands of times faster than real time |
| Safety and policy checks | gpt-oss-safeguard (Apache) | Classifies prompts and outputs against your own written policy |
| Routing and quick tool calls | gpt-oss-20b, Gemma 4 E4B, Nemotron 3 Nano | Small, fast models that triage requests; gpt-oss-20b runs at 60 to 80 tok/s on a DGX Spark |
The very largest open models
Muse Spark Open (Meta, 816B, released October 2026), GLM-5.3 (744B), Kimi K3 (2.8 trillion) and Nemotron 3 Ultra (550B) approach frontier quality, but need anywhere from 300GB to well over a terabyte of memory. They run on four-Mac clusters for a single user (Apple measured Muse Spark Open at 36.7 tok/s on four M5 Ultras) or on 8-GPU data-center servers for many users. Most companies don't need them; a well-chosen mid-size model plus good retrieval usually wins on cost and speed.
Origin and license: the questions your board will ask
- Defense and government work. The FY2026 defense authorization act bars DoD contractors from using DeepSeek in contract performance, with a ban for lower-tier suppliers scheduled to follow. Several states ban it on state devices. For defense-adjacent clients I default to a US-origin shortlist: gpt-oss, Gemma 4, Nemotron 3, Muse Spark Open, olmOCR, Whisper and Parakeet.
- Private companies. There's no blanket ban, and open weights running on your own hardware send nothing back to anyone. Some boards and clients still refuse Chinese-origin models, and that's a legitimate business decision to make up front.
- License terms. Apache 2.0 and MIT are simple. Custom licenses (Muse Spark Open, Kimi K3, GLM-5.3, the Qwen community license, Mistral's revenue threshold, Llama's legacy license) can require attribution, cap commercial scale or restrict use. Read them before you build a product on one.
Frequently asked questions
What is the best open-source LLM for business in 2026?
For general business use on a single machine, gpt-oss-120b is the safest default: US-made, Apache 2.0 licensed, about 65GB at 4-bit and fast on 128GB hardware. For coding and agents, DeepSeek V4 Flash and GLM-5.3-Flash lead, if Chinese origin is acceptable to you and your clients.
Which local model is best for long documents?
One with a long context window, served on hardware that reads prompts fast. DeepSeek V4 Flash and GLM-5.3-Flash handle up to 1M tokens, and Gemma 4 31B about 256K. Processing long documents in chunks across many parallel requests is usually faster and more reliable than one huge prompt.
Are open-weight models safe to use with confidential data?
Running on your own hardware, the model can't send data anywhere: the weights are just a file. Your risks are the usual ones: who can access the system, what it logs and whether its answers are verified. Those are handled by the gateway, permissions and evaluation around the model.
Can we use Llama models?
You can, but Meta's older Llama 3 models are now behind the field and come with a custom license that includes an acceptable-use policy. Newer models with Apache or MIT licenses are usually a better starting point.
How often should we change models?
Plan to review every quarter and switch only when a bake-off on your own test set shows a real gain. A model you've validated is worth more than a slightly better one you haven't.