Two years ago, "our own language model" meant a data centre, specialists and a six-figure budget for most companies. If you wanted AI, you rented an API in the cloud and paid per request.
That has shifted quietly but thoroughly. The models that run on a single graphics card today handle most of the tasks that come up in a business. They may be used commercially, and they can be set up in an afternoon.
What fits on one graphics card today
Three examples show how far this has come:
- gpt-oss-20b from OpenAI comes close to o3-mini on common benchmarks, according to OpenAI, and runs in 16 GB of memory, so on a good graphics card or a well-equipped laptop. Its big sibling gpt-oss-120b fits on a single 80 GB data-centre GPU.
- Gemma 4 from Google, released in April, goes up to 31 billion parameters, understands text, images and audio and more than 140 languages. It is the first Gemma generation under the permissive Apache 2.0 licence.
- Qwen 3.6-35B-A3B from Alibaba is currently the community's workhorse for a 24 GB graphics card.
All three are licensed under Apache 2.0. You may download them, run them in-house, adapt them and use them commercially without paying the maker.
The trick: stored large, computed small
How does a model with 35 billion parameters fit on a graphics card built for computer games? Many of the new models are built as a "mixture of experts". The whole model is stored, but only a small part of it does the computing for each word. For Qwen 3.6-35B-A3B that is about 3 billion parameters, for gpt-oss-20b about 3.6 billion.
Think of a large company where only the responsible department works on each request. The knowledge of the whole firm is there, the computing work stays small. On top of that, models for local use are compressed to about 4 bits per parameter, with barely noticeable loss of quality.
What you need
| Hardware | What runs on it |
|---|---|
| Graphics card or Mac with 16 GB | gpt-oss-20b, Gemma 4 12B |
| Graphics card with 24 to 32 GB or Mac with 32 GB | Qwen 3.6-35B-A3B, Gemma 4 26B and 31B |
| Mac with 64 GB or more, or an 80 GB graphics card | gpt-oss-120b |
For trying things out, tools like Ollama or LM Studio are enough. For continuous operation on a server that several workflows rely on, you use server software such as vLLM.
How good are they?
To be honest: the strongest cloud models are still ahead. According to the Artificial Analysis Intelligence Index, the best open models trail the best closed ones by about twelve points in October. And the open top models with more than a trillion parameters do not fit on any single graphics card.
For many tasks in a business, that gap does not matter. Reading a document and extracting the fields, routing an email to the right department, summarising a text, drafting a reply: these are the steps most AI workflows consist of, and a model running under the desk is enough for them.
Why this is interesting
No cost per request. Whether the model reads 50 or 5,000 delivery notes a day only makes a difference on the electricity bill. For high-volume tasks in particular, that adds up quickly.
It stays the way it is. A model on your own machine does not change overnight because a provider rolls out a new version. What worked in testing still works next month.
It runs where the data is. Next to the ERP server, on your own network, even when the internet connection drops.
Best of both
The strongest solution is often a combination. The many simple steps are handled by the in-house model, and only the rare hard questions go to a large model in the cloud. Anthropic describes exactly this pattern as "routing": a request is classified and passed to the model that suits it best. I described how such a workflow is put together in The best AI agent is often a good workflow.
Try it yourself
If there is a computer with a good graphics card in your office, you can test this this afternoon. Install Ollama, download gpt-oss-20b and paste in the text of ten real orders, asking it to extract customer, items and quantities. For scanned documents, use Gemma 4, which also reads images. After an hour you will have your own feel for what such a model can do.
Whether an in-house model is enough for your tasks or a cloud model is the better fit, I test in the AI Potential Check on your own examples.
Sources
- OpenAI: Introducing gpt-oss, 5 August 2025
- Google: Gemma 4: Byte for byte, the most capable open models, 2 April 2026
- agitracker.io: 10 Best Open Source LLMs in 2026, 15 October 2026
- vdf.ai: Best Local LLM by VRAM, 6 October 2026
- Anthropic: Building effective agents, 19 December 2024