26.4B Parameters, 3.4B Active, and Twelve Layers That Run Three Times: Thinking Machines Squeezes an Open Model Into a 20GB Laptop Download
Thinking Machines Lab released Inkling 2 Mini on November 20: a mixture-of-experts model with 26.4B total parameters that activates just 3.4B per token, shipped as a single roughly 20GB checkpoint under Apache 2.0. The company says it runs on a 32GB MacBook. What makes it unusual isn't the size — it's that the middle twelve layers execute three times for every token, trading repeated computation for reasoning depth instead of more parameters.
The short version
- What it is: an open-weights model that takes text and images and returns text, aimed at local devices and agent workloads.
- Why it matters: at roughly the same intelligence score, it activates about a fifth of the parameters Qwen3.8-27B does, and costs $0.59 per task versus $3.53 on one independent measurement.
- What it costs you: looping buys deeper thinking, not more memorized facts. Multi-turn chat and heavily constrained instructions are its weak spots, and the company publishes the scores that show it.
What looping actually changes
In a standard transformer, a token passes through the same stack of layers exactly once, so a model that needs more steps of reasoning has to write them out as text. Inkling 2 Mini does something else: of its 24 layers, layers 0–5 run once, layers 6–17 run three times, and layers 18–23 run once. Twenty-four layers are stored; each token triggers 48 layer visits.
All three passes reuse the same weights. Each pass restarts from the output of layer 6 and passes information forward through a learned message: every looped layer reads the previous pass with its own query, and a small gate decides how much of the current input to forget and how much of the previous message to add. According to Hugging Face's writeup, each pass refines what the one before it computed.
Combined with the MoE layout, the arithmetic works out like this: 26.4B parameters in total, 3.4B unique weights per token, and 5.4B weight activations once the repeated reads across loops are counted. Each layer holds 128 experts and routes every token to 8 of them.
That also explains its position next to Inkling 2 Small, which has 594B parameters. Small was too big for many people to run; Mini is what you get after pruning two-thirds of the experts and adding looping back in.
The numbers: it leads its size class, with caveats
Within the comparison the company draws — open-weights models under 100B total parameters:
- DeepSWE v1.1: 48.2% (Qwen3.8-27B, second, at 42.2%)
- SWE-bench Pro v2 Hard: 77.9% (Qwen3.8-27B at 73.8%)
- BrowseComp: 79.0% (with compaction enabled)
- DeepSearchQA: 85.3%
- MCP Atlas: 82.7%
- Terminal Bench v2.1: 71.6%, where it loses to Qwen3.8-27B at 73.0%
Artificial Analysis independently scores it at 20.1 on its Intelligence Index, level with Qwen3.8 27B while using roughly 5x fewer active parameters. In the same evaluation it emits about 38% more output tokens than that rival but consumes about 2.8x less compute — it saves compute by doing more work per token, and pays for it in verbosity.
The weak spots are stated plainly. It scores 50.0% on MultiChallenge and 33.4% on ComplexConstraints v1.1, behind Qwen3.8-27B on both, and trails larger models on reasoning-heavy benchmarks such as CritPt v2 and HLE-Diamond. On factuality (AA-Omniscience) it only matches models of the same size — looping adds serial depth, not knowledge capacity.
One more detail worth flagging: it barely controls its own chain of thought. Asked to keep reasoning within 750–1,250 tokens, it complies at most 0.1% of the time, against 1.3% for Qwen3.8-27B. If your plan for saving tokens is to cap thinking length, this model will not cooperate.
What it means in practice
The headline for most people: you may not need the cloud. The stated minimum is one NVIDIA DGX Spark, or an Apple silicon machine with 32GB of memory or more, with 32GB of VRAM recommended on other GPUs. The supported stack is already broad: Transformers, SGLang, vLLM, TokenSpeed, NVIDIA Dynamo, Unsloth (including quantized GGUF builds), llama.cpp, Llama App, plus Ollama and Exo on Apple silicon.
The company also offers a concrete reference point: a researcher took it on a flight, where it ran offline on an M5 Pro MacBook Pro with 48GB and built a playable flight simulator in a few minutes. That's not a benchmark score, but it's something you can compare against your own hardware.
Cost is the other shift. First-party pricing on the Tinker platform is $0.15 per million input tokens, $0.50 per million output tokens, and $0.015 per million cached tokens. Artificial Analysis converts that to $0.59 and 9.0 minutes for one Intelligence Index task, versus $3.53 and 38.2 minutes for Qwen3.8 27B.
How to use it: three routes, pick by goal
- Just want to chat with it: open the Tinker Playground. Nothing to install; find out whether you like how it talks before you commit.
- Want it running locally for agents or coding: pull a quantized build through Ollama, llama.cpp, or Llama App, or stand up a server using the SGLang or vLLM recipes in the model card. Start by measuring real tokens per second on a quantized build on your own machine before moving to full precision.
- Want to fine-tune: Tinker covers inference and customization; on the open-source side, Miles, SkyRL, and NVIDIA AutoModel support RL fine-tuning. Keep your training framework and your inference framework from getting crossed.
Cost and time: three bills to check first
- Memory: about 20GB of weights, plus KV cache and context. 32GB is the realistic floor, and a full 1M-token context is not something to expect on a 32GB machine.
- Tokens: it produces roughly 38% more text than Qwen3.8-27B for the same task, which shows up on a local power bill.
- Time: the published curve is a theoretical decoding ceiling, not your measured throughput. On a single card, the gap between a quantized build and full precision can be larger than the gap between models.
Who it's for
- Offline or air-gapped setups that need a model which writes code and calls tools.
- People who already own a 32GB-plus Mac or a DGX Spark and want the strongest open-weights model in that size class today.
- Agent pipelines willing to accept "less compute, more output tokens" as a trade.
- Researchers working on looped architectures or chain-of-thought monitorability — a variable-loop research variant ships alongside it.
Who should skip it
- Multi-turn chit-chat, or tasks with long lists of strict formatting constraints: it clearly trails Qwen3.8-27B on both.
- Trivia and closed-book factual QA: looping doesn't add knowledge, and the company itself points to retrieval and tools instead.
- Anyone who needs tight control over reasoning length to control spend: CoT controllability sits around 0.1%.
- Laptops with 16GB of memory: a 20GB checkpoint won't fit, so don't force it.
Pitfalls
- Don't read "looped" as "smarter." The gains land on long-context retrieval (MRCR-style), math and science reasoning, and long-horizon agent tasks. Closed-book factual QA barely moves.
- Check the harness behind each benchmark. Every row in the official table names its runner (mini-swe-agent, pi, openhands, and so on). Scores shift with the harness, so cross-table comparisons mean little.
- Independent replication is still pending. High scores currently come from the company and two evaluation outfits; broad third-party reproduction hasn't landed yet.
- Quantized is not the same model. Unsloth and llama.cpp GGUF builds are convenient, but low-bit quantization affects long-context and coding behavior in ways you need to measure yourself.
- Apache 2.0 still comes with an acceptable use policy. Read the Acceptable Use Policy linked from the model card before commercial deployment.
First step
If you have a 32GB Mac, the first thing to do today is pull a quantized build through Ollama or llama.cpp and run one real coding task of your own through it — and watch the output token count and wall-clock time. Don't start with the leaderboard. That 48.2% is a DeepSWE number, and your task probably doesn't look like DeepSWE.
Sources
- Thinking Machines Lab announcement: https://thinkingmachines.ai/news/inkling-2-mini/
- Official model card (Hugging Face): https://huggingface.co/thinkingmachines/Inkling-2-Mini
- Hugging Face technical blog (architecture details): https://huggingface.co/blog/inkling2mini
- Artificial Analysis evaluation: https://artificialanalysis.ai/articles/inkling-2-mini
- TestingCatalog report: https://www.testingcatalog.com/thinking-machines-releases-inkling-2-mini-looped-open-model/
- Unite.AI report: https://www.unite.ai/thinking-machines-lab-releases-inkling-2-mini-for-local-devices/
- Vals AI model page: https://www.vals.ai/models/thinkingmachines_inkling-2-mini
Launch details and architecture come from the official announcement and model card. Cost, Intelligence Index figures, and cross-model comparisons are compiled from multiple media reports.
Sources
- https://thinkingmachines.ai/news/inkling-2-mini/
- https://huggingface.co/thinkingmachines/Inkling-2-Mini
- https://huggingface.co/blog/inkling2mini
- https://artificialanalysis.ai/articles/inkling-2-mini
- https://www.testingcatalog.com/thinking-machines-releases-inkling-2-mini-looped-open-model/
- https://www.unite.ai/thinking-machines-lab-releases-inkling-2-mini-for-local-devices/
- https://www.vals.ai/models/thinkingmachines_inkling-2-mini