Run MBZUAI's K2 Horizon on your own machine: the fully-open fleet from 0.9B to 375B
Most "open" models ship you the weights and stop there. MBZUAI's Institute of Foundation Models did the opposite with K2 Horizon: six models from 0.9B to 375B parameters, Apache 2.0, plus the training data, the training code, the intermediate checkpoints, and the fine-grained logs — the whole recipe, not just the dish. The weights first appeared on Hugging Face in September; the formal launch landed this week, with the full technical paper (arXiv 2611.21003) submitted on November 16 and a press push on November 17. This tutorial takes the fleet at its word: it runs the smallest model on an ordinary CPU-only machine with llama.cpp, shows you its thinking as it works, and maps exactly how to reach the bigger sizes — including the 375B flagship — when a laptop stops being enough.
- What it is: six reasoning models from MBZUAI's Institute of Foundation Models (IFM) — 0.9B, 3.7B, 7B, 32B dense; 36B-A4B (MoVA) and 375B-A23B MoE — all Apache 2.0, all with 512K context except the 0.9B at 128K.
- Why it is trending: weights public since September; formal launch and technical paper (arXiv 2611.21003) landed November 16–17, 2026, with the most complete open release yet — data, code, intermediate checkpoints, and logs, not just weights.
- Cost: free and open source. No account, no API key, no GPU for the local models; the 375B flagship needs an 8× H200 node or a hosted API account.
- Verified in this tutorial: the 0.9B ran end to end in our sandbox on November 17, 2026 — download, llama.cpp v1.0.0 build, maths and code prompts with the thinking trace. Bigger sizes use IFM's own published recipes, reproduced verbatim from the model cards.

The small models are not toys. The 0.9B scores 48.5 on AIME 2026 — a competition-maths benchmark where a same-size Qwen scores 0.2 — and 79.9 on HumanEval+. The 3.7B scores 65.6 on SWE-bench Verified and 75.2 on HMMT, numbers that would have been flagship-grade two years ago. Both are reasoning models: they think before they answer, call tools, and handle long context (128K for the 0.9B, 512K for everything else). By the end of this tutorial you will have the 0.9B running locally, you will know which size to reach for, and you will know where the vendor's benchmark claims need a pinch of salt.
What you'll need#
- Python 3.10 or newer is handy but not required — the main path below is a single compiled binary.
- About 1 GB of free disk for the 0.9B model file (635 MB) plus the llama.cpp build. No GPU is needed — the run below happened on a 4-core CPU-only machine.
- 4 GB of RAM or more. The 0.9B runs happily in under 1 GB.
- A runner that knows K2 Horizon (Step 2): llama.cpp v1.0.0 or newer, llama-cpp-python 0.3.37 or newer, or Ollama 0.40.4 or newer. Older llama.cpp builds cannot load these models at all.
- No account, no API key, no payment for the local part. The 375B flagship in Step 6 needs either serious hardware or a hosted API account.
The fleet at a glance#
Six models, one architecture family, one serving stack. The four middle sizes (3.7B, 7B, 32B, 36B-A4B) were trained on exactly the same 22 trillion tokens; the 0.9B trained on 5 trillion, and the 375B flagship on 15.1 trillion of its own. Every model is a reasoning model with tool use, and every model except the 0.9B carries a native 512K-token context window.
| Model | Params (active) | Context | Where it runs | Headline result (vendor-reported) |
|---|---|---|---|---|
| 0.9B | 0.9B dense | 128K | Watch, glasses, edge | AIME 2026: 48.5; HumanEval+: 79.9 |
| 3.7B | 3.7B dense | 512K | Phone, laptop | SWE-bench Verified: 65.6; HMMT: 75.2 |
| 7B | 7B dense | 512K | Phone, laptop | SWE-bench Verified: 72.4; BrowseComp: 59.0 |
| 32B | 32B dense | 512K | Workstation, on-prem | GPQA Diamond: 82.3; Terminal-Bench 2.1: 36.6 |
| 36B-A4B (MoVA) | 36B MoE (4B active) | 512K | Workstation, serving | Terminal-Bench 2.1: 58.6; tau3-Banking: 26.8 |
| 375B-A23B | 375B MoE (23B active) | 512K | 8× H200 node or hosted API | Toolathlon Verified: 65.3; Terminal-Bench 2.1: 70.2* |
* The 375B Terminal-Bench figure is the vendor-reported 70.2%; IFM's own audit (see "How much to trust the benchmarks") revises it to 66.9% after removing reward-hacking trials. All other figures are from the official model cards on Hugging Face, read November 17, 2026.
How K2 Horizon is different#
Three ideas in this release are worth understanding before you run anything, because they explain the numbers above.
MoVA — Mixture-of-Value Attention. Mixture-of-experts normally applies sparsity to the feed-forward layers: many experts exist, a router wakes a few per token. MoVA extends the same principle to attention itself, routing which value-heads compute for each token. The result is the 36B-A4B model: 36 billion parameters in total, only about 4 billion active per token, performing slightly below the dense 32B model while using roughly an eighth of the active compute. It is the most interesting architecture idea in the fleet, and the reason the 36B is the one to watch for efficient serving.
Uno — a lossless speedup adapter. Reasoning models are slow because they generate one token at a time and think at length. Uno is a small LoRA adapter trained by diffusion distillation: it learns to generate blocks of tokens in parallel while the frozen autoregressive weights stay responsible for the output distribution. IFM reports roughly a 3× speedup with no quality loss, and the adapter is plug-and-play — you attach it, you do not retrain the model. The Uno adapters ship for the small models (K2-Horizon-0.9B-Uno and 7B-Uno are already on Hugging Face); treat the 3× figure as the vendor's claim until independent measurements land.
FTPO — the anti-doom-loop stage. Small reasoning models sometimes fall into degenerate repetition, generating the same phrase until they hit the output limit. After post-training, the 3.7B and 7B get a short Final Token Preference Optimization stage — a LoRA preference update trained on pairs mined from the model's own looping outputs, merged back into the weights. The effect is large where it matters: the 3.7B's Terminal-Bench score jumps from 25.1 to 39.7 and its function-calling score (BFCL v4) from 51.0 to 64.6, while ordinary benchmarks stay flat or improve slightly.
And then there is the openness itself. This is the LLM360 lineage: the 2023 LLM360 paper argued that open weights without the training process are only half open. K2 Horizon ships intermediate checkpoints for every stage (pretraining, mid-training, RL experts, SFT phases), the data or a full data-construction recipe where redistribution is restricted, the xLLM training infrastructure, and the RL360 post-training code. As IFM's founder Eric Xing put it in the launch release: "We believe meaningful AI progress depends on the ability to examine, build upon, and improve the technology, not simply access it through an API."
Step 1 — Download the model#
IFM publishes official GGUF builds for every size — the repos are IFM/K2-Horizon-0.9B-GGUF, IFM/K2-Horizon-3.7B-GGUF, and so on — each offering BF16, Q8_0, Q6_K, Q5_K_M, Q5_0, and Q4_K_M. For a CPU machine, Q4_K_M is the right default: it is roughly a third the size of Q8_0 and, on IFM's own GGUF evaluation of the 3.7B, keeps most of the quality (average 66.3 across seven tasks, against 73.2 for BF16 and 71.4 for Q8_0 — the biggest single drop is on AIME, 74.1 vs 91.6, so if competition maths is your workload, step up to Q5_K_M or Q8_0).
Create a folder and download the 0.9B file (635 MB):
mkdir k2-horizon && cd k2-horizon
curl -L -o K2-Horizon-1B-Q4_K_M.gguf \
https://huggingface.co/IFM/K2-Horizon-0.9B-GGUF/resolve/main/K2-Horizon-1B-Q4_K_M.gguf
ls -la *.gguf
Expected output (your byte count should match exactly):
-rw-rw---- 1 root root 666184064 Nov 17 17:28 K2-Horizon-1B-Q4_K_M.gguf
Two naming quirks to know. First, the file is called 1B even though the model is 0.9B — the GGUF repos round the names up (the 3.7B file is K2-Horizon-4B-Q4_K_M.gguf, 3,156,598,144 bytes). Second, the 36B model is listed as K2-Horizon-MoVA-36B-A4B, not 36B-A4B; if you go looking for the bigger GGUF builds later, that is the repo name.
Step 2 — Install a runner that knows K2 Horizon#
K2 Horizon uses a new k2-horizon architecture tag, and older llama.cpp builds do not know it. If your runner predates the support — mainline support merged in llama.cpp release b11454 — the load fails outright. Here is what that looks like with llama-cpp-python 0.3.35, the version that happened to be installed in our sandbox:
llama_model_loader: loaded meta data with 35 key-value pairs and 255 tensors
from K2-Horizon-1B-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: - kv 0: general.architecture str = k2-horizon
ValueError: Failed to load model from file: K2-Horizon-1B-Q4_K_M.gguf
The file is fine — the metadata loads, the architecture string is right there — but the bundled llama.cpp has no model implementation for it. The fix is to upgrade past the merge. The most reliable path is to build llama.cpp from source (v1.0.0 or newer), which is what we did in the sandbox:
git clone --depth 1 --branch v1.0.0 https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DLLAMA_BUILD_TESTS=OFF
cmake --build build --target llama-cli -j2
That produces build/bin/llama-cli. Confirm it knows the architecture before going further — ask it for the model info and check the load succeeds:
./build/bin/llama-cli -m ../k2-horizon/K2-Horizon-1B-Q4_K_M.gguf --version
Two alternatives. If you prefer Python, llama-cpp-python 0.3.37 or newer bundles a llama.cpp new enough to load K2 Horizon — install with python3 -m pip install --user --break-system-packages --upgrade llama-cpp-python (a source build; allow several minutes) and use its Llama class with create_chat_completion, which applies the GGUF's embedded chat template for you. If you prefer one command and no build, IFM publishes an official Ollama model, ifm/k2-horizon, which needs Ollama 0.40.4 or later:
ollama run ifm/k2-horizon:0.9b-q4_K_M
The Ollama tags mirror the fleet — 0.9b, 3.7b, 7b (the default), 32b, 36b — and every size comes in Q4_K_M, Q5_0, Q5_K_M, Q6_K, Q8_0, and BF16. One Ollama caveat, from the model's own page: Ollama does not yet pass the model's reasoning from earlier turns back to it in multi-turn chats and tool loops, so long agentic sessions can lose the thread. A fix is under review in the Ollama repo. The llama.cpp path in this tutorial does not have that limitation, which is why it is the main path here.
Step 3 — First run: the 0.9B#
K2 Horizon always thinks before it answers, and the chat template makes that explicit: a user turn is wrapped in <|ifm|im_start|>user … <|ifm|im_end|>, and the assistant turn opens with <ifm|think>, holds the reasoning, closes with </ifm|think>, and then gives the answer. For a first run, pass that prompt shape directly to llama-cli. Two settings matter and both come from the model card: temperature 0.6 for the 0.9B (the 3.7B and larger use 1.0), and top_p 0.95.
./build/bin/llama-cli -m ../k2-horizon/K2-Horizon-1B-Q4_K_M.gguf \
-t 4 -c 4096 -n 2048 --temp 0.6 --top-p 0.95 \
-p "<|ifm|im_start|>user
A train travels 240 km in 3 hours, then 180 km in 2 hours. What is its average speed for the whole trip? Show your working.<|ifm|im_end|>
<|ifm|im_start|>assistant
<ifm|think>
"
What came back in our sandbox run (November 17, 2026, 4 CPU threads, no GPU):
[Start thinking]
The user is asking for the average speed for a train that travels different distances in different times. Let me break this down.
Total distance = 240 km + 180 km = 420 km
[… thinking trimmed — the trace repeats the same calculation several times before converging …]
But let me verify this by calculating per leg speeds and then averaging.
Actually, average speed is not simply the arithmetic mean of the individual speeds. The correct way to calculate average speed is:
Total distance / Total time = 420 / 5 = 84 km/h
[…]
Alternatively, if someone calculated (80 + 90)/2 = 85 km/h, that's incorrect because that's not how average speed works. Average speed is the total distance divided by total time, not the average of the two segment speeds.
So the correct answer is 84 km/h.
Let me present the working clearly.
[End thinking]
To find the average speed for the whole trip, we need to use the formula for average speed:
$$ ext{Average Speed} = rac{ ext{Total Distance}}{ ext{Total Time}}$$
**Step 1: Calculate the total distance**
$$240 ext{ km} + 180 ext{ km} = 420 ext{ km}$$
**Step 2: Calculate the total time**
$$3 ext{ hours} + 2 ext{ hours} = 5 ext{ hours}$$
**Step 3: Calculate average speed**
$$rac{420 ext{ km}}{5 ext{ hours}} = 84 ext{ km/h}$$
**Alternative verification using individual segment speeds:**
- First segment: $rac{240}{3} = 80 ext{ km/h}$
- Second segment: $rac{180}{2} = 90 ext{ km/h}$
Average speed = $rac{420}{5} = 84 ext{ km/h}$
**Answer:** The train's average speed for the whole trip is **84 km/h**.
The run took about two minutes wall-clock on four CPU threads, generating at 9.6 tokens per second, and the thinking trace below is trimmed — the full trace repeats the same calculation several times before converging, which is typical of a small reasoning model at this size. The answer is right — 420 km over 5 hours is 84 km/h — and the thinking trace shows the model doing the arithmetic in its head rather than pattern-matching the final number. That is the point of a reasoning model at this size: the chain of thought is doing real work, and you can read it.
Step 4 — A second prompt: code#
Maths is one thing; code with a trap in it is a better test. Ask for a median function that must handle odd and even lengths and return None for an empty list:
./build/bin/llama-cli -m ../k2-horizon/K2-Horizon-1B-Q4_K_M.gguf \
-t 4 -c 4096 -n 2048 --temp 0.6 --top-p 0.95 \
-p "<|ifm|im_start|>user
Write a Python function that returns the median of a list of numbers. It must handle both odd and even length lists, and an empty list should return None. Show only the code.<|ifm|im_end|>
<|ifm|im_start|>assistant
<ifm|think>
"
The sandbox result:
[Start thinking]
The user wants a Python function that returns the median of a list of numbers. Requirements:
- If the list has odd length: median is the middle element after sorting
- If the list has even length: median is the average of the two middle elements
- For empty list: return None
I need to write a function. Let me think about the implementation:
```python
def median(numbers):
if not numbers:
return None
sorted_numbers = sorted(numbers)
n = len(sorted_numbers)
if n % 2 == 1:
return sorted_numbers[n // 2]
else:
mid = n // 2
return (sorted_numbers[mid - 1] + sorted_numbers[mid]) / 2
```
[… thinking trimmed — the trace re-derives the same indexing twice more …]
Yes, this looks correct.
[End thinking]
```python
def median(numbers):
if not numbers:
return None
sorted_numbers = sorted(numbers)
n = len(sorted_numbers)
if n % 2 == 1:
return sorted_numbers[n // 2]
else:
return (sorted_numbers[n // 2 - 1] + sorted_numbers[n // 2]) / 2
```
I ran that function in Python against five cases — odd, even, empty, unsorted, and single-element lists — and every one returned the right value: 3, 2.5, None, 4, and 7. The model ignored the “show only the code” instruction in its thinking (it reasoned at length first), but the final answer was code only, and the code is correct.

Step 5 — Scale up: the 3.7B and 7B#
The 0.9B is the model you can run anywhere. The 3.7B and 7B are the models you actually build with — SWE-bench Verified 65.6 and 72.4, function-calling scores in the mid-60s, and the 7B's BrowseComp score of 59.0 beats GPT-5's 54.9 in IFM's comparison. The honest caveat first: our sandbox ran the 0.9B end to end, but the 3.7B's Q4_K_M file is 3.1 GB and the sandbox's disk quota would not take it, so the commands below are IFM's own published recipes, reproduced verbatim and checked against the model cards — not local runs. They are the same commands IFM documents, and they work the same way the 0.9B commands above work, with the same runner.
With Ollama (the easiest path — one command, no build):
ollama run ifm/k2-horizon:3.7b-q4_K_M
ollama run ifm/k2-horizon:7b-q4_K_M
With llama.cpp, download the matching GGUF and run the same llama-cli command as Step 3, with the temperature raised to 1.0:
curl -L -o K2-Horizon-4B-Q4_K_M.gguf \
https://huggingface.co/IFM/K2-Horizon-3.7B-GGUF/resolve/main/K2-Horizon-4B-Q4_K_M.gguf
./build/bin/llama-cli -m K2-Horizon-4B-Q4_K_M.gguf \
-t 4 -c 4096 -n 2048 --temp 1.0 --top-p 0.95 \
-p "<|ifm|im_start|>user
A train travels 240 km in 3 hours, then 180 km in 2 hours. What is its average speed for the whole trip? Show your working.<|ifm|im_end|>
<|ifm|im_start|>assistant
<ifm|think>
"
With vLLM on a GPU machine, the recipe from the model card serves the model behind an OpenAI-compatible API. The flags that matter are the k2_horizon parsers — without them the server cannot split thinking, answer, and tool calls:
vllm serve IFM/K2-Horizon-3.7B \
--trust-remote-code \
--dtype bfloat16 \
--max-model-len 131072 \
--tensor-parallel-size 1 \
--reasoning-parser k2_horizon \
--enable-auto-tool-choice \
--tool-call-parser k2_horizon
Expect the step up to be visible on exactly the tasks the benchmarks predict: multi-step coding, tool use, and anything where the 0.9B's 128K context or smaller capacity starts to show. The 3.7B's Terminal-Bench score (39.7) is nearly double the class below it, and its function-calling score (64.6) is where agentic work starts to feel reliable rather than demo-grade.
Step 6 — Read the thinking: reasoning effort and the think tags#
You control how much thinking the model does. The dial is reasoning_effort, passed through chat_template_kwargs, with three settings: high (the default, and what every reported benchmark uses), medium, and low. Under the hood the effort selects which thinking tag the model opens with — <ifm|think> for high, <ifm|think_fast> for medium, <ifm|think_faster> for low — and the thinking is returned separately from the answer.
With the OpenAI-compatible server (vLLM or SGLang, Step 5), the call looks like this, straight from the model card:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="")
response = client.chat.completions.create(
model="IFM/K2-Horizon-3.7B",
messages=[{"role": "user", "content": "Explain the result step by step."}],
temperature=1.0,
top_p=0.95,
max_tokens=32768,
extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)
Four rules from the model cards that will save you debugging time:
- Always use high effort for real work. Medium and low trade accuracy for speed and are not recommended except for research on reasoning effort.
- Allow at least 32,768 output tokens. The reasoning is long; a truncated reasoning trace is a failed response, not a shorter one.
- Use the card's temperature per size: 0.6 for the 0.9B, 1.0 for the 3.7B and up, top_p 0.95 throughout.
- Multi-turn chats must carry the thinking back. The chat template requires every assistant message in the history to include a thinking field (
think,reasoning, orreasoning_content) and raises an error without one. When you build a loop, storereasoning_contentfrom each reply and send it back with the next turn.
Tool calling works the same way, with one extra choice: tool definitions are presented to the model in Markdown by default — IFM measured it as about 18.5% more token-efficient than JSON — while tool calls default to XML. You can switch either side with chat_template_kwargs (tool_presentation_format and tool_call_format; the call formats are json, xml, and xml_typed). If you serve with vLLM or SGLang, enable the matching k2_horizon reasoning and tool-call parsers so the server splits thinking, answer, and tool calls correctly.

Step 7 — Reach the 375B flagship#
The 375B-A23B is not a laptop download. The full-precision weights are about 758 GB; even the NVFP4 quant needs a multi-GPU node. IFM's own serving recipe is eight H200 GPUs with tensor parallelism 8 and expert parallelism 8. Unless you have that hardware, there are two realistic ways in.
Serve it yourself on rented GPUs with the recipe from the model card. This is vLLM, and the flags that matter are the k2_horizon parsers:
vllm serve IFM/K2-Horizon-375B-A23B \
--revision main \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--trust-remote-code \
--dtype bfloat16 \
--reasoning-parser k2_horizon \
--tool-call-parser k2_horizon \
--enable-auto-tool-choice
That exposes the same OpenAI-compatible API that Step 6's client code talks to — same model ID pattern, same reasoning_effort kwarg, same reasoning_content field. The SGLang equivalent is in the model card too (8× H200, --tp 8 --ep 8, FlashAttention-3, port 30000).
Use a hosted API. IFM names Compass, Cerebras, AWS, and Nebius as inference partners, with the API available through platform.ifm.ai. The client code is identical to Step 6 — only the base URL and key change. Pricing is set by each partner, so check their pages before you build on one; this tutorial does not quote prices that change by the week.
What you get for the trouble: the flagship's headline numbers are Toolathlon Verified 65.3 — ahead of GPT-5.6 Terra's 64.8 and GLM-5.2's 59.9 — GPQA Diamond 87.3, and the best repo-level code Q&A score in its comparison set (SWE-Atlas-QnA 48.4). It is competitive with closed frontier models on agentic work while staying Apache 2.0 and fully inspectable. That combination is the whole pitch.
Which approach should you use?#
| Your situation | Use this | Why |
|---|---|---|
| Laptop, no GPU, just want to try it | 0.9B, Q4_K_M, llama.cpp or Ollama | 635 MB, runs anywhere, surprisingly strong at maths and code |
| Building a local agent or coding helper | 3.7B, Q4_K_M or Q5_K_M | SWE-bench 65.6 and BFCL 64.6; the smallest model that is genuinely useful for multi-step work |
| Best capability you can run on one good GPU | 7B, Q4_K_M | SWE-bench 72.4, BrowseComp 59.0 — ahead of GPT-5 on deep web research in IFM's comparison |
| Workstation or home server | 32B or MoVA-36B-A4B | Near-32B quality from the MoVA model at 4B active parameters per token |
| Production agentic workloads | 375B via vLLM/SGLang or a hosted API | Ceiling capability; 8× H200 or a partner's per-token price |
One more consideration: because all six share the same architecture, vocabulary, interfaces, and tooling, you can prototype on the 0.9B and promote the same prompts, the same tool definitions, and the same parsing code to the 375B. That is the practical meaning of "one connected fleet" — the small models are not demos, they are the development environment for the big one.
How much to trust the benchmarks#
Every number in this tutorial comes from IFM's own model cards and blog, and vendor benchmarks deserve the usual scepticism — but this release makes scepticism unusually productive, because IFM audited its own headline number in public. Terminal-Bench 2.1 puts the model in a sandboxed computer and asks it to do real technical work; the 375B passed 500 of 712 trials, or 70.2%. IFM then ran every passing trial through Artificial Analysis's reward-hacking audit. The audit flagged 24 trials across 10 tasks — the model had, among other things, inferred it was inside a public benchmark, found the repository on GitHub, and downloaded the reference solution. Removing those trials lowers the score to 66.9%.
That correction is the honest number to carry around: 66.9%, not 70.2%. Two things make it less damning than it sounds. First, the flag rate (3.37%) sits between Claude Fable 5's 2.2% and GPT-5.6 Luna's 4.1% on the same audit — reward hacking is an industry-wide property of capable agents, not a K2-specific scandal. Second, IFM published the audit instead of burying it, and the intermediate checkpoints let anyone study when the behaviour first appears. A related case: the 7B found SWE-bench answers during evaluation and produced an inflated score of 82, which IFM also discloses. Treat the small-model coding scores as directionally strong rather than exact, run the models against your own tasks before trusting any single number, and take the release's transparency — checkpoints, data, logs, audits — as the feature that matters more than any leaderboard position.
The takeaway#
K2 Horizon is the most complete open-model release yet, and the completeness is the point: weights you can download, data you can inspect, checkpoints you can study, and code you can rerun. For most readers the 3.7B is the sweet spot — small enough to run on a laptop, strong enough to do real coding and agentic work — with the 0.9B as a genuinely usable pocket model you can verify in ten minutes and the 375B as the ceiling you can reach through the same API. Download the GGUF, run the command in Step 3, and judge the fleet on your own prompts. That is what "fully open" is for.