HUGGING FACE GITHUB
PAPER WHERE THE BITS GO · NOW ON ZENODO

THE GEHIRNS DENSITY PROGRAM

CAPABILITY PER GB

GEHIRNS makes open models smaller by measuring where their bits matter, then putting them there. First release: MAGI-1.0, Qwen3.5-4B in 1.8 GB, with no training.

MEET MAGI-1.0 READ THE PAPER

FIG. 0 THE MARK · 6 PIECES ON A 25-UNIT GRID

3.4X

LESS GPU MEMORY

2.65 GB VS 8.98 GB

96%

OF THE BENCHMARK SCORE KEPT

80.6% VS 84.0%, 12 SUITES

2.7X

FASTER DECODING

181 VS 66 TOKENS/S

3.0X

CAPABILITY PER GB

DENSITY 0.893 VS 0.294

MAGI-1.0 with an 8-bit KV cache vs Qwen3.5-4B BF16 · same RTX 5070 (12 GB) · same 2,313-task benchmark, thinking on

01 RELEASE

MAGI 1.0

Qwen3.5-4B in 1.8 GB, with no training. The weights are the Qwen team’s; we only chose how many bits each part of the model gets, in standard GGUF formats. It runs unmodified in llama.cpp and Ollama.

BASE
QWEN3.5-4B
FILE
1.79 GB GGUF
AVERAGE
3.65 BITS / WEIGHT
TRAINING
NONE
RUNS IN
LLAMA.CPP · OLLAMA
LICENSE
APACHE-2.0
ollama run hf.co/GEHIRNS/magi-1.0
GET THE WEIGHTS RECIPE + DATA
FIG. 1 GPU MEMORY AT 32K CONTEXT · AREA TO SCALE
8.98 GB
QWEN3.5-4B BF16
3.78 GB
Q4_K_M
2.65 GB
MAGI-1.0
+ 8-BIT KV

With the default 16-bit cache, MAGI-1.0 takes 3.02 GB. Sizes are what Ollama reports in use, not counting the CUDA context (about 1.05 GB for every model). One GB here is 1,073,741,824 bytes (a GiB), as in the paper.

02 HOW IT WORKS

THE BITS GOWHERE THE MODELNEEDS THEM.

  1. 01

    MEASURE

    Lower one kind of tensor at a time and count how often the model’s top next-token choice stops matching full precision. The most sensitive tensors are small: attention K, V and output, and the DeltaNet output projection.

  2. 02

    ALLOCATE

    Spend a fixed size budget where the measured damage per byte is largest. The sensitive projections and the embedding get 4.5 to 8.5 bits; the large FFN matrices get 3.06. Average: 3.65 bits per weight.

  3. 03

    CHECK

    Screen candidates on 126 reasoning tasks, 86 of which a 2-bit file fails and a 4-bit file solves. Against Q2_K, the gain is almost all fewer runaway reasoning loops.

BITS / WEIGHT16.00
WEIGHTS7.84 GB
TENSORPOINT AT ONE

FAILED TASKS THAT END IN A RUNAWAY LOOP

Q2_K59%
MAGI-1.021%

337 OF 575 FAILURES VS 99 OF 463 · FULL 2,313-TASK RUN

8.50 · Q8_0 4.50 · Q4_K 3.06 · IQ3_XXS 16 · BF16, BEFORE 249 QUANTIZED TENSORS · WIDTH = PARAMETERS · 177 TINY F32 TENSORS NOT DRAWN

03 RESULTS

SAME GPU.SAME 2,313 TASKS.

One RTX 5070 (12 GB) · 12 suites: coding, reasoning, knowledge, math and tool calls · thinking on, 4,096-token budget · greedy decoding · 32K context

How many times the error rate halves for each GB of GPU memory: −log2(1 − score) / GB. Higher is better.

Memory in GB of 1,073,741,824 bytes (GiB), as in the paper. MAGI-BETA-1’s memory was read from Ollama’s rounded display, so it is accurate to about 0.05 GB.

THE HONEST TRADE-OFF

Q4_K_M scores within 0.2 points of BF16 and 2.85 above MAGI-1.0, but needs 0.76 GB more memory: 1.13 GB more than MAGI-1.0 with the 8-bit cache.

IF ACCURACY MATTERS MOST

USE Q4_K_M

IF MEMORY IS THE LIMIT

USE MAGI-1.0

04 MODELS

EVERY BUILD STAYSON THE RECORD.

Retired experiments stay online with their scores, including the ones that did not beat their base.

RECOMMENDED

MAGI-1.0

QWEN3.5-4B · 1.79 GB GGUF

Sensitivity-allocated mixed precision. No training. 0.893 capability per GB with an 8-bit KV cache.

HUGGING FACE EXPERIMENTAL

MAGI-BETA-1

QWEN3.5-4B · 1.82 GB GGUF

Same recipe, full-attention key, value and output at Q6_K. One task short of the screen’s bar, so it stays experimental.

HUGGING FACE RETIRED

MAGI-ALPHA-1

QWEN3.5-9B · 5.24 GB GGUF

Supervised fine-tune for code. 49/150 on LiveCodeBench against 50/150 for its base: no gain. Kept for the record.

HUGGING FACE RETIRED

MAGI-ALPHA-2

QWEN3.5-9B · 5.24 GB GGUF

Second fine-tuning diet. 48/150, a dead heat with alpha-1. The fine-tuning line stopped here.

HUGGING FACE

05 RUN IT

ONE LINE.NO CUSTOM KERNELS.

Standard GGUF formats only. The benchmarks used a 32K context and no system prompt.

Straight from Hugging Face

ollama run hf.co/GEHIRNS/magi-1.0

Match the benchmark’s 32K context, inside the session

/set parameter num_ctx 32768

8-bit KV cache: start the server with

OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

06 PAPER

WHERE THEBITS GO

Sensitivity-allocated mixed precision for a 4B reasoning model on a 12 GB consumer GPU.

EASON SUN · 2026 · ZENODO · DOI 10.5281/ZENODO.23630199

READ THE PDF DOI
BIBTEX
@misc{sun2026wherethebitsgo,
  title     = {Where the Bits Go: Sensitivity-Allocated
               Mixed Precision for a 4B Reasoning Model
               on a 12 GB Consumer GPU},
  author    = {Sun, Eason},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23630199},
  url       = {https://doi.org/10.5281/zenodo.23630199}
}
  • REBUILD One llama-quantize call per model, from Qwen3.5-4B BF16. A rebuild matches the release byte for byte.
  • VERIFY A script checks every tensor’s type and data hash against the release.
  • CHECK The data behind every table and figure, with per-task pass, fail and loop flags.