flight 03 · 2026 · live at herodthegreater.github.io/pigeon
Pigeon
A website where you shrink a real image model with your own hands and watch what it costs. Underneath it is a method I came up with and call reverse training.
Pigeon is a small image model with one job: is there a bird in this photo, and where. On the site you quantize it, prune it and palettize it in your browser, and after every change it scores itself again on 500 real photos and reports its accuracy, how often its heatmap points at the actual bird, and its true size in bytes. The mascot is the scoreboard. Keep the model accurate and it slims from a rock pigeon to a dove, a swift, a hummingbird. Push too far and it starves, and if you break it before it got any lighter you get a dodo.
The pigeon flying around this page is that mascot. The demo beside this text uses the same birds, the same rule, on a toy network that trains in your browser, so you can feel the two findings the project is built on: pruning is a cliff you recover from by retraining, and quantizing is a slope that gets steep fast.
the method
Reverse training
Training grows a model up from noise until a skill appears. Reverse training runs it backwards: start from a model that already works, and take away everything the one job you want doesn't need.
- Cut. Drop precision, or remove whole channels.
- Measure the real accuracy of the one task you are keeping, never a proxy.
- Regrow. Fine-tune the wounded model so what survives re-specializes around the task.
- Repeat until the task is about to break.
The finding that shaped it: precision has slack, structure does not. Rounding weights to fewer bits degrades gracefully. Removing channels is a cliff, and regrowing is the other half of every cut.
- The cliff
- A one-shot cut of 5% of a YOLOv8n bird detector's channels, with no regrowth: bird mAP@50 falls from 0.4284 to 0.000.
- Small cuts, regrown
- Three cuts of about 7% each, 15 epochs of regrowth after every one: 0.2874, two thirds of native, with 31% fewer parameters. One 20% cut followed by 40 epochs: 0.0706.
- Where the levels sit
- At 4 bits, integer steps with one scale per channel keep 16% of native; FP4 levels, which crowd near zero where the weights are, keep 79% at the same bit count. Protect the three earliest stages at 8 bits and it is 92%.
- Regrow with the quantizer in the loop
- A 3.16-bit recipe that scored 0.003 after rounding came back to 70% of native in three CPU epochs of quantization-aware regrowth.
YOLOv8n, bird mAP@50. The prune rows are on a 125-image COCO bird set; the 4-bit rows are on a stand-in bird set and are relative. From the cockpit's notes in the Pigeon repo, which also say the regrown numbers lean optimistic because the eval set doubles as the fine-tuning validation split.
What's on the site
- The Coop: the sandbox. Pick a precision, a pruning pattern or a color palette for the weights, stack them, and watch the bird and a chart of every setting you have tried. A second cage holds a YOLOv8n detector that draws a box around every bird it finds, with its own cuts: one precision for all its weights, and a channel cut per stage. Every bird you hatch goes on your life list.
- The Field Guide: 43 compression methods, each with a write-up and the real code. 31 run in your browser on a toy network, 3 run in the reverse-training cockpit on your own machine, and 9 that need text, tokens or a KV cache are explained as reference.
- Flight School: today a reading map of 12 chapters in order, plus 10 coding lessons where you write the piece of the algorithm yourself in plain numpy, from softmax to GPTQ's error-compensation loop, and check it against tests in your browser.
- Results: 28 experiments on the lab's toy problem, each repeated with two or three seeds, in a miniature world where the best possible classifier is known exactly, so every method's cost is measured against a true ceiling.
Measured on the real Pigeon
- Specializing
- MobileNetV3-Small, 10.17 MB as a 1000-class model. Dropping the classifier it no longer needs removes 63% of its parameters; accuracy on birds rises 0.8 points. The size is the win.
- Full precision
- 3.69 MB · 94.0% accuracy · 98.0% pointing
- Best recipe on record
- 737 KB, 5× smaller · 94.4% accuracy. 8-bit with one scale per channel, and the squeeze-and-excitation layers (half the weights) at 4 bits.
- The trap
- 8-bit with one scale per tensor collapses to 50%. One scale per channel keeps 94.8%.
- Where it is fragile
- 4-bit on the squeeze-and-excitation layers, half the weights, costs nothing. 4-bit on the depthwise convolutions, 7% of the weights, drops it to 58%.
- The ceiling so far
- No recipe on record stays healthy at 7× smaller, where the swallow starts. That likely takes retraining, which the Coop's tools never do.
Scored on 500 Open Images photos on an M4; the numbers come from the project's findings notes.
How I built it
PyTorch trains it and exports it to ONNX with BatchNorm folded and every weight tensor as a graph input, so the page holds the weights as typed arrays, edits them with the method you picked, and re-runs the model in ONNX Runtime Web inside a worker, with no re-export per slider move. The size shown is computed from the real encoding: bits, scales, zero points, codebooks and sparsity metadata. The detector cage runs on TensorFlow.js; the lab's numpy runs in the browser through Pyodide. Astro builds the site and GitHub Actions deploys it on every push.
PyTorchONNX Runtime WebAstroNumPyPyodideYOLOv8n
Open the Coop Visit the site
