Dev Tools · 1h ago
Colibri Runs 744B-Parameter AI Model on a Laptop
Colibri is a C inference engine that runs GLM-5.2, a 744B-parameter MoE model, on a laptop with 25GB RAM by streaming experts from disk. It keeps dense layers in RAM (~10GB) and loads only active experts on demand, trading disk I/O for memory. The model on disk is ~370GB, requires an NVMe SSD, and achieves 2.2–2.8 tokens per forward with speculative decoding.
Meridian48 take
This is a clever engineering hack that makes a frontier model accessible on consumer hardware, but the extreme disk I/O and cold-start latency mean it's more a proof of concept than a practical daily driver.
colibriglm-5.2