Dev Tools · 1h ago
MoE model benchmark gains 32% throughput by reordering expert weights
A developer's benchmark showed reordering expert weights in a Mixture-of-Experts model by co-activation reduced disk reads 2.23× on an 80B model. Independent testers then measured +32.3% decode throughput and -26.3% time-to-first-token on a 235B model running on a 48GB MacBook. The process involved sharing data with inference engine maintainers, who refuted two of the original three pitches with their own measurements.
Meridian48 take
The real story isn't the speedup—it's the open, adversarial validation process that turned a solo benchmark into a reproducible result across multiple engines.
moe-modelinference-optimization