Dev Tools · 3h ago
Smolagents' memory replay causes O(n²) token cost per run
Hugging Face's smolagents framework resends its entire conversation history to the LLM at every step, leading to quadratic token consumption. A 20-step task can cost far more than 20 times a single step's tokens. The author provides a code-level audit and a one-hook fix.
Meridian48 take
The article highlights a hidden inefficiency in an otherwise elegant framework, but the fix is straightforward; the real issue is that many developers may not realize their token bills are growing quadratically.
Read the full reporting
smolagents replays its whole memory every step: the O(n ) token bill nobody mentions →
DEV Community
smolagentstoken-efficiency