Dev Tools · 1h ago
Running an LLM in production: real costs and caching tricks
A developer shares hard-won lessons from building a consumer app that relies on LLM calls for every user action. Semantic caching cut costs by 40-50% without degrading answer quality. The post details pitfalls like embedding normalization and perceptual hashing for images.
Meridian48 take
The piece is a practical antidote to demo-ware, but its caching strategies are well-known in production ML—the real value is in the concrete code and cost numbers.
llm-production-costssemantic-caching