Dev Tools · 1h ago
AI Streaming Bug Lets Responses Deliver While Billing Logs Zero Tokens
A streaming failure called "delivered but unbilled" occurs when AI responses render fully to users but the terminal usage frame is dropped, causing clients to log 0 output tokens for a paid call. A developer created stream_billing_gate.py, a post-hoc reconciliation tool that detects when text was delivered but billing shows zero. The tool uses word-count as a conservative floor to flag discrepancies, without intercepting live traffic or requiring API keys.
Meridian48 take
The bug highlights a fundamental design flaw in streaming architectures where billing data arrives after content, making silent revenue loss inevitable without offline reconciliation tools.
ai-streamingbilling-reconciliation