Dev Tools · 2h ago
How a fleet IoT reconnect storm exposed a hidden pipeline bottleneck
When thousands of IoT devices reconnected simultaneously, a managed MQTT broker handled the burst, but a downstream Kinesis-to-Lambda pipeline saw iterator age spike to 45 minutes. A single poison record froze an entire shard, and scaling out failed because each Lambda invocation wasted 30-50ms on MySQL handshakes. The real fix required addressing per-record processing cost, not just adding capacity.
Meridian48 take
A cautionary tale that scaling out is useless when the bottleneck is per-request overhead — a lesson many serverless architectures learn the hard way.
iot-pipelineserverless-bottleneck