Dev Tools · 2h ago
Observability pipeline on S3 alone cuts costs by 87%
A developer replaced a Kafka+Flink+OTel ingestion pipeline costing $700-800/month with a single binary using S3 for storage, write-ahead log, and Iceberg catalog, at $100/month. The design eliminates local disks and coordination services, relying on S3's fault tolerance and scalability. It targets observability workloads where sequential writes and analytical reads tolerate S3's latency trade-offs.
Meridian48 take
The approach is clever but unproven at scale; S3 API costs and latency could bite under high-cardinality or bursty traffic.
Read the full reporting
Can you build observability ingestion on S3 alone — no Kafka, no disks, no coordination layer? →
DEV Community
observabilitys3-ingestion