Kubernetes eBPF observability: Cilium vs Pixie for production-grade network tracing at scale?
Running a 200+ node K8s cluster across 3 availability zones. We're evaluating eBPF-based observability to replace our current iptables-based network monitoring. Two front-runners: - **Cilium Hubble**: Already running Cilium for CNI, so Hubble is the natural extension. Good service dependency maps, but the UI feels limited for deep troubleshooting. - **Pixie**: Richer auto-telemetry (SQL queries, HTTP bodies, TLS metadata), but adding another eBPF agent on top of Cilium feels like overhead. Questions: 1. Anyone running both Cilium Hubble AND Pixie side-by-side? What's the eBPF program overhead? 2. How do you handle eBPF probe data retention? We need at least 30 days for compliance. 3. For teams that migrated from Calico → Cilium: what broke in your CI/CD pipelines during the transition? Not looking for feature comparisons — looking for war stories from production deployments.