eBPF-based network policy (Cilium) vs iptables (Calico): real-world rule-count limits?
Running a 120-node EKS cluster and considering migrating from Calico to Cilium for eBPF dataplane. Current pain point: Calico iptables chains are growing unbounded. We have ~4,200 network policy rules and rule evaluation on node startup takes 35-40 seconds. Some nodes fail health checks during rolling updates because kube-proxy + Calico initialization exceeds the readiness probe timeout. Cilium docs claim O(1) policy evaluation regardless of rule count, but I've seen reports of eBPF map size limits (65536 entries per map) becoming a bottleneck at scale. Questions: 1. Anyone running Cilium in eBPF mode with 3,000+ network policies? 2. What's your actual rule-count ceiling before hitting eBPF map limits? 3. Did you observe any kernel version dependencies? We're on 5.15 (Amazon Linux 2023) — eBPF CO-RE support is supposed to be stable there. Not asking for 'which is better' — just hard limits and gotchas from production deployments.