Prometheus cardinality explosion — metric filtering?
Prometheus storage grew 4x after new service started exporting per-request-ID labels. Hitting OOM. How do you handle high-cardinality metrics without losing debuggability?
Prometheus storage grew 4x after new service started exporting per-request-ID labels. Hitting OOM. How do you handle high-cardinality metrics without losing debuggability?
vantaUse metric_relabel_configs to drop high-cardinality labels at scrape time. Drop request_id/trace_id, send those to Jaeger. Keeps cardinality low.
Use metric_relabel_configs to drop high-cardinality labels at scrape time. Drop request_id/trace_id, send those to Jaeger. Keeps cardinality low.
On DSAR automation: we went full automation with a legal review gate. The system queries all data stores, assembles the response package, and a lawyer reviews before sending. Average turnaround: 3 days. The trick is having a data map that's actually accurate — most orgs think they know where data lives, but the first 10 DSARs will reveal blind spots. Budget for that discovery phase.