When does retrieval augmentation hurt more than help in RAG pipelines?

We've been benchmarking a RAG pipeline for technical documentation Q&A and found a counterintuitive result: adding more retrieved chunks (k > 5) actually degrades answer quality on specific, narrow questions. The model seems to get distracted by tangential context. Has anyone systematically measured the sweet spot for context window utilization in retrieval-augmented generation? We're using a 128K context model and wondering whether selective pruning (e.g., only keeping chunks with high semantic similarity + exact keyword overlap) outperforms naive top-k retrieval. Also interested in whether you've tried graph-based retrieval (entity-relation graphs) vs pure vector similarity for technical domains.