Consistency Gap under Retrieval Corruption: Stress-Testing RAG Robustness with Adversarial Evidence

Abstract

Retrieval-Augmented Generation (RAG) significantly enhances the factual answering capability and contextual awareness of large language models by dynamically incorporating external knowledge sources into the generation process. However, the overall reliability of these systems critically depends on the accuracy and quality of the documents provided during the initial retrieval stage. When retrieval results are inadvertently or maliciously contaminated with adversarial evidence—defined as documents that are highly relevant to the user's query but contain fundamentally incorrect information—the model's generated content frequently exhibits systematic drift. This phenomenon forms what is formally termed a "consistency gap," wherein the model struggles to reconcile its internal parametric knowledge with the flawed external context. This paper investigates retrieval contamination as a primary research setting and constructs a comprehensive stress-testing framework based on adversarial evidence to systematically examine how erroneous documents affect the logical consistency of model outputs. Extensive empirical evaluations reveal several critical findings: under conditions of retrieval contamination, models exhibit a highly asymmetric sensitivity to incorrect evidence, often prioritizing flawed retrieved text over accurate internal knowledge. Furthermore, different decoding strategies demonstrate highly differentiated robustness characteristics, suggesting that generation parameters can mitigate or exacerbate the issue. Finally, the magnitude of the observed consistency gap is significantly correlated with an increased risk of severe hallucination. Ultimately, this work provides new analytical perspectives and robust testing methodologies for evaluating, benchmarking, and improving the resilience of retrieval-augmented generation systems in real-world applications.