AI · 1h ago
RAG Abstention Threshold Fails Silently When Embedding Model Changes
A developer found that a hard-coded cosine similarity threshold of 0.50 for abstention in a RAG system worked only by accident with one embedding model. Switching to bge-small caused every unanswerable query to be confidently answered, as the threshold never fired. The post shows that cosine similarity distributions vary widely across embedders, making fixed thresholds dangerous for safety guards.
Meridian48 take
This is a cautionary tale for any RAG system relying on magic-number thresholds: they can silently break when the embedding model changes, undermining safety guarantees.
Read the full reporting
Retrieval-Augmented Self-Recall — Part 5: The Gap Threshold That Didn't Transfer →
DEV Community
ragabstention-threshold