LLM-as-judge flakiness undermines CI gates; here's how to fix it

A developer found that using an LLM as a judge for a CI gate produced inconsistent scores on identical inputs, ranging from 0.79 to 0.82 across three runs. The jitter stems from sampling temperature, model version drift, prompt ambiguity, and tie-breaking. Solutions include setting temperature to 0, pinning model versions, averaging over multiple runs, and quantizing scores.