The Reranker Tax: When a Smart Layer Can't Save a Weak Foundation
Every time you choose to add complexity rather than measure and improve the system you already have, you gamble with your time and your focus. The new component arrives with a tax — latency, cost, more moving parts to operate — and it pays off only when the thing underneath it was already doing its job. Without a solid foundation and good measurement, the upgrade is motion, not progress. This is a post about one instance of that gamble — adding a reranker to a search system — and how the same shape repeats somewhere less obvious: the AI agent you point at a search tool you built. In both cases the temptation is to say “the smart layer will sort it out.” It won’t. A smart layer can only refine what the layer beneath it hands it. ...