Tune cost + quality with an eval set

You know your LLM feature costs too much, so you reach for the obvious levers — a cheaper model, a shorter prompt, more caching. Each one cuts the bill. But you have no way to tell whether it also quietly broke the answers, so you ship the cut on faith and find out from a user. This builds a small eval set first, then applies the cost levers one at a time and re-scores after each — so every cut is one you can see held quality, not one you hope did.