Guard, Embed, Cascade: A Framework for Cutting LLM Costs Without Cutting Quality

Most LLM pipelines are paying premium-model prices for work that doesn't need a premium model – sometimes for work that doesn't need a model at all.

Guard, Embed, and Cascade are the three levers for fixing that. They aren't interchangeable, and they aren't equally powerful.

• Guard and Embed are where the real money is – they remove work instead of discounting it.
• Cascade comes last, because routing to a cheaper model only makes sense once you've confirmed the model needed to run at all.

Build in that order: Guard first, Embed second, Cascade third.

Guard – stop the call before it happens

A guard is a cheap check sitting in front of an expensive one. Its only job is to decide: does this request even need the LLM behind it?

Most flows call the model unconditionally, on every single request, regardless of whether the request is one the model actually needs to handle: A chatbot that handles every question, even simple ones like "when are you open"; A moderation prompt that reviews every post, including the ones that are obviously spam; A summarizer that fires on every document, including the one-liners that don't need summarizing.

Each of those is a case where a much smaller check could have absorbed most of the volume.

A guard is usually one of:

1. A rules check – length, keyword, regex, or simple heuristic that catches the obvious cases
2. A cheap classifier – a small model or traditional ML classifier trained to catch a narrow, well-defined split
3. A cache or lookup – has this exact (or near-identical) input already been answered?

None of these need to be perfect. A guard that correctly filters 60% of traffic away from an expensive call is a 60% saving on that call, even if the other 40% still needs the full model. You're not trying to replace the LLM's judgment – you're trying to avoid asking for it when the answer was never in question.

The tell that you need a guard: look at your highest-volume flow and ask what fraction of requests are "easy." If a meaningful chunk of your traffic is clearly-not-spam, clearly-in-policy, or clearly-already-seen, you're paying full price for a decision that didn't require full price.

In Kuverly, a single guard action contributed to cutting the daily LLM bill by 90%.

Embed – replace the step, not the flow

Embedding, here, means swapping a specific action inside your flow for an embeddings-based or traditional ML approach, rather than routing that action to any LLM at all.

A handful of patterns show up again and again: classify, recommend, compare, rank, cluster, deduplicate, detect, retrieve. If an action fits one of those patterns, it's very likely solvable without a model call at inference time.

The reason this saves so much isn't just per-call cost – it's that embeddings and traditional ML models are usually orders of magnitude cheaper to run than even a small LLM, and they don't carry prompt or reasoning overhead. A step you replace this way doesn't get cheaper. It gets removed, permanently, from your LLM bill.

Where this tends to show up:

1. Retrieval – finding the relevant document, ticket, or product before generation, instead of asking the LLM to search
2. Similarity or dedup – catching near-identical support tickets, reviews, or content without asking the model to compare them Routing – deciding which downstream flow or team a request belongs to, as a pre-step instead of a prompt instruction
3. Scoring or ranking – ordering candidates by relevance or quality before anything gets generated

The trade you're making is upfront engineering time for a permanent removal. But once it's built, embedding solutions run at a fraction of the cost on every future call, forever, which is exactly why it outranks Cascade.

Back on Kuverly, topic matching and article suggestion were both originally LLM calls, and both turned out to be similarity problems, not reasoning problems. Moving them to embedding distance dropped their cost to a fraction of what the LLM pipeline was charging, for the same result.

Cascade – now let the model earn its place

Once Guard has filtered out what didn't need a model, and Embed has removed what didn't need an LLM specifically, what's left is the work that genuinely requires language understanding or generation. This is where Cascade belongs.

Cascading means routing each remaining action to the smallest model tier that can comfortably handle it. Not every remaining call needs your best model. Simple extraction, straightforward formatting, low-stakes classification-by-generation – these often do fine on a smaller tier. Complex reasoning, nuanced judgment calls, or anything where an error is costly should stay on the bigger model.

A cascade in practice usually looks like:

1. Try the cheap model first, with a way to detect low-confidence or malformed output
2. Escalate to the next tier only when the cheap model's output fails that check
3. Reserve your best model for the fraction of cases that reach it

This is a real saving, and for high-volume flows it can be substantial. But it's a discount on a cost that will continue to grow with every new user.

That's why it comes last: it's the easiest improvement to build once you're sure the call needs to exist, and the least durable one in the long run, since model pricing and capability shift underneath you (and seemingly always upwards) in ways a removed step never has to worry about.

Putting it together

The three levers compound, but not equally. Guard and Embed both attack whether a call happens at all – one at the request level, one at the action level. Cascade attacks what a call happens on, and it only earns its place once the first two have already been applied.

Build them in that order, and revisit them in that order too: a Guard rule or an Embed model that's drifted out of date will always be a bigger miss than a suboptimal model tier.

If you're not sure which of your flows to start with, a short pipeline audit is usually enough to surface the highest-cost candidates for each of these three levers.

If you'd like advice for your specific situation, you can email me and we can set up a meeting to discuss the opportunities present for you and your business.