Courses
Agent Harness with Spring AI
Part 5: Observability, Evaluation, and Cost
22. Cost optimization

Chapter 22: Cost optimization

Measure tokens per run first. Then add a cache with Spring Cache and Caffeine, keyed by error fingerprint, with a ten-minute expiry. Clear it when a runbook changes. Redis can replace Caffeine later with no code change.

@Cacheable(cacheNames = "triage", key = "#fingerprint", unless = "#result == null")
public TriageDecision triage(String fingerprint, String stackTrace) { ... }
Figure 22.1. A repeat fingerprint within ten minutes costs no tokens. A null result is never cached, so a failed triage is retried.

Provider prompt caching does not apply to LM Studio, so skip that step. Each step must lower the token count in the eval run. Remove any step that does not.

Figure 22.2. Every cost step has to show its saving in the eval run.
docs