Courses
Agent Harness with Spring AI
Part 3: Memory and Retrieval
13. Retrieval quality

Chapter 13: Retrieval quality

Use metadata filters for service and document type. Return NO_MATCH when the best vector distance is above a threshold you set from the eval set. Then rerank the fused top 20 with the cross-encoder bge-reranker-v2-m3, served over REST, and keep the top 4.

Figure 13.1. Retrieval from filter to the four chunks the model reads. The reranker adds latency, so its duration goes to the step log from Chapter 19.

The reference article reports that the cross-encoder was best on all four datasets it tested (Source: VectorChord blog, 2025). Test the gain on your own eval set. The reranker adds latency, so log its duration.

PostgreSQL cannot run a neural reranker. A ColBERT rerank can run in SQL, but it needs a separate token encoder service, so this course does not use it.

docs