Part 3: Memory and Retrieval | Garage Education

Chapter 10: Short-term memory with Spring AI Session

Sat, 10 Oct 2026 00:00:00 +0000

Spring AI Session stores every message and tool call in PostgreSQL. Spring AI’s JDBC chat memory drops tool call messages, so this course does not use it. The workflow id is the session id.

sessions.appendMessage(sessionId, new UserMessage(question));
List<Message> history = sessions.getMessages(sessionId);

Figure 10.1. The database keeps everything. The harness decides how much of it to send. The figure leaves out how the window is chosen; the chapter leaves that open.

The full history stays in the database. The window sent to the model is a separate choice. Verify the bean name and the create signature in the Session README.

Chapter 11: Embeddings and hybrid storage in PostgreSQL

Sat, 10 Oct 2026 00:00:00 +0000

Use one PostgreSQL database with VectorChord for vectors, and its BM25 extension for keywords. The table holds the text, the metadata, the embedding, and the keyword vector.

CREATE TABLE IF NOT EXISTS corpus (
 id TEXT PRIMARY KEY,
 service TEXT NOT NULL,
 doc_type TEXT NOT NULL,
 source TEXT NOT NULL,
 text TEXT NOT NULL,
 emb vector(1024),
 bm25 bm25vector
);

BGE-M3 runs in LM Studio and produces 1024-dimension vectors. Split documents into chunks of about 1200 characters with 150 overlapping. Embed each chunk, then insert it. A startup check confirms the vector size matches the table.

Figure 11.1. The ingest path for one document. The figure leaves out how the bm25vector column is filled; see the Verify note below.

Figure 11.2. Overlap keeps a sentence that crosses a chunk border searchable from both sides. Not to scale.

Verify: Create the BM25 index with the statement in the VectorChord-BM25 repository. The reference article does not show it.

Chapter 12: Hybrid retrieval inside the loop

Sat, 10 Oct 2026 00:00:00 +0000

Run two searches and merge them. The vector search returns the 20 closest chunks. The keyword search returns the 20 best BM25 matches. Reciprocal Rank Fusion merges the two lists by rank. Each chunk scores the sum of 1 divided by 60 plus its rank in each list.

Figure 12.1. Hybrid retrieval. RRF uses rank positions, not raw scores, so a vector distance and a BM25 score never need to be compared. Chapter 13 adds filters and a reranker.

Worked example of the RRF formula. A chunk ranked 1st in the vector list and 3rd in the keyword list scores 1/61 + 1/63 ≈ 0.0323. A chunk ranked 2nd in the vector list only scores 1/62 ≈ 0.0161. Appearing in both lists wins.

Expose the merged search as a tool. Require a citation for each claim. If the answer has no citation and does not say no match, send one correction. Stop if it still has none.

Figure 12.2. One correction, never a second. The chapter does not name the stop reason for an uncited answer.

Chapter 13: Retrieval quality

Sat, 10 Oct 2026 00:00:00 +0000

Use metadata filters for service and document type. Return NO_MATCH when the best vector distance is above a threshold you set from the eval set. Then rerank the fused top 20 with the cross-encoder bge-reranker-v2-m3, served over REST, and keep the top 4.

Figure 13.1. Retrieval from filter to the four chunks the model reads. The reranker adds latency, so its duration goes to the step log from Chapter 19.

The reference article reports that the cross-encoder was best on all four datasets it tested (Source: VectorChord blog, 2025). Test the gain on your own eval set. The reranker adds latency, so log its duration.

PostgreSQL cannot run a neural reranker. A ColBERT rerank can run in SQL, but it needs a separate token encoder service, so this course does not use it.