How to Choose a Retrieval Cutoff

A retrieval system can always return something. The closest match in the table may still be useless. The cutoff is the line that decides whether a source-text chunk is good enough to reach the model.

This is the deeper version of the cutoff step from the RAG guide. The short version is simple: choose a floor from real questions, pass the similarity score along, and refuse when the source is too weak.

For definitions of RAG and hallucination before this workflow, start with the AI buzzwords guide.

Start with the risk

What a wrong answer costs depends on who reads it.

Where the answer lands	What a wrong one costs	Who finds out, and when
An internal draft you'd rewrite anyway	A few minutes	You, on the spot
A number inside a quote you send	The time needed to correct the quote and explain it	The customer, after they've read it
A refund quoted off the policy you retired	Money you may have to return under the quoted policy	The customer, who will hold you to it

The cutoff should be stricter when the wrong answer leaves the team, changes money, or quotes a policy someone will hold you to.

Build the calibration rows

Take real questions from the places people already ask them and run each one through the same retrieval query your application uses.

For each question, take the row with the highest similarity score. Read its content, use its document_id to open the source document, and decide whether that source-text chunk would help answer the question.

Then record three things:

1. Question you asked. Copy the wording people used.
2. Similarity score of the top search result. Record the score returned by the retrieval query.
3. Did the source-text chunk help answer the question? Mark yes or no after checking the source file.

Those rows are your calibration set. Do not use made-up questions for this step. They will be cleaner than the questions the system has to answer later.

Run enough questions

The number of test questions depends on how large and varied your corpus is. Choose a margin of error, use a proportion sample-size calculator, and run at least that many real questions. [1]

Do not borrow the sample size from another system. A small, stable policy folder and a messy support archive are different measurement problems.

If you do not know what margin of error to use, start by deciding what kind of miss you are trying to prevent. A draft that a teammate rewrites can tolerate a rougher estimate than a customer-facing answer about refunds, prices, or legal terms.

Sort by score

Once the rows are recorded, sort them by similarity score. The useful matches should sit above the unrelated ones.

Example calibration rows:

Rank	Score	Source label
1	0.91	useful
2	0.62	useful
3	0.31	unrelated

In that set, the cutoff belongs somewhere below 0.62 and above 0.31. The exact number is a product decision: a lower floor lets more material through, and a higher floor refuses more often.

If useful and unrelated rows overlap, the sample is not ready. Keep collecting questions until the boundary becomes stable, or change the retrieval setup before choosing a floor.

If the scores are messy

A clean calibration has useful results clustering above unrelated ones. Messy results usually mean the cutoff is being asked to solve a retrieval problem.

The chunk is too broad. A long source-text chunk can contain one useful sentence and a lot of unrelated material. It may score well enough to pass, then give the model context that muddies the answer.

The chunk lost its label. A table row, price, policy exception, or checklist item can be true and still useless if the heading that explains it was split away during chunking.

The question is doing two jobs. "Can I refund this and what should I say?" may need a policy lookup and a draft response. Calibrate the source lookup first. Let the model write only after the right source is found.

Fix those problems before setting the floor. A stricter cutoff will not repair bad chunks, missing labels, or questions that need to be split.

Turn the cutoff into a refusal rule

The cutoff only matters if it controls the next step. A retrieval query that sorts by distance and returns the top five rows will still hand weak matches to the model.

The query needs a floor: [2]

SELECT content, 1 - (embedding <=> %s) AS similarity
FROM chunks
WHERE 1 - (embedding <=> %s) >= %s
ORDER BY embedding <=> %s
LIMIT %s

When no row clears the floor, the system refuses. It should say it could not find a useful source instead of writing from weak evidence.

Re-run the calibration

The cutoff is not permanent. Re-run the measurement when:

• the document set changes
• the questions start coming from a new place
• the embedding model changes
• the chunking strategy changes
• reviewers keep finding weak matches that cleared the floor

A cutoff chosen from last quarter's documents is a guess once the corpus, questions, or embedding model changes.

Quick recap

• A retrieval cutoff is the minimum similarity score a source-text chunk has to clear.
• Build the cutoff from real questions and checked source files.
• Record the question, the top score, and whether the source helped.
• Size the test before you start, then run the full count on real questions.
• Sort by score. The cutoff belongs where useful matches give way to unrelated ones.
• If useful and unrelated rows overlap, collect more questions or change the retrieval setup.
• The floor has to control the query. If no row clears it, the system refuses.

Start Here

The intake at daisyguti.ai/work-with-me is about nine questions and takes a few minutes. My AI assistant reads it and replies with whether a document-answering system like this, or a custom workflow automation, would fit.

Sources

1. OpenStax Statistics, "A Population Proportion" - https://openstax.org/books/statistics/pages/8-3-a-population-proportion - sample size for estimating a proportion is calculated from the confidence level, margin of error, and estimated proportion.
2. pgvector, open-source vector search for Postgres - https://github.com/pgvector/pgvector - shows vector distance operators and nearest-neighbor queries with ORDER BY embedding <=> ... LIMIT ...; this guide adds the similarity floor in WHERE.