How Do You Stop an AI Knowledge Base From Inventing Answers?
An AI knowledge base invents answers when retrieval fails and the model fills the gap. Judge a vendor on retrieval quality measurement, citation behaviour and access control before you sign.
You stop an AI knowledge base from inventing answers by testing whether it can refuse. A grounded system retrieves passages from your own documents, quotes them, and says it does not know when retrieval comes back empty. Three criteria decide the purchase: measured retrieval quality, a visible citation on every answer, and access rules enforced before text reaches the model. Ask for all three as evidence, not as promises.
If you are scoping AI knowledge base development, the useful question is not which model a vendor uses. It is what happens on the day a new hire asks about a policy that was rewritten last month.
What is retrieval augmented generation, and why does it reduce invented answers?
Retrieval augmented generation is a pattern in which the system searches your documents first and passes the matching passages to the language model as the only permitted source. The model writes an answer from those passages rather than from its training data, so the answer can be checked against a document you own.
The mechanism matters more than the label. A question is converted into a search query, the query runs against an index of your documents, the top passages are ranked, and only the highest-ranked ones enter the prompt. If nothing clears the relevance threshold, the correct behaviour is a refusal.
if a system cannot show which passage produced an answer, it is not grounded, whatever the vendor calls it. That single test separates a document search tool with a chat interface from a system you can put in front of staff.
How do you measure retrieval quality before you trust the answers?
Retrieval quality is measured with a fixed set of real questions, each paired with the document passage that should answer it, and a score for how often that passage appears in the top results. Without that set, nobody can tell whether a bad answer came from poor search or from the model.
Build the test set from your own history. Pull questions staff actually asked in the last year — the refund exception, the approval limit on a purchase order, the notice period for a supplier — and have the person who knows the answer mark the correct passage. A set of a few dozen questions is enough to catch regressions; the exact number depends on how many document families you have.
Then measure two things separately: whether the right passage was retrieved, and whether the final answer matched it. a retrieval score below the agreed threshold blocks release, even if the answers read well in a demo. Demo answers are chosen; test sets are not.
| Criterion | Weak answer from a vendor | What to ask for instead |
|---|---|---|
| Retrieval quality | The AI understands your documents. | A test set of real questions with a measured hit rate for the correct passage, re-run after every change. |
| Citations | Answers are accurate. | Every answer displays the document name, section and date, and refuses when no passage clears the threshold. |
| Chunking | We handle the technical details. | A written chunking strategy per document type, with the reason a policy is split differently from a product sheet. |
| Access control | Permissions are respected. | A demonstration that a restricted document never enters the prompt for an unauthorised user. |
| Updates | The system learns over time. | A named owner and a stated interval for re-indexing when a policy changes. |
What does a weak answer sound like, compared with a grounded one?
A weak answer is fluent, confident and untraceable; a grounded answer is narrower, cites a document and sometimes declines. The difference is visible in the first sentence, which is why a short pilot with real questions reveals more than a scripted demonstration.
Weak answers share recognisable shapes. They generalise from how similar companies usually operate, they merge two policies into one, they quote an old version of a document that is still in the index, or they answer a question about a specific client with a general rule.
A grounded answer names the source and its date. In a New York operation, that date is not cosmetic: a question about pay transparency or paid leave can turn on a state rule that changed recently, and an answer citing a superseded handbook page is worse than no answer. any answer without a document name and date should be treated as unverified.
Which documents belong in an enterprise AI knowledge base, and how should they be split?
Include documents that answer recurring questions and exclude drafts, duplicates and anything without an owner. Volume is not the goal; a smaller index of current, authoritative documents produces fewer contradictions than a large one that includes three revisions of the same policy.
A sensible starting scope:
- HR and leave policies, with the effective date on each version
- Product specifications and pricing rules that sales staff ask about weekly
- SOPs for order handling, returns and escalation
- Supplier and contract terms that operations reference
- Onboarding material new staff cannot locate in shared drives
The document chunking strategy decides much of the answer quality. A leave policy should be split by clause so a question about one entitlement retrieves one clause, while a product sheet is better kept whole so specifications stay together. if a chunk cannot be read on its own and still make sense, it is the wrong size.
For a multilingual knowledge base, index each language version rather than translating at query time, and mark which version is authoritative. A Chinese-language SOP and its English translation drift apart, and a system that silently mixes them will produce answers that are correct in one language and outdated in the other.
How should knowledge base access control work?
Access control must filter documents before retrieval, not after the answer is written. If a restricted document enters the prompt, the model may paraphrase it even when the final text is filtered, so the permission check belongs at the search step.
This matters most where one company holds several kinds of information. A family-owned distributor may keep supplier pricing visible to three people and HR files visible to none of the sales team; a single index without per-document permissions will leak both. The same care applies when staff in New York and Chengdu query the same system under different roles.
a permission test with a deliberately restricted document should be part of acceptance, not a later hardening phase. The delivery team controls the filtering logic; you control who is assigned to which role, and that assignment needs an owner after launch.
The same discipline applies when the knowledge base is wired into AI agent development, where an agent may act on what it retrieves. An agent that can send an email or change an order should inherit the same permission boundary as the person it acts for.
What should the pilot contract and handover include?
The pilot should end with a test report, not a demo. Ask for the question set, the retrieval scores before and after tuning, the list of documents excluded and why, and a written procedure for re-indexing when a policy changes.
Handover items that are often missed:
- A named internal owner for each document family
- The re-indexing procedure and who triggers it after a policy revision
- The refusal behaviour documented, so staff know an "I don't know" is correct
- A log of unanswered questions, reviewed monthly to find missing documents
- The permission matrix, tested with at least one restricted document
We build these systems from Chengdu for clients in North America, and the recurring failure is not model quality. It is an index nobody owns, filling with superseded documents until the answers contradict each other. if no one is assigned to retire old documents, answer quality decays within months of launch.
For businesses weighing this against other custom work, our articles on delivery practice cover scoping and handover in more detail, and the New York service page sets out how we work with teams in that time zone.
Frequently asked questions
What is retrieval augmented generation?
Retrieval augmented generation is a method where the system searches a document index first and passes the best-matching passages to a language model as its only source. The model writes from those passages, so each answer can be traced to a document you own. If no passage is relevant enough, the system should decline to answer.
How do you stop an AI knowledge base from inventing answers?
You stop it by making refusal a valid outcome and by testing retrieval separately from answer writing. Every answer should display the document name, section and date, and the system should refuse when no passage clears the relevance threshold. A fixed set of real questions with known correct passages, re-run after each change, shows whether the grounding still holds.
What documents should go into a company knowledge base?
Include current, authoritative documents that answer recurring questions: policies with effective dates, product specifications, SOPs, contract terms and onboarding material. Exclude drafts, duplicates and anything without a named owner. A smaller index of current documents produces fewer contradictory answers than a large one containing several revisions.
How does an enterprise knowledge base with AI handle permissions?
It filters documents at the retrieval step, before any text reaches the model, so restricted material never enters the prompt. The permission check should be demonstrated during acceptance with a deliberately restricted document. Role assignment stays with the client, and it needs an owner after launch.
How long does AI knowledge base development take?
Duration depends mainly on how many document families you have, how clean they are, and how quickly your staff can mark correct answers for the test set. A pilot covering one or two departments is a different scope from a company-wide index with multilingual versions. We size it after reviewing the documents rather than quoting a fixed period.
Questions we are asked
What is retrieval augmented generation?
Retrieval augmented generation is a method where the system searches a document index first and passes the best-matching passages to a language model as its only source. The model writes from those passages, so each answer can be traced to a document you own. If no passage is relevant enough, the system should decline to answer.
How do you stop an AI knowledge base from inventing answers?
You stop it by making refusal a valid outcome and by testing retrieval separately from answer writing. Every answer should display the document name, section and date, and the system should refuse when no passage clears the relevance threshold. A fixed set of real questions with known correct passages, re-run after each change, shows whether the grounding still holds.
What documents should go into a company knowledge base?
Include current, authoritative documents that answer recurring questions: policies with effective dates, product specifications, SOPs, contract terms and onboarding material. Exclude drafts, duplicates and anything without a named owner. A smaller index of current documents produces fewer contradictory answers than a large one containing several revisions.
How does an enterprise knowledge base with AI handle permissions?
It filters documents at the retrieval step, before any text reaches the model, so restricted material never enters the prompt. The permission check should be demonstrated during acceptance with a deliberately restricted document. Role assignment stays with the client, and it needs an owner after launch.
How long does AI knowledge base development take?
Duration depends mainly on how many document families you have, how clean they are, and how quickly your staff can mark correct answers for the test set. A pilot covering one or two departments is a different scope from a company-wide index with multilingual versions. We size it after reviewing the documents rather than quoting a fixed period.
