How an AI agent answers using your company's documents
Knowledge bases and RAG explained: why uploading files is not enough and how to keep answers current.
An assistant needs verified information to answer questions about a company. A knowledge base connects responses to documents, conditions and procedures. It works only when the material is current, relevant and accessible to the right person.
A useful knowledge base starts with good documents and access rules. Retrieval can reduce some mistakes, but it does not guarantee a correct answer.
How retrieval works
A common approach is RAG: the system finds passages related to a question and provides them to the model to prepare an answer. This differs from training a new model on every company document.
A cancellation question should retrieve the current cancellation policy. If search finds an old instruction or another service's conditions, a confident answer can still be wrong.
Prepare the source material
Collect service descriptions, limitations, support answers and internal procedures. Each document needs an owner, version and clear status. Resolve contradictory conditions before connecting an agent.
An accurate price table is often more useful than several presentations with different prices. Frequently changing values are usually better retrieved from a primary system through a tool than copied into a static file.
Handle missing answers
The assistant should recognise insufficient verified information and hand over to a person. Showing a source helps staff check conditions and exceptions.
A document link alone does not prove an answer follows from it. Review whether the retrieved passage addresses the question and whether the model added unsupported details.
Preserve access boundaries
A customer must not see internal discount notes, and one department must not retrieve another's restricted documents. Enforce permissions during retrieval and data access, rather than solely through the model's instruction.
Treat externally supplied documents as untrusted. A sentence telling the assistant to ignore its rules is content to process, not permission to change system behaviour.
Test and maintain the knowledge base
Create questions with known answers and questions the documents cannot answer. Test old versions, similar service names and access restrictions. Confirm that the displayed source actually supports the response.
Assign a knowledge owner and an update process. When conditions change, update the source and re-test related questions. Otherwise the assistant can efficiently spread outdated information.
Example: booking cancellation terms
A guest asks about cancellation two days before arrival. Retrieve the selected rate's current rules; general conditions may not apply to non-refundable or holiday bookings.
Keep the document, version and relevant passage visible for review. Missing rate information should prompt clarification or handoff. Retrieved context helps but does not guarantee interpretation. Check both the source passage and the final answer when business commitments are involved.
Prepare useful documents
Start with frequent questions, prices, services and handoff rules. Assign owners and review dates, remove obsolete active versions and resolve contradictions before connecting them.
Separate public conditions, internal instructions and individual agreements by purpose and permissions. Not everything belongs in the customer-facing knowledge collection. Check text extraction from scans and tables, including headings, units and row relationships. Uploading a large archive does not ensure accurate retrieval.
Handle updates and conflicting rules
Define who publishes updates, when they become available and how old versions disappear. Test control questions after changing prices or conditions.
Mark each document's product, region, rate or date. Request clarification when necessary instead of selecting the first plausible file. Preserve the link between answers and materials. Complaints can then be traced to a source problem, retrieval failure or mistaken wording.
Accept and maintain the knowledge base
Test real questions, similar services, different rates, old conditions and unavailable answers. Under different roles, verify that customers cannot retrieve internal information.
Check how easily staff can review answers and update materials. Maintenance effort should be understood and included in the support arrangement. Assign ongoing content ownership. A knowledge base is a controlled collection of current materials, not a promise that an agent will know everything forever.
What to check
- Resolve contradictions and assign document owners.
- Retrieve changing values from the primary system.
- Check supporting sources and missing-information cases.
- Enforce access limits and an update process.
Sources and documentation
Have a business task in mind?
Tell us what you need. We'll suggest where to start and how to check the result later.
Discuss your projectFree · 30 minutesExplore the service
