Most "the AI gave a bad answer" complaints resolve into a documentation problem. Not a missing page — an unretrievable one. The information was there; it was written in a shape that search cannot get hold of.
The encouraging part is that the fixes are the same ones a good technical editor would ask for anyway. Nothing here is written for the machine at the expense of the human.
Put the answer in the first sentence
Retrieval works on passages, and a passage is judged on what it contains, not on what the paragraph three screens above it established. A section that opens with "As mentioned, this only applies in the second case" is unusable on its own.
Write sections that survive being read in isolation. Lead with the conclusion, then explain it.
| Instead of | Write |
|---|---|
| Once you have completed the steps above, it is worth noting that the window may vary. | Refunds are available within 30 days of purchase. |
| This can be configured in the usual place. | Set the reply delay in Settings → Chat widget → Behaviour. |
| See the previous section for the limits. | The Free plan allows 50 conversations per month. |
Use the words your customers use
Semantic search closes a lot of the vocabulary gap, but not all of it, and it cannot help at all if a term appears nowhere in your content. If customers write "cancel my subscription" and every page says "manage plan lifecycle", retrieval is doing translation work it should not have to do.
Your ticket queue is a free list of the real vocabulary. Mine it, then work those phrasings into headings and opening lines — not as keyword stuffing, but as the honest naming of the thing.
One page, one job
The 4,000-word "Complete guide to billing" is a retrieval trap. It matches almost every billing question weakly and no billing question strongly, so it crowds out the precise page that would have answered properly.
Split it. Ten short pages — how to change a card, how to get an invoice, what happens when a payment fails — each retrieve cleanly, and each is a better page for a human in a hurry.
Give every heading a full thought
Headings carry disproportionate weight, both for chunking (they are where sensible splits happen) and for ranking. "Overview" tells a retriever nothing. "What happens when a payment fails" tells it everything.
- "Overview" → "What the shared inbox is for"
- "Notes" → "Limits on file uploads"
- "Troubleshooting" → "The widget does not appear on my site"
Write the tables out
A pricing matrix reads beautifully and retrieves terribly: the cell says "40 MB" and nothing nearby says which plan or what is being measured. Keep the table — humans need it — but add a sentence underneath in plain prose: "The Hobbyist plan includes 40 MB of training storage per assistant."
The same goes for anything carried by layout rather than language: screenshots with the answer only in the image, callouts that depend on colour, steps whose meaning lives in their numbering.
Delete the pages that are no longer true
Stale documentation is worse than missing documentation, because retrieval treats it as equally valid. The 2024 migration guide will be returned with total confidence next to this year's. If a page is obsolete, archive it out of the index — do not leave it lying around with a note at the top saying it is outdated, because the note is a separate passage from the wrong instructions.
Every page you leave in the index is a page you are willing to have read aloud to a customer, verbatim, at three in the morning.
A two-hour audit
- 1Take the 20 most common questions from last month's tickets.
- 2Ask the bot each one and note which page it retrieved.
- 3For each miss, classify it: no page, wrong page, or right page, badly written.
- 4Fix the third category first. It is the fastest, and it is usually the largest.
Teams that run this loop once a month tend to stop having AI accuracy problems altogether, which is a slightly deflating discovery: the model was never the bottleneck.
- Documentation
- Content
- Retrieval
