Developers and admins ask ChatGPT and Perplexity how to install, configure and fix software all day. When the answer comes from your docs, the assistant cites your docs, and the person stays on your product. When it doesn't, the answer comes from a forum thread, a competitor's guide, or the model's memory of an old version.
These are the changes that make documentation easy to cite, roughly in order of impact.
1. Make the docs fetchable
- Public by default. Pages behind a login can't be cited. If some content must stay private, keep the public reference and task pages open and gate only account-specific material.
- Server-rendered HTML. Many docs sites are single-page apps that load content with JavaScript. Most AI crawlers don't execute JavaScript, so they see an empty page. Use static generation or server rendering, then check with
curlthat the text is in the raw HTML. - Crawler access. Allow OAI-SearchBot (ChatGPT search), ChatGPT-User (when a user asks ChatGPT to open a link), PerplexityBot and Perplexity-User in
robots.txtand in your CDN or firewall. Also allow Bingbot and Googlebot, since Copilot and Google's AI features use their indexes. - Sitemap with real dates. List every docs page with an accurate
lastmod, so crawlers know what changed.
A quick test:
curl -s -A "OAI-SearchBot" https://docs.example.com/install | grep -c "npm install"
If that prints 0 for a page that shows the command in a browser, the crawler can't see it.
2. One task per page, written as an answer
- Name pages the way users ask: "Install the CLI on Windows", "Rotate an API key", "Connect to Tally". Avoid internal feature names in titles unless users know them.
- Put the answer in the first lines: the command, the setting, the limit. Explanation comes after.
- Keep code examples complete and copyable, with the version they were tested on.
- State requirements and limits explicitly: supported versions, rate limits, plan restrictions, regions. Assistants quote these, and missing limits lead to confident wrong answers.
- Use stable URLs. If you must move a page, redirect it permanently.
3. Handle versions on purpose
Versioned docs often produce several near-identical pages per topic, and assistants may cite an old one.
- Pick one canonical URL per topic for the current version, and point older versions to it with
rel="canonical"or mark them clearly as archived. - Put the version number and last-updated date near the top of each page.
- Keep a dated changelog. It answers "when did X change" questions and gives assistants a source for recent changes.
4. Write pages for error messages
People paste error messages into ChatGPT exactly as they appear. A page titled with the exact error text, saying what causes it and how to fix it, matches those searches. Pull the most frequent errors from support tickets and logs, and give each one a short page: the message, the cause, the fix, and how to prevent it.
5. Structure that models parse well
- One H1 per page, then H2 and H3 headings in order
- Tables for parameters, options and limits
- Numbered lists for procedures
- Descriptive link text ("see Rotate an API key"), not "click here"
- Short pages over very long ones, with links between related tasks
6. llms.txt and Markdown copies
llms.txt is a proposal from 2024 for a Markdown file at the root of a site that lists its most useful pages for language models. Several documentation platforms generate it automatically, along with llms-full.txt (all docs in one file) and .md versions of each page. These help when a developer pastes your docs into an AI coding tool.
No major AI search provider has said it uses llms.txt when choosing what to cite, so treat it as a small extra. Fetchable HTML and clear pages come first.
7. Check that it works
- Logs: look for ChatGPT-User and Perplexity-User requests on docs pages. These are live fetches made to answer a user, which is the closest signal you'll get to "my docs were used". How to analyze server log files explains the setup.
- Prompt tests: ask ChatGPT and Perplexity your 20 most common support questions each month and record whether your docs are cited.
- Referrals: segment docs traffic from chatgpt.com and perplexity.ai in your analytics.
- Support tickets: if assistants cite your docs well, tickets for those topics should drop.
If you're building your own assistant on top of the docs, the same structure helps your retrieval. See combining LLMs with real-time company data.

