Currently, development using generative AI to write code is becoming widespread, but relying solely on model knowledge often results in the inclusion of non-existent parameters and endpoints. To reduce errors, it is effective to have the AI ​​refer to the latest official documentation.

While the generating AI can read HTML documents, it may not be able to find the necessary information even if given the homepage, and tags and navigation outside the main text also consume tokens. Until now, the AmiVoice API documentation was only provided in HTML format for humans to read in a browser, but we have decided to also deliver it in a format that is easier for AI to access.

The idea was inspired by a lightning talk on the Ekispert API at Zennfes Spring 2026. An llms.txt file was placed at the entry point of the API documentation, Issues and Pull Requests were exposed to generative AI through GitHub MCP, and development guidelines were documented in AGENTS.md. What particularly impressed me was their emphasis on creating an environment where AI could independently gather the information it needed before being assigned work, enabling it to produce meaningful results. I believe that users of the AmiVoice API will increasingly expect this kind of AI-driven development workflow. As a first step toward supporting it, I focused on improving how our official documentation is published and delivered.

This article introduces three measures taken to make it easier for the generating AI to refer to the official documentation, along with their implementation and effects.

Actions taken

I did the following three things:

1. Create a Markdown version of each page and return it when Accept: text/markdown is specified.
2. Provide an llms.txt file that lists the pages users should read first.
3. Add HTTP Link headers that point to the Markdown versions and llms.txt. 

In addition, I added a robots.txt file to explicitly state that crawling is allowed and to indicate the location of sitemap.xml. The site previously did not have a robots.txt file. Since crawling is permitted even when robots.txt is absent, this change did not alter the scope of publicly accessible content.

System configuration

The AmiVoice API documentation site (docs.amivoice.com) is built using Docusaurus, an open-source documentation website generator developed by Meta.

Previously, manuscripts written as MDX (an extended Markdown format that allows the use of React components) on Docusaurus were built using GitHub Actions to generate HTML, which was then placed on Amazon S3. Clients accessed the files via Amazon CloudFront.

The current content generation and distribution flow is explained in detail below.

1. Provide a Markdown version.

Step 1: Create a Markdown version during the build process.

The content is written in MDX as mentioned above and then converted to HTML for publication. This time, we decided to remove information that is meaningless to publish, such as MDX-specific React components and meta information (front matter) at the top of the page, and create Markdown for each page with the same content as the HTML. Users can now access a Markdown version of any page simply by appending the .md extension to the page URL.

HTML content:
https://docs.amivoice.com/en/en/amivoice-api/manual/getting-started

Markdown content:
https://docs.amivoice.com/en/en/amivoice-api/manual/getting-started.md

Since there was no original Markdown source for the top page, I created an index.md file based on the information displayed in its cards.

Step 2: Use the Accept header to differentiate between HTML and Markdown.

We added support for the Accept: text/markdown header, which is seeing growing adoption among AI agents.

For example, if the following Accept header is specified, the server returns Markdown instead of HTML.

curl -H "Accept: text/markdown" \
     https://docs.amivoice.com/en/en/amivoice-api/manual/getting-started

To implement this, we used a CloudFront Functions viewer-request handler to rewrite requests internally to the corresponding .md URL whenever the header is present.

However, support for this mechanism is still limited. A survey cited in a Cloudflare article from April 2026 found that only three of the seven AI agents evaluated, Claude Code, OpenCode, and Cursor, sent the Accept: text/markdown header by default. The same survey also found that support on the content-provider side remains limited, at only 3.9%.

2. Place llms.txt

Even when a generative AI is provided with the URL of a documentation site's home page and asked to write code or perform research, it does not always reach the pages containing the information it needs, which can lead to incorrect answers. To help solve this problem, an llms.txt file has been proposed as the first resource AI systems should read, and it is typically placed at the root of a site. We have also made an llms.txt file available for the AmiVoice API documentation at the following URL.

https://docs.amivoice.com/en/en/llms.txt

This file is not automatically generated. Instead, it's a collection of links to pages that we recommend you read first, covering topics such as interface selection, implementation, response verification, and operation. Each link leads to a Markdown version of the respective page.

However, this llms.txt file is not an IETF or W3C standard, and not all generative AI systems automatically look for it. Google has also stated that Google Search does not use llms.txt and that it has no impact on search rankings. At present, the primary use case appears to be providing the URL explicitly to a generative AI system so that it can access the information.

3. Indicate the related file in the Link header.

We also added HTTP Link headers to HTML responses to point to related Markdown versions and the llms.txt file. This approach has been adopted by organizations and platforms such as Cloudflare, the documentation platform Mintlify, and the cloud platform Vercel. It allows AI systems to discover the availability of AI-friendly documentation. 

The top page points users to the llms.txt file.

Link: ; rel="describedby"; type="text/markdown"

Each document page also provides a Markdown version of the same content in an alternative format.

Link: ; rel="alternate"; type="text/markdown"

Link headers are added only to HTML pages for which a Markdown version exists. Existing Link headers are preserved rather than overwritten, and duplicate values are automatically removed. This Link header processing is implemented in a CloudFront viewer-response function.

However, mechanisms for AI systems to discover documentation are still evolving, and no common standard has yet emerged. Although the Link header is standardized by RFC 8288, there is currently no consensus on how AI-oriented resources should be advertised. Rather than introducing custom relation types or headers, we adopted the IANA-registered alternate and describedby link relations. We use describedby to associate the top page with llms.txt, and alternate to associate each HTML document with its equivalent Markdown version. As AI agent support and industry standards continue to evolve, we may revisit this approach as needed.

Advantages

Markdown accounts for 7.5% of HTML.

The HTML of a document site includes not only the main text but also navigation, sidebars, headers, footers, styles (CSS), and JavaScript, all of which consume tokens generated by the AI. We investigated how much Markdown reduces token consumption and how much smaller the Markdown version becomes compared to the original HTML.

We used OpenAI's BPE tokenizer, tiktoken, for the analysis. The encoding was o200k_base, the encoding used by GPT-4o. For each page, we compared the token counts obtained by encoding the following three types of content with the same tokenizer configuration.

1. The complete HTML returned by a normal request
2. The text extracted from the HTML after parsing and removing all tags
3. The Markdown content retrieved from the same URL with Accept: text/markdown

The counting logic is shown below. For each page, we used the number of token IDs returned by encode() as the token count. Since the absolute values vary depending on the tokenizer, they should be interpreted as reference values rather than exact measurements.

import tiktoken
encoding = tiktoken.get_encoding("o200k_base")
token_count = len(encoding.encode(text))

As of July 2026, the published manuals and reference materials totaled 123 pages. The number of tokens for each was as follows:

The Markdown version was about 1/13th the size of the raw HTML. In terms of file size, the Markdown was 0.82MB compared to a total of 5.73MB for the HTML.

The difference was bigger than I expected. It seems certain that using the Markdown version can reduce the token consumption of the generating AI.

We investigated 71 items and found 11 specification differences across 6 articles.

We analyzed 71 articles submitted to the Zenn contest, "Building Voice Experiences with the AmiVoice API and Generative AI," to evaluate whether generative AI could correctly understand the AmiVoice API specifications. Using llms.txt as the entry point, we asked the model to identify statements that appeared to be inconsistent with the documentation. We then manually reviewed those findings against the official specifications. The model used in this evaluation was Claude Fable 5.

First, most of the articles were accurate. However, we found 11 specification discrepancies across 6 articles. The most common were errors regarding the WebSocket command scheme and the interfaces and parameters usable with speaker dilation.

Areas where there were many discrepancies in the specifications may have made it difficult for the generating AI or users to search the documentation and find the relevant specifications. We have taken these as areas for improvement in the documentation and are working to address them. We have also shared the feedback with the contributors who wrote the articles and asked them to revise their articles.

This investigation also helped us identify shortcomings in the llms.txt content. In some cases, information was explicitly documented, yet the model was unable to determine whether a statement was correct and instead flagged it for human review. Based on these findings, we improved the content of llms.txt.

How to Use

While AI agents could eventually make these mechanisms seamless for users, many AI agents still do not make use of Accept: text/markdown or llms.txt. Therefore, the following sections explain how users can explicitly provide Markdown content and llms.txt to AI systems.

As shown below, the AI is provided with the URL of the llms.txt file and instructed to rely on the official documentation as the authoritative source.

Using https://docs.amivoice.com/en/llms.txt as the starting point, 
please refer to the linked official AmiVoice API documentation. 

{{Instructions go here}} 

Do not guess specifications, and show the URLs of the official documentation you referred to.

For more detailed guidance, see the manual section Development with Generative AI.

Things we decided to pass on this time

When evaluating which AI-agent-related features to support, the website isitagentready.com was an invaluable reference for assessing a site's level of AI agent readiness. I initially planned to implement all of the listed capabilities, but ultimately decided not to support DNS-AID, Content Signals, Web Bot Auth, MCP Server Cards, Agent Skills, WebMCP, or x402. These technologies are still largely at the proposal stage, and either require significant implementation effort or fall outside the scope of this project, which focuses on serving static API documentation.

We plan to consider it when the need arises in the future.

Summary

To summarize, search, classify, and integrate audio content with other systems, the first crucial step is to convert it into accurate and reusable text. While some generative AI models can directly handle audio, for applications involving Japanese business speech, real-time processing, and large volumes of audio, we believe that a division of labor—where speech recognition is handled by a specialized engine and the generative AI interprets and processes the results—makes it easier to manage accuracy, processing time, and cost.

In this configuration, the use of generative AI for both writing code and researching specifications will increase more and more in the future. While people will still read documentation in browsers, the proportion of documentation accessed via generative AI will undoubtedly increase. We believe that creating an environment where generative AI can correctly access official documentation is a high priority for us as an API vendor, and that is why we have taken this action.

We will continue to improve our AI-generated documents and services based on actual usage results, so please look forward to future updates.

Reference link

• Is IT Agent Ready?
• Proposed specification for llms.txt — llmstxt.org
• How to Create a Development Environment Where You Can Delegate Tasks to AI ~Zennfes Spring 2026 Lightning Talk Presentation~ — Ekispert API
• The story of how an AI created a web application by following the specifications after being given a single line of URL from llms.txt — Ekispert API
• AI Meets API Docs: The Why Behind /llms.txt — ReadMe
• Announcing Twilio Docs Support for llms.txt and Markdown — Twilio
• Improved agent experience with llms.txt and content negotiation — Mintlify
• Introducing Markdown for Agents — Cloudflare
• Markdown for Agents — Cloudflare Developers
• Google Search Optimization Guide for Generative AI — Google Search Central
• RFC 8288: Web Linking — IETF
• RFC 7763: The text/markdown Media Type — IETF
• How to create robots.txt — Google Search Central
• CloudFront Functions — Amazon Web Services
• tiktoken — OpenAI