Cloudflare's Markdown for Agents is a service designed to optimize web content consumption for Artificial Intelligence (AI) systems and agents by converting HyperText Markup Language (HTML) pages into Markdown format in real-time. This conversion is crucial because Markdown's explicit structure is more amenable to AI processing, leading to improved AI results and reduced token waste compared to raw HTML.

The core methodology involves content negotiation at the network edge. When an AI system or client requests a web page from a Cloudflare-enabled zone with Markdown for Agents activated, it can express a preference for text/markdown by including an Accept: text/markdown header in its HTTP request. Cloudflare's network intercepts this request, transparently fetches the original HTML content from the origin server, and performs an on-the-fly conversion of the HTML to Markdown. The converted response is then served to the client with a Content-Type of text/markdown. Additionally, the response includes an x-markdown-tokens header, which provides an estimated count of tokens in the Markdown document, valuable for managing context windows and chunking strategies in AI applications. A Content-Signal header, defaulting to $ai-train=yes, search=yes, ai-input=yes$, indicates permissions for AI training, search indexing, and general AI input, with future provisions for custom policies.

To use Markdown for Agents, clients send requests with the specified Accept header. For example, a curl command would be curl https://example.com/page/ -H "Accept: text/markdown", and in TypeScript using fetch, await fetch(url, { headers: { Accept: "text/markdown" } }). Enabling the feature requires administrative action at the zone level or for specific custom hostnames. For an entire zone, users with Pro or Business plans can activate it via the Cloudflare dashboard's "Quick Actions" section or programmatically via the Cloudflare API by sending a PATCH request to /client/v4/zones/{zone_tag}/settings/content_converter with a JSON payload of {"value": "on"} and appropriate Authorization headers (Bearer token with Zone Settings edit permissions). For Cloudflare for SaaS users, Markdown for Agents can be enabled for all custom hostnames via the SaaS zone dashboard. For specific custom hostnames, it necessitates an advanced subscription providing access to custom metadata. This involves two steps: first, setting custom_metadata: { "content_converter": "enabled" } when creating or updating the custom hostname via an API PATCH to /client/v4/zones/{zone_id}/custom_hostnames/{custom_hostname_id}; second, creating a Configuration Rule via an API PUT to /client/v4/zones/{zone_id}/rulesets/phases/http_config_settings/entrypoint that matches hostnames with this specific metadata using the expression lookup_json_string(cf.hostname.metadata, "content_converter") eq "enabled" and sets the content_converter action parameter to true.

Markdown for Agents is available at no additional cost for Cloudflare's Pro, Business, Enterprise plans, and SSL for SaaS customers.

Limitations include that the service will not convert and will instead return the original HTML response if the origin response lacks a content-length header, exceeds 1 MB (1,048,576 bytes) in content-length, or if the origin response is compressed. It exclusively converts from HTML, though future support for other document types is anticipated. As a zone-level setting, it does not support differing configurations for sub-domains within the same zone, necessitating separation into distinct zones for varied settings. For scenarios where content is outside Cloudflare's network or Markdown for Agents is unavailable, Cloudflare offers alternative conversion tools such as Workers AI's AI.toMarkdown() for multiple document types and summarization, and the Browser Rendering /markdown REST API for converting dynamically rendered pages.