--- title: "AI enablement for websites: making ihork.link agent-ready" date: 2026-09-24 description: How I made a JavaScript-only personal site readable and usable for AI agents — robots.txt rules and Content Signals, Markdown for agents, an A2A agent, an MCP server, an Agent Skills index, WebMCP tools, an API catalog and Link headers — and what I learned on the way. tags: ai, agents, webmcp, a2a, mcp --- More and more visits to a website don't come from a person looking at it. They come from an AI agent acting for a person: a crawler collecting training or search data, ChatGPT or Claude fetching a page to answer a question, or a browser agent clicking through the site on someone's behalf. I spent a day making this site work well for those visitors. This article walks through every step, with the code, so you can do the same for your site. ## The problem: what an agent actually saw ihork.link is a macOS-style terminal in the browser. Everything you see — the welcome text, the commands, the answers — is drawn by JavaScript. That's fun for people and useless for most agents, because most crawlers and fetch tools don't run JavaScript. Here is everything a non-JS agent could read on my homepage: ```text Ihor Kruchynskyi Github LinkedIn Email ``` Four words. No bio, no resume, no projects. An agent asked "who is Ihor Kruchynskyi?" would learn more from LinkedIn than from my own site. It helps to think of three kinds of agent visitors, because each one needs something different: | Visitor | Runs JavaScript? | Examples | What helps | |---|---|---|---| | Crawlers | No | GPTBot, ClaudeBot, PerplexityBot, Google-Extended | robots.txt rules, a sitemap, content in the HTML or Markdown | | Fetch tools | No | ChatGPT-User, Claude-User, an agent's `fetch` tool | Markdown for agents, `llms.txt` | | Agents with tools | Yes / API | Browser agents, AI assistants, other AI agents | WebMCP tools, an MCP server, an A2A endpoint, Agent Skills | To measure progress I used the scanner at [isitagentready.com](https://isitagentready.com), which checks a site against the current agent standards: ```bash curl -s -X POST https://isitagentready.com/api/scan \ -H 'Content-Type: application/json' -d '{"url":"https://ihork.link"}' ``` The first scan put the site at level 1, "Basic Web Presence": no robots.txt, no sitemap, no Markdown, no agent discovery of any kind. ## Step 1: robots.txt with explicit AI rules A robots.txt tells crawlers what they may fetch, per [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309). Mine welcomes AI crawlers and keeps them out of endpoints that are useless to them: ```text User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-User User-agent: Google-Extended User-agent: PerplexityBot User-agent: CCBot Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Disallow: /api/ Disallow: /forecast.php User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Disallow: /api/ Disallow: /forecast.php Sitemap: https://ihork.link/sitemap.xml ``` Three details matter here: - **A crawler obeys only the most specific group that names it.** GPTBot reads its own group and ignores `*` entirely, so every named group must repeat the `Disallow` rules. Forget that and you have silently allowed everything. - **Content Signals** ([contentsignals.org](https://contentsignals.org/)) declare what the content may be used for: `search` (indexing), `ai-input` (grounding answers in real time) and `ai-train` (training models). Decide your policy and make the signals match your `User-agent` groups. If you say `ai-train=no`, also block the training-only crawlers. - **Don't list secret paths.** robots.txt is public. `Disallow: /admin-backdoor.php` is an invitation, not a lock. ## Step 2: a sitemap that updates itself A [sitemap](https://www.sitemaps.org/protocol.html) lists the canonical URLs of your public pages. I started with a static `sitemap.xml`, then replaced it with a small PHP script as soon as this blog appeared, so a new article shows up in the sitemap the moment its Markdown file lands on the server: ```php $urls = [[SITE_URL . '/', ...], [SITE_URL . '/blog/', ...], [SITE_URL . '/weather.html', ...]]; foreach (blogPosts() as $post) { $urls[] = [$post['url'], $post['updated']]; } ``` ## Step 3: Markdown for agents This was the biggest win. Agents are happiest with clean text, and Markdown is the format they read best. The idea, also used by Cloudflare's [Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/), is content negotiation: a request with `Accept: text/markdown` gets Markdown, browsers keep getting HTML. Cloudflare can do the conversion at the edge, but it converts your HTML — and my HTML is an empty shell. So I wrote the Markdown version myself (`index.md`: profile, experience, education, links) and serve it from the origin: ```php // True when the Accept header prefers text/markdown over text/html function prefersMarkdown($accept) { $q = ['text/markdown' => 0.0, 'text/html' => 0.0]; foreach (explode(',', strtolower($accept)) as $part) { $params = array_map('trim', explode(';', $part)); $type = array_shift($params); if (!isset($q[$type])) continue; $weight = 1.0; foreach ($params as $param) { if (strpos($param, 'q=') === 0) $weight = (float)substr($param, 2); } $q[$type] = max($q[$type], $weight); } return $q['text/markdown'] > 0 && $q['text/markdown'] >= $q['text/html']; } header('Vary: Accept'); if (prefersMarkdown($_SERVER['HTTP_ACCEPT'] ?? '')) { header('Content-Type: text/markdown; charset=utf-8'); header('Cache-Control: private, max-age=600'); readfile(__DIR__ . '/index.md'); exit; } ``` Two headers do important work. `Vary: Accept` tells caches that the same URL has two representations. `Cache-Control: private` keeps the CDN from caching the Markdown and serving it to the next browser — Cloudflare ignores `Vary` for most content, so don't rely on it alone. Next to that, `/llms.txt` gives a short plain-text summary, and every article on this blog has a Markdown twin: add `.md` to any post URL, or request it with `Accept: text/markdown`. ## Step 4: an A2A agent The [Agent2Agent protocol](https://a2a-protocol.org/latest/specification/) (A2A) lets one AI agent talk to another. An agent publishes an **Agent Card** at `/.well-known/agent-card.json` describing its skills and endpoint, and other agents send it messages over JSON-RPC. Before writing any code I had to decide something important. An agent card isn't a description file — it advertises a live service. My terminal's chat endpoint is protected by Cloudflare Turnstile, which agents can't pass, so a card pointing at it would pass the scanner and fail every real agent. I chose to build a small dedicated endpoint with two skills: - **profile** returns the Markdown profile instantly, without calling a model. - **ask** answers a free-text question about me with the same model and resume as the terminal. The card, shortened: ```json { "name": "Ihor Kruchynskyi", "description": "Answers questions about Ihor Kruchynskyi ...", "supportedInterfaces": [ { "url": "https://chat.ihork.link/a2a.php", "protocolBinding": "JSONRPC", "protocolVersion": "1.0" } ], "version": "1.0.0", "capabilities": { "streaming": false, "pushNotifications": false }, "defaultInputModes": ["text/plain"], "defaultOutputModes": ["text/markdown", "text/plain"], "skills": [ { "id": "profile", "name": "Profile", "description": "...", "tags": ["profile", "resume"] }, { "id": "ask", "name": "Ask about Ihor", "description": "...", "tags": ["q&a", "career"] } ] } ``` And a conversation with it: ```bash curl -s https://chat.ihork.link/a2a.php \ -H 'Content-Type: application/json' -H 'A2A-Version: 1.0' \ -d '{"jsonrpc":"2.0","id":1,"method":"SendMessage","params":{"message":{ "messageId":"b1c2d3e4-0000-4000-8000-000000000001","role":"ROLE_USER", "parts":[{"text":"What does Ihor work on at Adobe?"}]}}}' ``` ```json {"jsonrpc":"2.0","id":1,"result":{"message":{ "messageId":"…","contextId":"…","role":"ROLE_AGENT", "parts":[{"text":"At Adobe, Ihor …","mediaType":"text/markdown"}]}}} ``` Things I learned implementing it: - **Read the spec source, not a summary.** A summarized version of the spec told me the result is the `Message` itself. The protobuf definition ([a2a.proto](https://github.com/a2aproject/A2A/blob/main/specification/a2a.proto)) says `SendMessageResponse` is a `oneof`, serialized as `{"message": {…}}` or `{"task": {…}}`. Roles are `ROLE_USER` / `ROLE_AGENT`, a text part is just `{"text": "…"}`, and the card needs `tags` on every skill. - **Versions matter.** Clients send an `A2A-Version` header; no header means 0.3, which used `message/send` and `kind` fields. Supporting both takes a few lines and keeps older clients working. - **Return real errors.** Streaming, tasks and push notifications aren't supported here, so those methods return the spec's error codes (`-32004`, `-32001`, `-32003`) instead of a generic failure. Because the endpoint has no human check, it needs its own guardrails: 5 questions per minute and 50 per day per IP, 500 per day in total to protect the model quota (shared with the MCP server below, so a second protocol doesn't double the exposure), no tools (no web search, no image generation), and a system prompt that refuses anything that isn't about me. Ask it to write a Python function and it replies "This agent only answers questions about Ihor Kruchynskyi." An open endpoint with a model behind it is a free LLM for anyone who finds it, unless you scope it. ## Step 5: Agent Skills discovery The [Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) lets a site publish instructions for agents as `SKILL.md` files — the same format coding agents use for their skills. An index at `/.well-known/agent-skills/index.json` lists them: ```json { "$schema": "https://schemas.agentskills.io/discovery/0.2.0/schema.json", "skills": [{ "name": "ihork-profile", "type": "skill-md", "description": "Look up Ihor Kruchynskyi ... through ihork.link's Markdown pages, its A2A agent and its WebMCP browser tools.", "url": "/.well-known/agent-skills/ihork-profile/SKILL.md", "digest": "sha256:…" }] } ``` My skill tells an agent the cheapest way to answer questions about me: read the Markdown profile and articles first, connect over MCP or ask the A2A agent only when that isn't enough, and use the WebMCP tools when it's driving a browser. The `digest` is a SHA-256 of the file, so it must change whenever the skill changes. I generate the index in my build script instead of maintaining it by hand — a stale digest is exactly the kind of bug nobody notices. ## Step 6: WebMCP tools in the page [WebMCP](https://webmachinelearning.github.io/webmcp/) is the newest piece: a browser API that lets a page expose tools to an AI agent running in the browser, instead of making the agent guess where to click. Chrome is trialing it ([announcement](https://developer.chrome.com/blog/webmcp-epp)). A tool has a name, a description, a JSON Schema for its input and an `execute` callback: ```js navigator.modelContext.registerTool({ name: "get_profile", title: "Ihor's profile", description: "Get Ihor Kruchynskyi's profile and resume as Markdown.", inputSchema: { type: "object", properties: {} }, annotations: { readOnlyHint: true }, async execute() { const res = await fetch("/index.md"); return { content: [{ type: "text", text: await res.text() }] }; }, }, { signal: controller.signal }); ``` The site registers five tools: `get_profile`, `get_projects`, `get_blog_posts`, `run_terminal_command` and `ask_about_ihor`. Two design choices I'd recommend: - **Reuse the real UI.** `run_terminal_command` types the command into the visible terminal and returns what it printed, and it only accepts the built-in commands. The person watching sees exactly what the agent did, and there is no second code path to maintain. - **Don't make agents wait for a human click.** My first version sent `ask_about_ihor` through the terminal's Turnstile check. When I tested it as a browser agent, the check wanted a click and the tool just hung. Now, while the visitor isn't verified, the question goes to the MCP server's `ask_about_ihor` instead — no Turnstile, but also no tools and the shared agent rate limit — and the answer still appears in the terminal. Verified visitors keep the full chat. The rule: an agent-facing tool should either work or fail fast with a way forward. The `signal` option unregisters the tools when the page goes away. I abort it on `pagehide` and register again if the page comes back from the back/forward cache. ## Step 7: an MCP server The [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) is how AI assistants and coding agents connect to tools and data. A site can advertise an MCP server with a **server card** at `/.well-known/mcp/server-card.json` ([SEP-1649](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2127)). Same rule as with A2A: a card is a promise, so I built the server first — one PHP file, `https://chat.ihork.link/mcp.php`, no SDK. It exposes five tools and the matching resources: | Tool | What it returns | Uses the model? | |---|---|---| | `get_profile` | Profile and resume as Markdown | No | | `get_projects` | Public GitHub repositories (JSON) | No | | `list_blog_posts` | Articles with URLs (JSON) | No | | `read_blog_post` | One article's Markdown, by slug | No | | `ask_about_ihor` | An AI answer grounded on my resume | Yes, rate limited | The interesting part was the protocol version. The newest revision, `2026-07-28`, made MCP **stateless**: there is no `initialize` handshake any more. Every request carries its protocol version and client capabilities in `_meta`, and the HTTP headers mirror the body so gateways can route without parsing JSON: ```bash curl -s https://chat.ihork.link/mcp.php \ -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2026-07-28' -H 'Mcp-Method: tools/call' -H 'Mcp-Name: get_profile' \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"get_profile","arguments":{}, "_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28", "io.modelcontextprotocol/clientCapabilities":{}}}}' ``` The server must check that `MCP-Protocol-Version`, `Mcp-Method` and `Mcp-Name` match the body and answer `400` with error `-32020` when they don't; an unknown version gets `-32022` with the list of supported versions, and every result carries `"resultType": "complete"`. A new `server/discover` method replaces the handshake. Most MCP clients in the wild still speak `2025-11-25` or older, which start with `initialize`. The spec calls a server that handles both a **dual-era** server, and it's cheap to be one: if the request has the new `_meta` fields, apply the stateless rules; if it's `initialize`, answer the handshake and treat the rest the old way. ```php $meta = is_array($params['_meta'] ?? null) ? $params['_meta'] : []; $modern = isset($meta['io.modelcontextprotocol/protocolVersion']) || in_array(mcpHeader('MCP-Protocol-Version'), ['2026-07-28'], true); ``` Two details that make the server nicer to use from a real assistant: - **Return follow-up state where the model can see it.** `ask_about_ihor` returns a `conversationId` for follow-up questions both in `structuredContent` and at the end of the text, because many clients only show the text to the model. - **One budget, two doors.** The MCP tool and the A2A agent call the same PHP function, `agentAsk()`, and share one rate limit. Adding a protocol shouldn't double what the model can cost you. ## Step 8: an API catalog and Link headers With a Markdown profile, a blog, an MCP server and an A2A agent, there are now several entry points. Two standards tie them together. The **API catalog** ([RFC 9727](https://www.rfc-editor.org/rfc/rfc9727)) lives at `/.well-known/api-catalog` and lists a publisher's APIs as a [Linkset](https://www.rfc-editor.org/rfc/rfc9264): each API is an `anchor` with a `service-desc` (a machine-readable description, here the MCP and A2A cards) and a `service-doc` (documentation, here the `SKILL.md`): ```json { "linkset": [ { "anchor": "https://chat.ihork.link/mcp.php", "service-desc": [{ "href": "https://ihork.link/.well-known/mcp/server-card.json", "type": "application/json" }], "service-doc": [{ "href": "https://ihork.link/.well-known/agent-skills/ihork-profile/SKILL.md", "type": "text/markdown" }] }, { "anchor": "https://ihork.link/.well-known/api-catalog", "item": [ { "href": "https://chat.ihork.link/mcp.php" }, { "href": "https://chat.ihork.link/a2a.php" }, { "href": "https://ihork.link/blog/index.json" } ] } ] } ``` It has to be served as `application/linkset+json` with an RFC 9727 profile, and a `HEAD` request must return a `Link` header pointing to itself. The file has no extension, so the web server needs to be told. In lighttpd: ```text $HTTP["url"] == "/.well-known/api-catalog" { mimetype.assign = ( "" => "application/linkset+json; profile=\"https://www.rfc-editor.org/info/rfc9727\"" ) setenv.add-response-header = ( "Link" => "; rel=\"api-catalog\"" ) } ``` **Link headers** ([RFC 8288](https://www.rfc-editor.org/rfc/rfc8288)) put the same pointers on the homepage response itself, so an agent that fetches `/` — even with `HEAD` — learns where everything is without guessing well-known paths: ```text Link: ; rel="api-catalog", ; rel="service-desc"; type="application/json", ; rel="service-desc"; type="application/json", ; rel="service-doc"; type="text/markdown", ; rel="describedby"; type="text/plain", ; rel="alternate"; type="text/markdown" ``` Every article sends one too, pointing at its Markdown version. The result is a discovery chain an agent can follow from any page: `Link` header → API catalog → cards → endpoints. ## Checking the result After these steps the scanner reports a pass for robots.txt, the sitemap, Link headers, AI crawler rules, Markdown negotiation, the API catalog, the MCP server card, the A2A Agent Card, the Agent Skills index and WebMCP — it found all five tools by loading the page in a browser. One check kept failing for a while for a boring reason: **Cloudflare had cached the old robots.txt**. Text files are cached at the edge by default, so the new rules and Content Signals were invisible until I purged the cache. If a check fails right after a change, fetch the file with a cache-busting query string before you start debugging. ## Lessons - **Start with what agents see.** `curl` your homepage and strip the tags. If it's four words, nothing else matters until you fix that. - **Markdown is the highest-value, lowest-effort step.** One content-negotiated Markdown file made the site readable for every agent. - **Don't advertise what you don't serve.** An agent card, a skill or a tool that leads nowhere is worse than none. - **Every open door needs a budget.** Rate limits, a narrow scope and no tools turned an open LLM endpoint into a small, predictable one — and one shared budget means a new protocol doesn't add new cost. - **Specs move fast; support the old and the new.** MCP dropped its whole handshake in `2026-07-28` while most clients still use it. Reading the spec's own text for the latest version, then accepting both eras, is cheap and keeps every client working. - **Keep one source of truth.** My resume now lives in the model's prompt, the Markdown profile and `llms.txt`. When one changes, all three must. There's still more on the list, like DNS-based agent discovery; OAuth metadata only matters once an API needs sign-in. And this blog itself follows the same rules — every article is a Markdown file on the server, rendered to HTML for you and served as-is to agents at `/blog/.md`.