Your customers still Google. But Google now answers with an AI Overview, and they ask ChatGPT, Claude, and Perplexity directly too. Your website has two readers — and if it can't speak to both, you're invisible. Here's what we found when we audited our own site.

Your website has two readers now, the human and the agent they sent, and on Cloudflare’s network the agent already accounts for 4.2% of HTML requests with user-action crawls up fifteen-fold year over year.

“In 2027, the amount of bot traffic online will exceed the amount of human traffic that’s online.” — Matthew Prince, CEO of Cloudflare, at SXSW, March 2026

If your site can’t speak to both readers, you’re invisible in the answers your customers are asking for.

Two Readers, Not One

A human skims headlines, glances at images, reads what looks useful. An agent makes an HTTP request, receives your HTML, and tries to extract the fact the user asked about. Ads, modals, pop-ups, and client-side rendering are hostile territory. Clean, semantic markup is treasure.

The numbers come from Cloudflare’s 2025 Year in Review: AI bots are already 4.2% of HTML requests on its network, and user-action crawling — the kind triggered when someone asks an AI about a specific site — grew fifteen-fold year over year. Not 15%. Fifteen times. Prince’s SXSW quote isn’t a forecast; it’s a line on a chart the rest of us are about to walk along.

Cloudflare also shipped the cleanest proof of the shift in February 2026: Markdown for Agents. Send Accept: text/markdown, get the page back as clean markdown — no HTML parsing, no wasted tokens. One toggle. The web is quietly growing a second interface, a parallel machine-readable layer underneath the human one.

The Standards Are Being Written Right Now

The specs for that layer are still in draft. That is the point. A handful of them are landing in real time:

  • RFC 8288 (Web Linking) — HTTP Link: headers that tell agents where else to look.
  • RFC 9727 (API Catalog) — a .well-known/api-catalog where agents discover your APIs.
  • Content Signals — AI training, search, and input preferences declared in robots.txt.
  • MCP Server Card (SEP-1649) — .well-known/mcp.json for agent discovery.
  • Agent Skills Discovery RFC — a skills index published under /.well-known/.
  • WebMCP — browser-native, Chrome Canary only, early.

None of these are household names yet. They will be. The web’s 2026 looks a lot like the web’s 2005, when robots.txt, sitemaps, and canonical tags had to be invented before search engines could do their job. Agents need the same scaffolding. Someone has to put it up.

We Ran the Audit on Ourselves — Eight Gaps, Roughly Two Engineering Days

Here is the asymmetry you came for: each gap below is less than a hundred lines of configuration, the eight together are roughly two engineering days of work, and the cost of skipping them is the next five years of your brand showing up — or not — in agent-mediated answers.

We ran alsheikhmedia.com through Cloudflare’s Agent Readiness scorecard, free and public, shipped during Agents Week 2026.

We scored 25 out of Level 1. The basics were in place — robots.txt, sitemap.xml, semantic HTML — and we failed eight specific checks:

  1. No Link: response headers for agent discovery
  2. No Markdown-for-Agents support
  3. No AI-specific rules in robots.txt (no GPTBot, no Claude-Web, no opt-in / opt-out)
  4. No Web Bot Auth directory at /.well-known/http-message-signatures-directory
  5. No Content Signals declaring our AI training and search preferences
  6. No API Catalog
  7. No OAuth / OIDC discovery metadata
  8. No MCP Server Card, no Agent Skills index, no WebMCP tools

Most of them are files under /.well-known/ — the same folder that already holds security.txt. We are going to one hundred, publicly. Every fix gets documented on this blog so you can watch the work, copy what helps, and push back when we get it wrong. We are not writing the standards — the IETF, W3C, and Cloudflare are. We are going first in the Arabic-speaking web and journaling what works.

Why This Matters for the Arab Web

About 348 million people in the Arab world are online, per Orient Planet Research drawing on DataReportal, GSMA, and Statista. In the UAE, internet penetration is 99% — about 11.3 million users. That is a lot of brands and a lot of customers asking agents for answers in Arabic every day.

As of this week, we have not found a single major Arabic publisher or regional brand that has gone public with an agent-readiness program. We have also not yet seen a public case study quantifying citation lift after agent-readiness work, which is part of why we are writing this one. The conversation hasn’t started here. We want to start it by doing the work in public.

The Honest Counter-Argument

The specs are drafts. Agent traffic is still a minority of total traffic. Some of this architecture will be replaced by something else in three years.

The asymmetry stands anyway: two engineering days of configuration against five years of organic reach if the bet plays out the way the numbers say it will. You either build the new web, or you react to it.

What To Do This Week

One thing. Not three.

Run your site through isitagentready.com. Thirty seconds. You get a score and a punch list.

Then:

  • If you know what to do with the list — start.
  • If you don’t — that’s our work. We built this for ourselves first, and we are doing it for clients who want to skip the learning curve. Say hi: hello@alsheikhmedia.com.

And if the post helped — share it. The MENA web has not had this conversation yet. That’s the conversation we are trying to start.

AI AgentsMCPAgent ReadinessWeb StandardsMENASEO