Policy · 1h ago
Cloudflare gives AI firms ultimatum: pay publishers or get blocked
Cloudflare is requiring AI companies to separate crawlers for search from those for AI training by September 15, or risk being blocked on many publisher sites. The policy aims to enforce fair compensation for publishers whose content is used to train AI models. Non-compliant firms will face default blocking across Cloudflare's network.
Meridian48 take
This shifts enforcement from voluntary robots.txt to a technical mandate, but the real test is whether Cloudflare can reliably distinguish crawler types.
Read the full reporting
Cloudflare’s new policy pushes AI companies to pay for publishers’ content →
TechCrunch AI
ai-crawlerspublisher-rights