OpenAI, Google, Microsoft and Anthropic back one switch to block AI training

When it starts: Anthropic, the maker of Claude, has "already started" rolling it out, AFP reported. Microsoft will begin honoring it on Feb. 1, 2027, according to its Bing blog. Google said it aims to support it "in 2027," and OpenAI said it will put it in place "in the coming months." Meta spokesman Andy Stone called a common signal "an important step" but did not say whether Meta will honor it, The Verge reported.

Where it came from: The signal was proposed by Kevin Kelley, a standards lead at Anthropic, in a draft filed Friday with the Internet Engineering Task Force (IETF), the volunteer group that writes many of the internet's technical rules. The IETF's own page says the draft is an individual proposal that "has no formal standing" in its standards process yet. A working group there has spent more than a year on how sites should state AI preferences.

From the draft: The opt-out is two lines a site owner adds to robots.txt. "User-Agent: __usage_ai-training" followed by "Disallow: /" says don't train on anything on the site. Adding a line such as "Allow: /press-kit/" carves out pages the owner is happy to share. The draft says a general rule aimed at every crawler ("User-Agent: *") does not count as an AI-training opt-out; the owner has to use the new token.

What it doesn't do: Microsoft said the setting applies only to future model training and does not block crawling, and that training is allowed by default unless a site opts out. It also said the setting creates no licensing or payment deal, and that "neither choice guarantees inclusion, traffic, or payment." Google said its own Google-Extended control keeps working alongside the shared one. Microsoft is adding a separate page-by-page setting for whether content can be quoted in AI answers from Bing and Copilot, and said opting out of that will reduce a site's visibility in those answers.

The case that this is real progress: Site owners get one switch instead of chasing a growing list of AI crawlers, and the four companies behind most popular chatbots say they will obey it. Google tied the effort to feedback from publishers, regulators and civil-society groups, and to the European Union's AI code of practice, which treats robots.txt as the main way for sites to state their wishes.

The case that it falls short: Following the signal is voluntary, the default is still yes to training, and nothing in it touches content already used to train today's models. It doesn't pay anyone. It arrives after years of fights over AI scraping, including The New York Times' 2023 lawsuit against OpenAI and Microsoft, which the companies are defending by arguing their use is legal fair use, AFP reported. Some site owners have gone further already: the web-security company Cloudflare has blocked AI bots by default on new customer sites since July 2025, AFP noted.

What's still unknown: Whether Meta, xAI, Apple or other AI developers will adopt the signal; exactly when OpenAI's rollout lands; whether the IETF turns the draft into a formal standard before it expires in May 2027; and how anyone outside the companies would check that the opt-out is being honored.

Sourcing: Google's Search Central blog, OpenAI's announcement and Microsoft's Bing blog, all dated Oct. 30, for each company's commitment, timing and limits. The IETF datatracker for the draft's text, author and status. The Verge for Meta's response and Anthropic's role in proposing the signal. AFP, via The Straits Times, for Anthropic's rollout, the New York Times lawsuit and Cloudflare's bot blocking. The IETF draft is an individual proposal and is not an approved internet standard.