Meta Removes Controversial Language That Appeared to Permit News-Site Scraping

Introduction

Meta has revised its commercial terms again, removing language that drew criticism from publishers, legal academics, and people working in the web-content industry. The controversy does not establish that Meta was actually scraping news sites at scale. Instead, it centered on how the wording could be interpreted: the text appeared to give Meta broad permission to use material from websites, webpages, or URLs to develop and train its AI products.

Key points

• The deleted passage said Meta could use any content from relevant domains, webpages, or URLs to provide, develop, improve, research, train, and maintain its products and services.
• Nieman Lab and James Grimmelmann, a Cornell professor specializing in digital and information law, said the language could be read as an attempt to use scraped website material for free and potentially override robots.txt instructions.
• Meta previously pushed back through its communications team, saying its use of web crawlers had not changed and that suggestions to the contrary were false.
• In the latest update, Meta removed the disputed passage and clarified that the consolidated crawling terms applied only to its advertising and business products.
• Meta also said the revision does not grant it new rights that supersede the preferences of content owners. Websites can still use robots.txt to indicate whether they want to be crawled.

Why the dispute matters

The episode highlights the increasingly unsettled boundary between AI companies and online publishers. As training data, web crawling, and generative-AI products become more commercially important, a single sentence in a terms-of-service document can raise questions about authorization, permitted uses, and the role of technical opt-out mechanisms. Even when a company has not changed its underlying technology, broad contractual language can create the impression that it is claiming rights first and explaining limits later.

For publishers, robots.txt is a widely used technical way to communicate crawling preferences. Whether those instructions create a binding legal restriction in every commercial and legal context is a more complicated question, however. It can depend on the wording of a contract, the actual conduct of a crawler, and the law that applies. Meta’s deletion therefore does not settle wider disputes over copyright, licensing, or compensation for AI training data, but it does reduce the room for the terms to be misunderstood.

Meta’s statements also draw a line between two issues: what its commercial terms say about crawling, and how its existing web crawlers operate in practice. The revision looks more like a clarification and risk-control measure than an announcement of new scraping capabilities. More broadly, the incident suggests that AI platforms may need to distinguish more clearly among advertising services, business tools, and AI training, while explaining more openly how third-party content is handled.

The available report does not establish that Meta used the disputed language to scrape or train on news content.

Source: The Verge AI