Reddit blocks all search engines except Google as it implements AI paywall

Reddit has begun blocking all search engines except Google from indexing its site, effectively creating a paywall for AI training.

This follows a deal where Google pays Reddit over $60 million annually for access to user posts for AI model training.

The updated robots.txt file now prevents web crawlers from indexing Reddit, except for Google's. This move is intended to prevent misuse of public content, with Reddit stating that search engines must agree not to use their data for AI training.

While Reddit's spokesperson claimed this change isn't related to the Google deal, the exclusive access for Google indicates a "pay-for-play" model. Other search engines like Bing and DuckDuckGo no longer show recent Reddit posts.

Engadget reported that Reddit and Microsoft have failed to reach an agreement regarding Bing’s access, due to conflicts over AI data usage and Reddit's data privacy policy.