News on Medial

Multiple AI companies bypassing web standard to scrape publisher sites, licensing firm says

Economic TimesEconomic Times · 5m
Multiple AI companies bypassing web standard to scrape publisher sites, licensing firm says

Artificial intelligence (AI) companies are finding ways to bypass web standards used by publishers to block content scraping for AI systems, according to a letter from content licensing startup TollBit. While the letter did not name specific companies or publishers, it highlighted a dispute between AI search firm Perplexity and Forbes. Perplexity has been accused of plagiarism by generating AI summaries of Forbes' stories without permission or proper citation. TollBit tracks AI traffic to websites and helps publishers establish licensing fees. The use of the "robots.txt" protocol by AI agents is being bypassed, posing challenges for publishers seeking to control their content.

Comments

Download the medial app to read full posts, comements and news.