In its latest blog, cloud connectivity company Cloudflare accused Aravind Srinivas’s firm, Perplexity AI of “repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity.” The company stated that this behavior is “in direct conflict with explicit no-crawl preferences expressed by websites.”
Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites. https://t.co/yToVAmwcwn
— Cloudflare (@Cloudflare) August 4, 2025
Mathew Prince, the CEO of Cloudflare, quipped, “Some supposedly “reputable” AI companies act more like North Korean hackers. Time to name, shame, and hard block them.”
Reacting to one of the several replies on his post, the Cloudflare CEO stated that “There are other bad actors. And we’ll call them out too. But others, like OpenAI, are actually being extremely responsible. You don’t have to be a bad actor to be a good AI company.” He also pointed out that Chinese company Bytedance “is very aggressive,” while DeepSeek “is quite well-behaved.”
When users pointed out that Google can also access content like Perplexity, Prince remarked that
Google too will have to pay for access to content as they transition from being a Search Engine to an Answer Engine.”
Meanwhile, one user, a founding investor at TechCrunch, asked, “Is an AI bot that learns in order to produce original content the same as a crawler that indexes in order to reproduce?” Prince responded by dubbing the query as “an interesting legal question in the context of copyright” but “irrelevant.” He continued, “I’d look somewhere else: is there a fair value exchange? With Search Engines, the fair value exchange was you let them have your content in exchange for them sending you traffic you could derive value from.” “With Answer Engines, they take your content and send you…? If you’re not getting anything in return, why would you give up your content?” he further added.
Some supposedly “reputable” AI companies act more like North Korean hackers. Time to name, shame, and hard block them. https://t.co/vqMzGRHZPf
— Matthew Prince (@eastdakota) August 4, 2025
There are other bad actors. And we’ll call them out too. But others, like OpenAI, are actually being extremely responsible. You don’t have to be a bad actor to be a good AI company.
— Matthew Prince (@eastdakota) August 4, 2025
Bytedance is very aggressive. Deepseek is quite well behaved.
— Matthew Prince (@eastdakota) August 4, 2025
Google too with have to pay for access to content as they transition from being a Search Engine to an Answer Engine.
— Matthew Prince (@eastdakota) August 4, 2025
Think that’s maybe an interesting legal question in the context of copyright, but sort of irrelevant. I’d look somewhere else: is there a fair value exchange? With Search Engines, the fair value exchange was you let them have your content in exchange for them sending you traffic…
— Matthew Prince (@eastdakota) August 4, 2025
Meanwhile, Perplexity AI responded with a blog entitled, “Agents or Bots? Making Sense of AI on the Open Webs.” The company stated that
Modern AI assistants work fundamentally differently from traditional web crawling.”
It claimed that “Perplexity’s user-driven agents do not store the information or train with it.” it further slammed Cloudflare, stating, “When companies like Cloudflare mischaracterize user-driven AI assistants as malicious bots, they’re arguing that any automated tool serving users should be suspect—a position that would criminalize email clients and web browsers, or any other service a would-be gatekeeper decided they don’t like.”
Perplexity AI further remarked that
Overblocking hurts everyone.”
Taking a dig at Cloudflare CEO’s posts, Perplexity AI stated, “The bluster around this issue reveals that Cloudflare’s leadership is either dangerously misinformed on the basics of AI, or simply more flair than cloud.”
The bluster around this issue reveals that Cloudflare’s leadership is either dangerously misinformed on the basics of AI, or simply more flair than cloud. https://t.co/NgliGZCspP
— Perplexity (@perplexity_ai) August 5, 2025
See Also: Cloudflare Launches Way To Charge AI Bots For Crawling Sites
See Also: One Company’s Devious Plan To Stop AI Web Scrapers From Stealing Your Content
See Also: Perplexity AI CEO Warns Gen Z: Stop Doomscrolling on Instagram, Use AI Instead
Cover: Patrick Gawande / Mashable India
Great Job James Paul & the Team @ Mashable India tech Source link for sharing this story.
