In a recent LinkedIn post, Linas Beliūnas discusses a significant development from Cloudflare that he views as ironic given the company’s history in web security. Beliūnas highlights Cloudflare’s new “.crawl” API, which he suggests simplifies the process of web scraping, a practice Cloudflare has historically worked to prevent.
The Irony of Cloudflare’s New API
Linas Beliūnas points out the apparent contradiction in Cloudflare’s strategic shift. For years, Cloudflare has been a prominent player in providing anti-bot and anti-scraping solutions, essentially building infrastructure to protect websites from automated data extraction. Now, the company has introduced an API designed to facilitate this very activity.
“Tech loves irony: Cloudflare, the company that built half the internet’s anti-scraping infrastructure, just made scraping easier 😳”
Beliūnas elaborates on the functionalities of this new API, emphasizing its ease of use compared to traditional scraping methods. He notes that the API allows users to:
- Send a single API call.
- Have Cloudflare render the page.
- Receive content in HTML, Markdown, or JSON formats.
- Benefit from automatic extraction of links and page elements.
The absence of browser automation or the need for complex scraping scripts is a key feature that Beliūnas underscores. This streamlined approach, according to Beliūnas, significantly lowers the barrier to entry for web scraping.
Implications for AI and Data Extraction
According to Linas Beliūnas, this new offering from Cloudflare is particularly noteworthy for companies in the artificial intelligence sector. The ability to easily and efficiently gather large amounts of web data is crucial for training AI models.
“AI companies will love this,” Beliūnas states in his post.
Beliūnas’s observation suggests that the “.crawl” API could become an indispensable tool for AI developers and data scientists who require vast datasets for their work. This development potentially shifts the landscape of data acquisition, making it more accessible and less resource-intensive.
A Shift in Cloudflare’s Stance?
The core of Linas Beliūnas’s analysis revolves around Cloudflare’s apparent pivot. By offering a tool that simplifies scraping, Cloudflare seems to be moving from being solely a gatekeeper against it to also being an enabler. As Beliūnas puts it:
“Cloudflare spent years fighting scrapers and selling anti-bot protection. Now they launched a /crawl API that can scrape an entire website with one request 🤖”
This move raises questions about the future of web scraping and the evolving role of companies like Cloudflare in managing internet traffic and data access. Beliūnas’s post invites reflection on how such tools might be used and the potential consequences for website owners and the broader internet ecosystem.
In his post, Beliūnas also shared a personal note about leveraging AI for productivity:
“P.S. check out how I Turned Claude Cowork Into My Personal COO that does work while I sleep 🧠: https://lnkd.in/eS6JCk2G”
While this personal anecdote is separate from the main discussion on Cloudflare, it reflects Beliūnas’s broader engagement with AI and automation tools.
📝 About This Content
This article is based on insights shared by Linas Beliūnas on LinkedIn.
📅 Originally posted on March 12, 2026 | View original post on LinkedIn →