Rand Fishkin’s LinkedIn Post: Navigating AI’s Content Crawling Demands for Publishers and Brands

In a recent LinkedIn post, Rand Fishkin discusses the critical implications of Artificial Intelligence (AI) content crawling and ingestion for both publishers and brands. Fishkin offers a starkly different perspective for each group, urging publishers to protect their content while advising brands to embrace AI’s reach.

Publishers’ Stance: Protecting Content Value

Fishkin’s primary message to publishers is one of caution and protection. He strongly advises against allowing AI models to freely crawl, index, and ingest their content without compensation. The core of his argument rests on the perceived lack of sufficient value exchange for publishers in such a scenario.

Do NOT let AI crawl, index, and ingest your content without forcing them to pay. It’s just not worth it.

As Fishkin notes, the potential downsides of allowing unfettered access far outweigh any immediate, albeit often invisible, benefits. He implies that the long-term degradation of content value and potential loss of direct audience engagement are significant risks that publishers cannot afford to ignore. The emphasis here is on asserting control and demanding fair compensation for the data that fuels these powerful AI systems.

Brands’ Approach: Embracing AI’s Influence

In contrast to his advice for publishers, Fishkin suggests a more permissive approach for brands. He argues that brands should generally permit AI to crawl and ingest their content without demanding payment. This recommendation stems from a recognition of the evolving marketing landscape, particularly the rise of AI-driven search and information consumption.

Brands – You probably *should* let AI crawl, index, and ingest your content for free. The return of Zero Click Marketing may not be visible in your analytics, but if your audience is influenced by LLMs, it makes sense.

According to Fishkin, while the direct impact on traditional analytics might be negligible, the indirect influence of Large Language Models (LLMs) on consumer behavior is undeniable. He posits that even if the ‘Zero Click Marketing’ phenomenon isn’t directly measurable through current metrics, the strategic advantage of having brand content accessible and utilized by AI that influences potential customers is substantial. This perspective highlights a shift towards understanding influence beyond direct traffic metrics.

The Underlying Rationale

Fishkin’s differentiation between publishers and brands underscores a nuanced understanding of the AI ecosystem. For publishers, whose business model often relies on direct traffic and ad revenue generated from unique content, free AI ingestion can cannibalize their audience and devalue their core asset. As Fishkin points out, their content is the raw material that trains these AI models, and without a payment structure, they are essentially providing a valuable service for free, with little to gain.

Conversely, brands may benefit from increased visibility and influence within AI-driven information retrieval. Even if a user doesn’t click through to a brand’s website directly from an AI’s summary, the brand’s message and presence can still shape perception and decision-making. Fishkin’s analysis suggests that for brands, being part of the AI’s knowledge base, even without direct attribution in analytics, can be a strategic play for long-term brand building and market influence.

Fishkin concludes his post by referencing an article from Shahzad Abbas at Define, suggesting further reading on the topic. This indicates his belief that the conversation around AI and content ownership is complex and multifaceted, requiring ongoing discussion and analysis.