In a recent LinkedIn post, Hiten Shah discusses the practical limitations of current AI tools when applied to conversion rate optimization (CRO), particularly in the context of landing page analysis. Shah shares insights from an experiment conducted by Crazy Egg, where AI landing page analyzers were used to optimize a page, with surprising results.
AI’s Shortcomings in Understanding Page Purpose
The core of Shah’s analysis centers on the disconnect between AI’s analytical capabilities and the nuanced purpose of specific web page elements. While AI tools can suggest structural and linguistic changes, they may fail to grasp the underlying ‘job’ each component is meant to perform for the user. Shah highlights this in his post:
“The tools missed the purpose of the page. They shifted structure and language without understanding the job each element performs.”
This observation is critical for businesses relying on AI for CRO. As Hiten Shah points out, the AI’s recommendations, while seemingly data-driven, can be misaligned with the actual user journey and the intended function of the page.
Experimental Findings and Data-Driven Insights
The experiment involved feeding a Crazy Egg page through major AI landing page analyzers. The top-ranked suggestions were implemented precisely as instructed, and the resulting page was then A/B tested against the original control version. After a three-week testing period, the AI-optimized page significantly underperformed the control.
Hiten Shah emphasizes that this outcome was not due to random changes but a fundamental misunderstanding by the AI tools. He elaborates on the gap between AI perception and page design intent:
“The gap between what these tools can see and what these pages are designed to do is still wide.”
According to Shah, this indicates that while AI can process and analyze data, its ability to interpret the strategic ‘why’ behind a page’s design and content is still developing. The experiment underscores the importance of human oversight and strategic understanding in CRO, even when leveraging advanced technology.
Evaluating AI for CRO: A Data-Grounded Approach
Shah suggests that teams evaluating AI for CRO should look beyond superficial demos and focus on data-grounded evidence. He directs readers to a more detailed breakdown of the experiment, emphasizing its empirical nature.
“If your team is evaluating AI for CRO, the full breakdown is worth reading. It’s grounded in data, not demos.”
In Hiten Shah’s view, the findings from this experiment provide valuable lessons for marketers and CRO specialists. While AI offers potential benefits in speeding up analysis, its current iteration may not be a substitute for a deep understanding of user behavior and page objectives. As he concludes:
“It’s a useful read.”
The insights shared by Hiten Shah serve as a timely reminder that while AI is a powerful tool, its application in complex areas like conversion rate optimization requires careful consideration, validation, and a clear understanding of its current limitations.
📝 About This Content
This article is based on insights shared by Hiten Shah on LinkedIn.
📅 Originally posted on December 9, 2025 | View original post on LinkedIn →