In a recent LinkedIn post, product strategist Melissa Perri shares critical insights into the unpredictable nature of working with AI models, drawing from her team’s experience rebuilding a website using an AI coding assistant. Perri highlights the challenges that arise when AI behavior shifts unexpectedly, impacting product development timelines and team trust.
Perri begins by recounting a jarring experience where the AI tool her team was using, Claude, seemed to dramatically decrease in capability overnight. This shift occurred without any changes to their own processes or instructions, leading Perri to question the stability of the AI itself. She notes:
“Nothing on our end had changed. The model had changed underneath us. And that is one of the strangest things about building with AI. Your product can be working beautifully on Monday and feel broken on Tuesday, not because your team did anything, but because the model did.”
The Shifting Sands of AI Dependencies
Perri theorizes that the perceived degradation in performance might be linked to the release of new AI models. She suggests that older models may be intentionally made to seem less capable as a prelude to a new release, a phenomenon her team experienced when transitioning back to a previous version of the model after testing a newer one.
“When we switched back to Opus, we were running the exact same processes and instructions we had used the week before. It is hard to describe the feeling unless you have lived it. One day the thing just gets it. The next day it feels so stupid, missing things it had handled easily days earlier,” Perri describes.
Rebuilding Trust in an Unreliable System
This unpredictability, Perri argues, fundamentally erodes trust. When a tool that has been consistently reliable suddenly falters, teams may begin to over-rely on it, only to be blindsided by regressions. As Perri points out, this reliance can quickly “turn into a liability.”
The team’s response was to adapt their processes, reinforcing the need for increased oversight and a healthy dose of skepticism. Perri elaborates on their strategy:
“Our instinct became ‘the more instructions the better,’ adding structure to hold the behavior steady when the model got shaky. We added checkpoints, reviewed each other’s work more closely, and brought back a healthy dose of mistrust.”
The Importance of Process Over Tool
Perri emphasizes that the established processes and guidelines her team had previously implemented were crucial in catching these regressions before they impacted the final product. This underscores her central argument: while the AI model itself cannot be controlled, the team’s processes, standards, and judgment are within their purview.
“You cannot control the model. You can only control your process, your standards, and your judgment. That is the part that is actually yours,” Perri concludes.
She frames this challenge as an operational problem rather than a purely technical one, suggesting that most teams currently lack a framework for continuously monitoring and adapting to the evolving nature of AI models. Perri’s experience serves as a stark reminder that building with AI requires a dynamic approach, prioritizing robust processes and vigilant oversight to navigate the inherent instability of these powerful tools.
📝 About This Content
This article is based on insights shared by Melissa Perri on LinkedIn.
📅 Originally posted on August 4, 2026 | View original post on LinkedIn →