In a recent LinkedIn post, Teresa Torres discusses the complex challenges faced by content platforms in moderating user-generated content, particularly when off-the-shelf solutions fall short and the alternative involves human reviewers exposed to traumatic material. Torres highlights a conversation with the team from Musubi, an AI-native trust and safety toolkit, exploring their approach to building custom-trained machine learning models and LLM-powered moderation tools.
Torres introduces the core problem: the inadequacy of generic moderation scores and the ethical quandary of relying on human contractors for large-scale content review. She shares insights from her discussion with Nikki Marinsek, Brian McCaffrey, and Dan Means from Musubi, detailing their journey in developing AI solutions.
“What do you do when off-the-shelf moderation scores aren’t good enough—and the alternative is paying human contractors to spend their days reviewing traumatizing content at scale?”
The Limitations of Off-the-Shelf Moderation
As Torres points out, standard moderation tools often fail to capture the nuances required by diverse platforms. Each platform, whether a dating app, a social network, or an AI inference endpoint, has unique policies and content types that demand tailored moderation strategies. Off-the-shelf solutions, by their nature, cannot accommodate this specificity.
Torres elaborates on Musubi’s approach, explaining that they build custom-trained models. This process, as described in the post, involves more than just applying a pre-existing algorithm. It requires understanding the client’s specific needs and training AI to recognize patterns and violations relevant to those unique policies.
Custom AI: A Superior Alternative
The discussion with the Musubi team, as relayed by Torres, revealed a significant finding: their custom-trained AI models were sometimes more effective than human moderators. This discovery challenges conventional thinking and suggests a powerful role for AI in trust and safety operations.
AI Outperforming Human Moderators
Torres highlights the implications of AI’s potential to surpass human capabilities in certain moderation tasks. “You’ll hear how they balance latency, accuracy, and cost for clients handling hundreds of millions of actions per month,” she notes, underscoring the efficiency and scale that AI can offer. This capability is crucial for platforms dealing with massive volumes of content.
Iterating on Policies with AI
A key innovation discussed is Musubi’s policy optimizer. According to Torres, this tool utilizes agentic flows, allowing teams to refine their moderation policies without constant involvement from data scientists. This democratization of policy optimization is a significant step towards empowering non-technical trust and safety teams.
“building a policy optimizer that uses agentic flows to help teams iterate on their moderation policies without needing a data scientist in the room.”
Product Strategy and Future Directions
Torres also touches upon Musubi’s product strategy, emphasizing the importance of providing evaluation (eval) tools directly to customers. This approach, she explains, is central to their business model.
“why pushing eval tools directly to customers is their core product strategy”
Looking ahead, Torres indicates that Musubi is focused on developing flexible orchestration workflows. These tools aim to further support non-technical trust and safety teams, making advanced AI moderation more accessible and manageable.
The Role of AI as a Referee
The conversation, as detailed by Torres, also explored the use of AI as an intermediary. This includes AI acting as a judge to resolve discrepancies between AI-driven decisions and human judgments, creating a more robust and consistent moderation system.
“Using AI as a judge to referee disagreements between AI and human decisions”
In summary, Teresa Torres’s LinkedIn post provides a valuable overview of how custom-trained AI and LLM-powered tools are addressing critical challenges in content moderation, offering a more effective and ethically sound alternative to traditional methods.
📝 About This Content
This article is based on insights shared by Teresatorres on LinkedIn.
📅 Originally posted on June 11, 2026 | View original post on LinkedIn →