In a recent LinkedIn post, Betsy Tong sounds a cautionary note regarding the unchecked use of Artificial Intelligence in generating critical business documents, sharing a stark example of how AI-generated content, even after internal review, can lead to significant professional and financial repercussions.
Tong highlights a concerning incident where a 237-page report delivered to the Australian government, which had passed an internal Deloitte review and been accepted by the client, was later found to contain fabricated references and a fake quote from a Federal Court case. This error necessitated a costly redo of the report and a refund of fees.
“Perfectly footnoted made up references. A fake quote from a Federal Court case.”
The author points out that the government’s insistence that the recommendations were still sound is what truly worries her, especially as she is currently involved in building a substantial investment case. This incident underscores a broader concern about the potential for AI to generate plausible-sounding but fundamentally flawed information.
The Perils of AI-Generated ‘Logic’
Betsy Tong explains that her own experience with AI has led her to question the underlying logic and sourcing of AI-generated outputs. When she began probing the AI tool used for her investment model, asking about its reasoning, data origins, and sources, the answers were unsettling.
As Tong states:
“AI fills blanks with the most probable answer. ❌ Not your logic. ❌ Not your constraints. ❌ Not your risk tolerance.”
This tendency for AI to fill gaps with probable, rather than factual or constraint-aligned, information is a critical flaw, according to Tong. She argues that AI does not inherently understand or incorporate an individual’s specific logic, business constraints, or risk tolerance. This lack of deep contextual understanding means that AI-generated content might appear polished on the surface but can falter under rigorous scrutiny by those who intimately understand the data and business context.
Strategies for Responsible AI Integration
While acknowledging the potential of AI, Betsy Tong emphasizes that a “better prompt cannot fix this” fundamental issue. Instead, she offers a practical, three-step approach for individuals who need to deliver work that will be deeply reviewed by knowledgeable audiences.
1. Define the Decision-Maker and Expectations
Tong advises clearly naming the intended user of the AI-generated output and outlining what they will expect as a next step. This provides crucial context for the AI.
2. Supply Verifiable Evidence
Instead of letting AI assume missing information, users should provide the actual facts, data, and source material. Tong stresses instructing the AI not to fill in gaps with assumptions.
3. Implement Constraint Locking and Red Teaming
A key recommendation is to explicitly define and lock down constraints such as budget, timing, authority, regulations, or internal politics. Furthermore, Tong suggests having the AI “red team” its own work, meaning it should be prompted to expose its assumptions, identify unsupported claims, and argue against its own recommendations.
Betsy Tong concludes by reiterating that without these safeguards, AI will invariably fill in the blanks with probable, but potentially inaccurate, information. She urges a more thoughtful and rigorous approach to leveraging AI, particularly for high-stakes business deliverables.
📝 About This Content
This article is based on insights shared by Betsy Tong on LinkedIn.
📅 Originally posted on September 1, 2026 | View original post on LinkedIn →