Could a “dumber” AI model actually outperform the smartest state-of-the-art systems? Jesse Zhang and Ashwin Srinivas of Decagon argue that the industry’s obsession with frontier models like GPT-4 often overlooks a critical truth: for specific enterprise tasks, a fine-tuned, smaller open-source model is frequently faster, cheaper, and more accurate than its massive counterparts. By shifting 90% of their workflow to open source, Decagon has moved away from the “black box” approach of general-purpose AI toward a “model factory” that optimizes for latency and precise business process execution.

The conversation challenges the popular narrative that frontier labs (OpenAI, Anthropic) are the “last startups” and will eventually swallow all application-layer companies. The founders contend that while the “brains” are being commoditized, the “body”—the infrastructure, business logic, and regulatory guardrails required to deploy AI in a Fortune 100 company—remains a massive moat. They view their product not just as a customer support bot, but as an “AI Concierge” that serves as the front door to a business, handling everything from reactive support to proactive sales qualification.

A central theme of the discussion is the evolution of the “Forward Deployed Engineer” (FDE). Drawing on the Palantir model, the founders explain that FDEs are essential in the early days of AI because the workflows are currently being discovered in real-time. However, they warn that relying on FDEs long-term is a “trap” that creates a glorified consulting firm. The goal must be to “eat pain and excrete product,” turning those manual customer interventions into scalable software features, such as “Duet,” an agent that automatically writes its own operating procedures and tests.

Surprising Insights

• The Intelligence Trade-off Myth: Using smaller models isn’t necessarily about “dumbing down” the AI to save money; fine-tuning a small model for one specific task often results in higher performance than a general-purpose frontier model.
• Jevons Paradox in AI: Reducing the cost of customer support doesn’t always lead to layoffs; instead, it often increases the total demand for support as companies make it more accessible to free users or embed it on every page of their site.
• The “Glass Box” Advantage: Large enterprises are increasingly choosing platforms that give them direct control over AI journeys rather than “black box” services where they must rely on a vendor’s engineers to make every small change.
• AGI and Career Persistence: The founders argue that while AGI may eliminate specific “jobs” (mundane, repeatable tasks), it won’t eliminate “careers,” as humans will always create new roles and abstractions to serve other humans.

Practical Takeaways

• Audit Your Model Stack: If you are scaling a specific AI task, experiment with fine-tuning smaller open-source models to reduce latency and cost while potentially increasing accuracy.
• Productize the “Pain”: If you use forward-deployed engineers or consultants to implement AI for clients, treat every manual fix as a bug report. The goal is to automate that manual intervention into a core product feature.
• Focus on Business Logic, Not Just Models: Build the “infrastructure of trust” around your AI—including regulatory guardrails, collaboration tools for experts, and end-to-end testing—as this is where the long-term value resides.
• Adopt a “Concierge” Mindset: Move beyond seeing AI as a cost-cutting tool (reactive support) and start designing it as a revenue-generating tool (proactive sales and operational workflows).