Fireworks AI co-founder Chenyu Zhao told the Slush conference in Helsinki that open-weight AI models now trail frontier systems by three to six months, and that companies spending $10,000 to $100,000 a month on tokens should start moving routine work to them. The pitch comes from a vendor that disclosed two security incidents this fall. Those incidents exposed AI chat transcripts from Notion workspaces and, according to TechCrunch, personal data on about 150,000 Uber drivers.
Key takeaways
Zhao’s answer: a frontier-only stack is the right call early and the wrong one at scale. Before product-market fit, paying for the best frontier model is low-risk: the team is still learning what the product is, and model quality covers a lot of mistakes. Once a product reaches 10,000 or 100,000 users and token spend lands between $10,000 and $100,000 a month, he argues open source deserves a serious look.
He claims three gains: smaller models that cost less, faster responses, and systems that competitors can’t replicate just by calling the same API. “If you’re just using Opus or Sonnet, that’s not a differentiator,” he said. He offered no model versions or benchmarks, so treat this as a thesis to test on your own workloads.
The threshold is what’s useful: it turns “should we switch models?” into a line item on the monthly invoice.
By Zhao’s estimate, open-weight AI models now trail frontier systems by at most six months, “probably closer to three.” A few years ago he put the gap at one to one-and-a-half years, and he credits DeepSeek and the wave of strong open releases that followed with closing most of that gap.
One number hides variance: a model can match the frontier on summarization and lag badly on multi-step tool use. Measure the gap on your own evaluation set.
“Open-weight” means published parameters, not access to training data or code, and some licenses restrict commercial use or redistribution. Put license review in the same ticket as the latency tests.
Fireworks is built on this bet. The four-year-old Bay Area company runs and tunes open-weight models on each customer’s data; Uber, Shopify and Notion have been customers. Zhao calls it “a specialized intelligence platform,” in contrast to OpenAI and Anthropic, which he says “are building general intelligence.”
Zhao, who spent a decade at Google and ran a 50-person engineering team on Google Cloud Vertex AI, co-founded the company with Lin Qiao and Dima Dzhulgakov.
Zhao expects frontier models to handle high-level planning while cheaper tuned models take the basic tasks that never needed frontier intelligence, with a router assigning each request by complexity and cost. “That’s why there’s so much interest in routing, and why people now think about cost per task, not just cost per token,” he said.
Cost per task is the right metric, and it is harsher than it sounds. A cheap model that fails a third of the time, triggering retries, escalations or human review, can cost more per completed task than the expensive one did.
The router also becomes production code on the decision path. It needs:
We cover the operational side in our playbook for AI agents in production.
Some Fireworks customers worry that frontier labs absorb their domain know-how, possibly even through prompts, and Fireworks has made that worry part of its pitch. It echoes Microsoft CEO Satya Nadella’s July warning that users of proprietary models pay twice: once in money, once in know-how.
Zhao frames specialization as ownership. “That way you own your AI rather than being beholden to giant AI conglomerates,” he said. Tuned weights encode your workflows and data, and a rival cannot rent them.
For pharma, healthtech and fintech teams, a model running in an environment you control is easier to explain to an auditor, one argument for private LLM deployment. But if a vendor still hosts your tuning data and batch jobs, you have moved the trust problem, not removed it.
Fireworks disclosed two separate incidents this fall. In the first, attackers reached API keys and tokens stored as secrets for a limited number of accounts. In the second, a breach at Fireworks’ cloud provider let attackers into a development system, where they took customer data, including billing records and some datasets.
Zhao blamed much of it on “LLM jacking”: attackers hunting across providers for keys that unlock token access, then selling that access on illicit marketplaces. Fireworks named no threat actor, malware or initial-access technique. Zhao called the attack frequency unprecedented but offered no numbers.
Fireworks called the affected group “a small subset” and declined to name others. Under GDPR, controllers generally have 72 hours from awareness to notify regulators, and most US state breach laws cover driver’s license numbers. Your vendor contract has to deliver evidence fast enough to meet your own clock.
Not necessarily. Fireworks promises zero data retention for inference traffic, and Zhao said “there’s no indication that has been compromised.” Notion’s exposure came from another product: “Notion was using batch prediction, where you upload a dataset that’s stored at rest, so it’s not a zero-retention product by nature.”
Both statements can be true, which is the problem for buyers. Batch jobs scatter copies of your data across inputs, outputs, job queues, temporary objects, error logs and backups. A promise that covers only the serving endpoint says nothing about the rest.
Fireworks says it is moving batch services to an online flow that keeps data only as long as needed. It gave no completion date or broader remediation plan. Ask every inference vendor to list, per product, where your data rests and which copies are deleted versus left to expire.
The money keeps flowing despite the incidents. Fireworks was reportedly valued at $17.5 billion this summer, is backed by Sequoia and Benchmark, and is said to be in early talks for another round; Zhao declined to comment. Together AI and Baseten raised large rounds in recent months.
On Wednesday, Palo Alto Networks agreed to buy Applied Compute, a one-year-old startup founded by former OpenAI researchers that, like Fireworks, trains and runs custom models on customer data. The price is $4.5 billion, all in stock.
Zhao spoke on the day Anthropic went public in what was described as the largest IPO on record; he said he would probably buy a few shares as a long-term bet. Nvidia has backed Fireworks since 2024, and Zhao expects it to keep investing in Nemotron, its open-model family.
Valuation buys runway, not security maturity. Score vendors on controls and contract terms.
Start with credentials, because that is what LLM jacking targets. A leaked key’s blast radius depends on its permissions, lifetime and network restrictions.
We wrote about the wider problem in non-human identities and AI agents, and this audit is usually step one of our AI integration and MLOps work.
Fireworks still has not said how the secrets were taken, which cloud provider was breached, how many customers were affected in total, or when the batch-to-online migration will finish.
Frontier models are the most capable general systems from labs such as OpenAI and Anthropic, usually accessed through an API. Open-weight models publish their parameters so you can run and tune them yourself, though licenses may limit commercial use.
Fireworks' Chenyu Zhao suggests evaluating open models after product-market fit, at roughly 10,000 to 100,000 users and $10,000 to $100,000 a month in token spend. Measure quality on your own tasks before moving traffic.
Fireworks disclosed stolen API keys and tokens for a limited number of accounts, plus a cloud-provider breach that reached customer data. Notion transcripts from under 1% of workspaces were exposed, and TechCrunch tied a 150,000-driver Uber theft to Fireworks.
LLM jacking is the theft of API keys or cloud credentials to run AI models at the victim's expense, often with access resold on illicit marketplaces. Unusual token volumes from unfamiliar networks are a key warning sign.
If you are weighing a frontier-plus-open-model architecture or auditing an inference vendor, Artezio builds and reviews these systems for regulated teams. See our AI development services.