|
Welcome back!
Chinese AI that anyone can download for free has become good enough to break into corporate networks, said Ariel Zeitlin, co-founder and chief technology officer of Tenzai, an Israeli startup that sells hacking software for testing corporate defenses, known as continuous penetration testing.
At the same time, the most advanced AI that had an early lead in cybersecurity capabilities is getting harder and costlier to improve, said Micah Hill-Smith, CEO of AI benchmarking firm Artificial Analysis. That means Chinese models won’t be far behind and makes it unlikely that any one developer pulls far ahead of its rivals.
The founders made the comments while speaking to The Information subscribers on Wednesday in a conference call after their companies appeared in our list of the 50 most promising startups of 2026. Security startups featured more heavily in this year’s list than in past editions.
“No one is restricting access to” the freely available Chinese open-weight models, Zeitlin said. “I think this is the real threat for…organizations.”
He pointed to an open-source hacking tool, free to download, that has already been used to breach organizations. He didn’t name it, but security firms recently documented that attackers have used ARTEX, an AI hacking framework posted to GitHub in July. CrowdStrike said a China-linked group used it against more than 200 organizations in 11 days.
“You don’t need to be a skilled hacker to do that,” Zeitlin said.
Breaches Coming
Zeitlin expects companies to endure a rough stretch of at least several months, if not years, with breaches disclosed daily, mostly because of software misconfigurations that AI makes easier to find. The biggest companies Tenzai works with, which employ thousands of security staff, are vulnerable too, he said. “Fixing is harder than finding,” he said.
While Zeitlin has a financial incentive to make such statements, many security officials, executives and AI researchers developing models share such sentiments.
Recent security lapses at AI developers such as OpenAI and Anthropic reflect startups’ looser habits, Zeitlin said, whereas big banks are better prepared. In the long run, he’s optimistic, saying that AI-written code will help companies avoid some bug types, such as injection flaws, but not the ones that arise when AI and applications interact.
The free models worrying Zeitlin aren’t likely to fall far behind the most advanced ones, according to Hill-Smith. He said the race among AI developers has stayed “more competitive than expected.” OpenAI and Anthropic hold an edge of a few months at the frontier—“arguably Anthropic this week,” he said—with other U.S. and Chinese labs close behind.
He said companies should keep investing in using open-weight models because “the ecosystem will be there,” meaning closed-source models won’t race ahead too far. Despite the lead held by closed-source models from the likes of Anthropic, open-weight models have continued to improve more than Hill-Smith expected because developers of such models get a lot of commercial opportunities from the notoriety their releases generate, he said.
A Limit on Scaling Laws?
Hill-Smith seemed to dismiss the idea of recursive self-improvement. He said that a so-called “fast takeoff” in which one developer’s models improve themselves so quickly that rivals can’t catch up was “pretty unlikely.” And in general, AI progress is bound to slow rather than speed up, in part because it will get harder for human experts to evaluate the work of new AI under development because it may already be on par with their level of expertise.
In fact, he said, AI capability improvements per dollar of investment will keep shrinking, which is why companies like Anthropic and OpenAI need so much capital. And measuring their progress is going to get tricky if such companies are spending 10 times more to produce a model that’s twice as good.
“The idea that it gets harder to make progress, I think, is true and will continue to be true,” he said.
On the bright side, the cost to use AI for a given task at the same quality level is falling five to 10 times a year, he said, even as some of his firm’s tests of long-running agents now cost hundreds of dollars per task.
Thank you for reading the Applied AI newsletter! I’d love your feedback, ideas and tips: aaron@theinformation.com.
If you think someone else might enjoy this newsletter, please pass it forward or they can sign up here.
|