Mozilla has detailed its successful use of the Anthropic Mythos AI model to identify 271 security vulnerabilities within Firefox source code over a two-month period, attributing the high-fidelity results to both model improvements and a custom analysis harness. The engineers emphasized that this deployment yielded "almost no false positives," a significant improvement over earlier AI-assisted detection efforts that were plagued by hallucinated details and required extensive manual validation. The article does not specify the nature of the individual vulnerabilities discovered, their CVSS scores, affected versions, fixed versions, or any workarounds.
The disbelief was palpable when Mozilla’s CTO last month declared that AI-assisted vulnerability detection meant “ zero-days are numbered ” and “defenders finally have a chance to win, decisively.” After all, it looked like part of an all-too familiar pattern: Cherry pick a handful of impressive AI-achieved results, leave out any of the fine print that might paint a more nuanced picture, and let the hype train roll on. Mindful of the skepticism, Mozilla on Thursday provided a behind-the-scenes look into its use of Anthropic Mythos—an AI model for identifying software vulnerabilities—to ferret out 271 Firefox security flaws over two months. In a post , Mozilla engineers said the finally ready-for-prime-time breakthrough they achieved was primarily the result of two things: (1) improvement in the models themselves and (2) Mozilla’s development of a custom “ harness ” that supported Mythos as it analyzed Firefox source code. "Almost no false positives" The engineers said their earlier brushes with AI-assisted vulnerability detection were fraught with “unwanted slop.” Typically, someone would prompt a model to analyze a block of code. The model would then produce plausible-reading bug reports, and often at unprecedented scales. Invariably, however, when human developers further investigated, they’d find a large percentage of the details had been hallucinated. The humans would then need to invest significant work handling the vulnerability reports the old-fashioned way. Read full article Comments