clet.xyz

The Ethereum Foundation’s Protocol Security team has successfully deployed coordinated Artificial Intelligence (AI) agents to probe the complex systems underpinning the Ethereum network, yielding tangible security vulnerabilities. Among the significant findings is a critical, remotely-triggerable panic in libp2p’s gossipsub, a fundamental component of the peer-to-peer layer used by Ethereum consensus clients. This vulnerability, now publicly disclosed as CVE-2026-34219, highlights the growing efficacy of AI in uncovering previously elusive security flaws within mission-critical infrastructure.

While the discovery of bugs was anticipated, the Ethereum Foundation team expressed surprise at the efficiency of the process and the significant effort required to differentiate genuine vulnerabilities from plausible-sounding but ultimately false alarms. This development signals a pivotal shift in how protocol security is approached, moving beyond traditional manual auditing to a more automated, AI-assisted model. The insights gleaned from this initiative are intended to benefit both Ethereum client development teams and the broader security research community.

The methodology employed by the Ethereum Foundation team is part of a larger, emerging trend in cybersecurity. Similar efforts are being undertaken by prominent organizations. Anthropic’s Frontier Red Team, for instance, developed an AI agent capable of generating property-based tests, which subsequently uncovered genuine bugs across the Python ecosystem. Cloudflare has also integrated frontier models into their security research frameworks, applying them to their own systems. The common thread across these initiatives is a recurring loop: directing advanced AI models toward codebases, enabling them to conduct extensive searches, and then meticulously triaging the results. The critical challenge, therefore, lies in navigating the sheer volume of output from these agents without being overwhelmed by misleading information.

This report focuses on the enduring methodologies rather than the rapidly evolving tooling. The specific technical implementations are subject to swift obsolescence, but the underlying principles of agent organization, validation criteria, and maintaining trustworthy results remain constant. Disclosure protocols are a complex topic that warrants separate, detailed examination.

AI Agents as Sophisticated Search Tools

At its core, an AI agent directed at a codebase functions as a sophisticated search tool, akin to a fuzzer. However, the nature of their output distinguishes them significantly. While a fuzzer typically presents a crash alongside a stack trace, an AI agent delivers a more comprehensive package. This includes detailed reports, often comprising a call chain, an assessment of potential impact, and a proposed severity level, all supported by demonstrable artifacts such as proofs-of-concept that can be executed against the actual code.

This rich output facilitates both ease of review and a high degree of confidence in the findings, with the executable proof-of-concept serving as the most compelling evidence. Consequently, the true measure of an AI agent’s success lies not in the quantity of candidate vulnerabilities it generates, but in the proportion of those candidates that are validated as genuine security flaws.

Organizing the AI Agent Workforce

The protocol security team’s operational strategy involves deploying numerous AI agents in parallel against a single target system. Coordination is achieved organically through the code repository itself, leveraging shared state within version control systems and eschewing a centralized process for task distribution. An agent articulates a potential vulnerability claim, making it visible to other agents, undertakes its investigation, and then commits its findings.

This decentralized coordination model was inspired by Anthropic’s approach to building a C compiler using a fleet of AI agents, which operates on a similar principle of shared repository state. This method minimizes the need for a central coordinator, thereby reducing potential points of failure and simplifying maintenance.

The roles and responsibilities within this framework emerge dynamically based on the nature of the vulnerabilities discovered. The workflow generally follows established stages:

• Reconnaissance: Initial exploration of the target codebase to identify potential attack surfaces and areas of interest.
• Parallel Hunting: Multiple AI agents concurrently search for vulnerabilities within their designated scopes.
• Independent Validation: Candidate vulnerabilities are subjected to rigorous, independent verification processes.
• Deduplication: Mechanisms are in place to ensure that the same vulnerability is not reported multiple times by different agents.
• Reporting: Verified vulnerabilities are documented with comprehensive details.

This structured pipeline draws inspiration from methodologies outlined by Cloudflare, which emphasizes a similar sequence of reconnaissance, parallel hunting, independent validation, deduplication, and reporting.

Defining a Verified Finding

For a candidate vulnerability to be officially recognized as a finding, it must meet a stringent set of criteria:

• Target: Identification of a specific component and entry point that an attacker can realistically exploit within the network’s normal operational configuration.
• Invariant: A clearly defined property or condition that must consistently hold true within the system’s operation.
• Mechanism: A precise description of how the invariant might be violated or compromised.
• Success: Observable and verifiable proof of the vulnerability’s manifestation, such as a system panic, a stall, or the acceptance of invalid input that leads to an exploitable state.
• Reproducer: A self-contained artifact, such as a script or code snippet, that reliably reproduces the identified failure when executed against the actual target code.
• Dedup: A unique identifier or key associated with the vulnerability to prevent redundant investigations by multiple agents.

This schema is deliberately designed to enforce specificity and a clear definition of "done." By requiring agents to articulate an observable proof of compromise, the process inherently discourages vague claims based on perceived risk or intuition.

The Imperative of Reproducibility

The most critical tenet governing AI-driven security audits is absolute reproducibility. A candidate vulnerability is not considered a genuine finding until a self-contained artifact can reliably reproduce the failure against the live code, and this reproducer must function for an auditor who did not author it.

The reproducer operates independently of the AI agent’s narrative, unconcerned with the confidence expressed by the model. It either works, demonstrating the vulnerability, or it does not.

The primary value of this strict reproducibility lies in its ability to swiftly identify and discard false positives. Over repeated applications, three common types of false positives emerge, each representing a scenario where the AI agent achieves a "pass" for incorrect reasons:

• The Spec-Based Bug: The agent correctly identifies a discrepancy between the code and its specification but fails to demonstrate a practical exploitability within the actual runtime environment. The code may deviate from the spec, but this deviation does not lead to a security compromise.
• The Trivial Bug: The agent flags a condition that technically constitutes a bug but has no meaningful security implication. This could be an edge case that is practically impossible to trigger or one that results in a harmless error.
• The "Fixed" Bug: The agent identifies a vulnerability that has already been addressed and patched in a previous version of the code. Without a mechanism to track known issues, agents might repeatedly discover and report already-resolved problems.

These issues echo the familiar problem of a test case that passes because it fails to adequately scrutinize the code it is meant to validate. The novelty with AI agents is the sheer volume and speed at which they can generate these misleading candidates, all delivered with an equally convincing demeanor. Therefore, the validation process must be automated and robust, as relying on the AI agent to self-correct is not a viable strategy.

The Triage: Navigating the Signal-to-Noise Ratio

The overwhelming majority of candidate vulnerabilities generated by AI agents are ultimately found to be erroneous, duplicates of previously identified issues, or outside the defined scope of the audit. This outcome is not indicative of a flawed methodology but rather a fundamental characteristic of the process. The objective is to efficiently filter out incorrect candidates while providing irrefutable proof for genuine vulnerabilities.

Each candidate that progresses through the initial stages undergoes two independent verification checks. The first assesses whether a real-world attacker, operating under normal network conditions, could actually trigger the vulnerability. The second evaluates the cost-benefit analysis for the attacker: what is the resource investment required to exploit the vulnerability, and conversely, what is the impact on the network if the exploit is successful? A vulnerability that can be triggered by any single peer carries significantly different implications compared to one requiring specialized access or substantial computational resources.

Furthermore, all candidates are cross-referenced against a continuously updated database of known, resolved, or rejected vulnerabilities. This is crucial to prevent the agents from repeatedly surfacing issues that have already been addressed.

The acceptance rate of candidate vulnerabilities varies considerably depending on the target codebase. Auditing mature, extensively vetted codebases typically results in very low acceptance rates, a finding that is itself valuable. The conclusion, "We conducted a thorough investigation and found no new vulnerabilities," is a legitimate and important security outcome. Conversely, when applied to less scrutinized code or systems that have undergone formal verification (where machine-checked proofs cover a model, and the deployed bytecode is assumed to align), a higher proportion of candidates may be accepted.

The consensus among organizations like Cloudflare is that a focused scope is more effective than broad, indiscriminate scanning. Anthropic’s property-based testing agent, for example, generated approximately a thousand candidate reports, which were then ranked and subjected to expert review, ultimately yielding a smaller set of high-confidence findings that held up with roughly 86% accuracy. The generation phase was comparatively straightforward; the refinement and validation were the more demanding aspects. Specific numbers for the Ethereum Foundation’s internal audits are not publicly disclosed, as they are highly contextual to the specific target code and may offer less insight into the general methodology.

AI’s Strengths and Limitations in Security Auditing

The discourse surrounding AI in cybersecurity often oscillates between extremes of hype and skepticism. A balanced perspective acknowledges both the capabilities and the inherent limitations of these tools:

The effectiveness of AI agents is not uniform across all tasks. Stanislav Fort’s research on testing various AI models against real-world vulnerabilities describes this phenomenon as a "jagged frontier," where a model capable of reconstructing a full exploit chain for one codebase may struggle with basic data-flow tracing on another. This variability underscores the necessity of validating each candidate vulnerability independently, as a single successful outcome does not guarantee consistent performance.

The final point in the table is particularly significant. While a single AI agent session excels at one-shot reasoning, it struggles with identifying vulnerabilities that manifest across a sequence of operations, where each individual step is legitimate, but their specific order creates a security flaw. In such cases, the AI agent’s role is not to perform the exhaustive search but rather to suggest which sequences warrant investigation using a stateful test harness. When employed in this supportive capacity, AI agents prove highly effective. However, when used as a direct replacement for such harnesses, they are likely to miss the most costly and insidious vulnerabilities – those that only emerge through complex, multi-step interactions.

Maintaining the Integrity of AI-Driven Audits

Several fundamental practices are instrumental in ensuring the trustworthiness of findings generated by AI agents. These habits, while not revolutionary, are critical for reliable security auditing:

• Reproducible Failures: Every candidate must be accompanied by a self-contained, executable artifact that consistently reproduces the identified failure. This artifact serves as the ultimate arbiter of truth, independent of the AI’s claims.
• Live Oracle Verification: The validation process must involve executing the reproducer against a live, operational instance of the target code. This ensures that the vulnerability is present in the actual deployed environment, not merely in a theoretical model or an outdated codebase.
• Comprehensive Triage: A structured and thorough triage process is essential. This involves independent review of the candidate, verification of the reproducer, assessment of the impact, and cross-referencing against known vulnerabilities.
• Centralized Known Issues Tracking: Maintaining a comprehensive and up-to-date database of all identified, resolved, and rejected vulnerabilities is paramount. This prevents redundant discoveries and ensures that efforts are focused on novel threats.
• Independent Security Review: Ultimately, human security experts must provide the final judgment. While AI agents can identify potential issues at scale, human expertise is indispensable for contextualizing findings, assessing true risk, and making strategic decisions about remediation.

The Evolving Bottleneck in Security Auditing

The advent of AI in security auditing has not rendered human security researchers obsolete; rather, it has fundamentally shifted the nature of their work. The time previously dedicated to formulating hypotheses and manually tracing potential exploits is now being reallocated to the critical task of evaluating these hypotheses at scale. This includes the development and maintenance of validation oracles, the rigorous execution of triage processes, the meticulous tracking of known issues, and the management of vulnerability disclosure.

The bottleneck has not disappeared; it has merely migrated. It has moved from the initial discovery phase to the phase of trusting and validating the AI-generated results. This is a more appropriate location for the bottleneck, as it is precisely where human judgment and expertise are most crucial. However, it remains a significant bottleneck, and neglecting its importance can lead to a false sense of security, where systems are deemed "fine" without adequate verification.

The practices that underpin effective AI-driven security auditing are not novel. Reproducible failure analysis, the establishment of reliable validation oracles, and meticulous triage are the same principles that transformed fuzzing from an academic research topic into an industry-standard practice over the past fifteen years. The tools may be new and rapidly evolving, but the foundational methodologies remain consistent.

The pace of change in AI tooling presents an ongoing challenge. Nicholas Carlini, a respected figure in AI security research who was initially a skeptic, argues that the exponential trajectory of AI capabilities warrants serious consideration, even while acknowledging the inherent uncertainties. If the rate of AI-driven vulnerability generation continues to accelerate, the capacity for human judgment and validation must keep pace. Failure to do so will result in an ever-widening chasm between the volume of claims generated by AI and the actual verification of those claims.

For the critical infrastructure upon which Ethereum relies, this dimension of rigorous verification is paramount. AI agents enable the exploration of a far broader attack surface than would be feasible through manual efforts alone. In return, these agents demand a more stringent and comprehensive approach to judgment, applied across a significantly larger pool of confident-sounding assertions. This exchange is undeniably beneficial, provided that the critical role of human judgment in the validation process is consistently recognized and prioritized.