We are being forced to rediscover something older and, in the long run, more robust: that public truth has always depended on representation, provenance, procedure, corroboration and the ability to contest.
- Audrey Tang, “Humanism After Optimization” (2026)
Dario Amodei’s recent blogpost, We Must Pace the Frontier, calls for “embedded evaluators” as the first of three policy steps necessary to slow the pace of frontier AI development. He writes:
Embedded Evaluators. Each frontier AI company [should] commit to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work. (emphasis added)
Amodei later explains:
Embedding evaluators may sound like a small or inconsequential step, but often the things that sound most boring or procedural are actually the most essential. Embedded evaluators are in fact a quite radical practice that goes far beyond what any AI company is doing today.
Sam Altman and Elon Musk quickly endorsed the idea, signaling that this may become an industry norm.
Amodei’s banking supervision analogy is apt. Based on my experience, though, reliable access to information will need to be complemented by information sharing across evaluators and a credible path from findings to remediation to truly be effective.
Credible verification is invaluable, but not enough
In 2004, I left the Federal Reserve as a lawyer and joined a newly-created unit at the Securities and Exchange Commission as a supervisor. The unit’s mandate was to prudentially supervise the large independent investment banks — Bear Stearns, Goldman Sachs, Lehman Brothers, Merrill Lynch, and Morgan Stanley. Until then, the SEC had been only a markets regulator, not a prudential supervisor.
Our approach to supervision was “trust, but verify”. Each investment bank had risk management policies and procedures that looked good on paper. Our job was to take them at face value and kick the tires to see if things in practice worked as well as claimed.
We had access to a wide range of internal reports and employees, and spent most of our time onsite interfacing with risk managers, treasurers, controllers, and internal auditors. When we found weak practices and pointed them out, firms generally addressed them. Escalation was rarely needed.
What made this work was trust. Certain other units at the SEC had adversarial relationships with the investment banks. Those meetings typically involved lawyers, where information flows were highly managed, communications were formal and written, and the threat of an enforcement action loomed large.
Our “trust, but verify” approach stood in contrast to that. We asked a lot of questions, conducted sound analysis, provided credible feedback, and diligently followed-up. We defined success as identifying the right weaknesses and getting them addressed, not by playing gotcha. The substance mattered more than the form. Sometimes we wrote a report comparing the firms’ practices, as we did for independent price verification and leveraged lending. More often than not, though, the Columbo approach of asking sharp questions was enough to steer firms to address the key issues we flagged.
This worked well, until 2007.
That year, cracks began to appear in the capital and money markets. A range of financial institutions, including several investment banks, had gotten out over their skis aggressively chasing profits in the rapidly growing structured finance and derivatives markets. Our supervisory feedback, which had gotten sharper, increasingly was not heeded. At some firms, risk managers, treasurers, and internal auditors, who agreed with our findings, were themselves getting rolled by their front office colleagues and ignored by senior management.
By the time we sought more formal escalation, it was too late. In October, Merrill Lynch took the first of several multi-billion write-downs. In December, Morgan Stanley booked a $9 billion writedown. In March 2008, Bear Stearns collapsed, purchased for $2/share by JPMorgan with support from a novel Fed backstop. In mid-September, Lehman filed for bankruptcy and Merrill sold itself to Bank of America. A week later, Goldman Sachs and Morgan Stanley opted to become bank holding companies, gaining access to the Federal Reserve’s discount window and coming under full Fed oversight.
The SEC’s prudential supervision program, which was strong on verification, but weak on authorities, ended up being a short-lived experiment.
Vertical and horizontal perspectives are both needed
After the collapse of Lehman, I moved to the Treasury Department to help firefight the crisis. In 2010, I returned to the Fed as a supervisor to help reorganize large bank supervision.
The reorg was eye opening. Legally, supervision at the Fed is delegated from the Board of Governors in DC to the twelve Reserve Banks. This meant, for instance, that the Federal Reserve Bank of New York oversaw Citi and JPMorgan while FRB San Francisco oversaw Wells Fargo and FRB Richmond oversaw Bank of America.
Even though each bank was subject to the same regulations and supervisory expectations and Fed supervisors were required to generate the same outputs, the variation across supervisory teams in what they did and how they did it was enormous. The inconsistencies contributed to blindspots that gave the Fed a false sense of comfort leading into the crisis.
The reorg sought to address this, partly through greater centralization and partly through greater use of “horizontals”. Prior to the crisis, the vast majority of supervisory activities were carried out by vertical onsite teams: one team per bank. While this allowed supervisory teams to gain intimate knowledge of each bank’s people, processes, and systems, it created silos with limited information sharing between them. Insights and learnings stayed within each silo and credible comparisons across banks were rare, enabling a Lake Woebegone effect, where each team thought their bank was above average.
The post-crisis large bank reorg at the Fed tackled this head on. The horizontal teams covering different risk stripes — market risk, credit risk, liquidity, etc. — were elevated. Supervisory assessments of firms were structured to weigh both vertical and horizontal perspectives. Most importantly, barriers to information sharing between teams were eliminated, allowing for stronger risk identification and greater consistency across supervisory assessments.
Interestingly, at the SEC, we were able to structurally balance vertical and horizontal perspectives by using a staggered team configuration. Each supervisor sat on three firm-specific, partially-overlapping teams, covering a different risk stripe at each firm. Thus, the team members covering any one firm collectively had visibility into all five firms in the portfolio. And every supervisor was part of three horizontal risk teams. (For instance, I covered credit risk at Goldman, liquidity risk at Morgan, and market risk at Merrill.) This meant that every team member was just one degree of separation away from detailed information anywhere in the portfolio, and that every piece of information had vertical and horizontal context — all with a minimum of bureaucratic process.
Formal authority is necessary, but must be wielded with care
In contrast to the SEC’s short-lived prudential supervision program, federal banking agency supervisors can utilize a wide range of formal tools to compel remediation at banks — from issuing Matters Requiring Attention (MRAs) and Matters Requiring Immediate Attention (MRIAs) to downgrading banks’ supervisory ratings to putting on public enforcement actions and imposing civil money penalties or even asset caps.
With great power comes great responsibility. While banking supervisors should never hesitate to use the right tool when necessary (see SVB) they must take great care when exercising power with consequences. Supervisory actions that lack credibility or supporting analysis sow the seeds of distrust in supervision. This distrust can quickly become self-fulfilling for both sides: banks withhold information to mitigate the risk of un-credible actions, which raises the suspicions of supervisors and sensitizes them to see issues where they otherwise wouldn’t, which reaffirms banks’ concerns about overreach and the need to withhold information, and so on.
To be clear, an adversarial posture between supervisors and supervised institutions is unavoidable and at times can be necessary to achieve public interest objectives, especially where the supervised institution is unwilling or unable to fix what is needed. But it isn’t a sustainable steady state condition. To be effective, supervisors must be able to traverse non-coercive and coercive states.
Academics Peter Conti-Brown and Sean Vanatta have studied the history of banking supervision and found that supervisors play a wide range of roles — from cop on the beat to fire warden to management consultant. They plot them along two axes: public to private and coercive to non-coercive.
Historically, building and sustaining trust in the banking system has required robust mechanisms in all four quadrants. No one mechanism has been able to do all of the work needed to sustain trust through business cycles. As Conti-Brown, and Vanatta note, when supervisors were most effective, they successfully exercised discretion to play different roles in different contexts, as situations demanded.
If one were to draw this graphic for the AI industry today, it might look like this:
The only two mechanisms that are live today are in dark blue — the U.S. government’s ad hoc review of frontier models that Anthropic and OpenAI appear to be abiding by, and private alignment efforts at the labs, some of which are disclosed publicly via publications and blogposts.
Currently under consideration, in light blue, are Amodei’s embedded evaluators and Demis Hassabis’s self-regulatory organization (SRO) proposal, modeled on FINRA. As proposed, the embedded evaluators have two functions — verifying that a lab is adhering to its safety protocols and investigating incidents. The former is private and for the labs, while the latter is public through the issuance of investigation reports.
Much farther behind are the coercive mechanisms of litigation and enforcement, as well as a formal regime for licensing and a formal incident review body like NTSB (which I’ve argued for here).
As one can see, the current set of trust-building initiatives proposed for AI are clustering in the lower right (private, non-coercive). As a start this makes sense, as this is where the possibility space is today. To build durable trust with the public in the long-term, though, robust mechanisms across all four quadrants will be needed.
Three lessons
Thus, at a high level, three key lessons from banking supervision are applicable to the embedded evaluator idea:
Verification needs to be paired with authority. An independent entity — e.g., an evaluator or a government body — will need the authority to compel remediation when a lab is unwilling or unable to do so.
Vertical teams need to be able to collaborate. Evaluator teams will need to be able to share information and compare notes with each other to be effective and mitigate Lake Woebegone risk.
Embedded evaluators are just one part of the trust building solution. Public trust requires “representation, provenance, procedure, corroboration, and the ability to contest”. Embedded evaluators would represent a significant step towards that and should be strongly supported. But the AI industry should also be clear-eyed that evaluators are just one of several mechanisms that need to be established to build robust trust in AI.
Anthropic’s unilateral commitment to adopting embedded evaluators should be supported and copied. How the industry responds will shape the trajectory of AI safety and the public’s trust in the technology. Learning and incorporating lessons from banking supervision can help.