Artificial intelligence is advancing so quickly that even the companies building it are beginning to question whether society’s systems of oversight can keep pace with the technology’s capabilities.
That is the message underlying a recent essay from Anthropic, one of the world’s leading AI developers. The company reports that its Claude AI system now writes the majority of the code added to its software platform and increasingly performs work once reserved for highly trained researchers. Human experts remain responsible for setting direction and priorities, but AI is accelerating the volume of work each person can oversee.
The implications extend far beyond software development.
Much of the public conversation surrounding AI focuses on jobs. Will AI replace workers? How many jobs will disappear? Which professions are most vulnerable? The honest answer is that no one knows. History suggests that technological revolutions eliminate some roles, transform others, and create entirely new forms of work. What we do know is that AI capabilities are advancing faster than many organizations can adapt.
That acceleration raises a more immediate question.
As technology becomes more capable, how do we ensure that the systems governing it evolve just as quickly?
Anthropic’s answer is not primarily technological. It is institutional. The company argues that society may eventually need mechanisms to temporarily slow development if AI capabilities begin to outpace our ability to understand and control them. Whether one agrees with that conclusion is less important than the underlying concern: technological capability alone is not enough. Oversight, transparency, accountability, and verification matter as well.
Healthcare should pay close attention.
The Nuclear Lesson
Anthropic compares the challenge of managing advanced AI to the international agreements developed to prevent the spread of nuclear weapons. The comparison is not perfect, but it highlights an important principle.
The Nuclear Non-Proliferation Treaty did not stop nations from using nuclear technology. Nor did it halt scientific progress. Instead, it established expectations, transparency mechanisms, monitoring processes, and verification systems designed to reduce risk while preserving the benefits of the technology.
Trust did not emerge because nations made promises.
Trust emerged because mechanisms existed to verify compliance.
Verification transformed trust from an aspiration into an operational reality.
Healthcare faces a similar challenge with artificial intelligence today.
Across the country, hospitals, physician practices, insurers, and technology vendors are deploying AI systems to support clinical documentation, operational efficiency, population health management, and decision support. Innovation is occurring rapidly. New products arrive almost weekly. Existing products evolve continuously.
The challenge is not a lack of innovation.
The challenge is determining how we know these systems remain safe, effective, fair, and trustworthy as they evolve.
Governance Is Not Verification
Healthcare has already begun building governance frameworks for AI.
The Joint Commission and the Coalition for Health AI recently released guidance on the responsible use of AI in healthcare. The framework addresses governance, privacy, security, monitoring, bias assessment, and workforce education. These are important and necessary steps.
But governance and verification are not the same thing. Governance establishes expectations. Verification determines whether those expectations are being met.
Most current approaches rely on individual organizations to independently evaluate, monitor, and report on the performance of AI systems. Each hospital develops its own oversight processes. Each health system conducts its own evaluations. Each organization learns from its own successes and failures.
This distributed approach is understandable. It reflects how healthcare has traditionally adopted technology.
Yet as AI capabilities accelerate, fragmentation becomes a liability.
If one organization discovers a performance issue, how quickly does that lesson reach others?
If a model exhibits bias in one patient population, how do other organizations learn about it?
If a vendor updates a model and performance changes unexpectedly, who identifies the problem and who communicates it across the healthcare system?
Today, the answers are often unclear.
The result is duplication of effort, delayed learning, and missed opportunities to improve safety and outcomes.
Every organization becomes responsible for solving the same problems independently.
That approach may have been adequate when technology evolved slowly. It is increasingly difficult to justify when AI systems can change in weeks rather than years.
The Missing Infrastructure
The most important insight from Anthropic’s essay may have nothing to do with AI development itself.
It is the recognition that powerful technologies require verification infrastructure.
Verification infrastructure is different from regulation.
It is different from governance committees.
It is different from policy documents.
Verification infrastructure creates shared visibility into performance, risk, and outcomes. It enables organizations to learn from one another. It transforms isolated observations into collective knowledge.
Healthcare already understands this principle.
Aviation safety improved dramatically when the industry moved beyond isolated incident investigations and embraced systematic reporting and learning. Drug safety advanced through pharmacovigilance programs that monitor performance after products enter the market. Patient safety efforts rely on reporting systems that identify patterns invisible to individual organizations.
Healthcare AI requires similar infrastructure.
As AI becomes more deeply integrated into care delivery, organizations need more than local governance structures. They need mechanisms that allow lessons learned in one institution to benefit all institutions. They need visibility into how AI systems perform across diverse populations and settings. They need independent evaluation and continuous monitoring that extend beyond the walls of any single organization.
In short, they need infrastructure designed for verification.
A Path Forward
This need for shared verification is why I previously proposed the Healthcare AI Review and Transparency initiative, or HART.
The concept is straightforward.
Healthcare organizations should not be expected to independently evaluate every AI system, monitor every performance change, identify every safety issue, or discover every emerging pattern. The resources required are too substantial, the technology evolves too quickly, and the consequences of failure are too significant.
A centralized, transparent, public-private partnership can help bridge that gap.
Such a model would not replace local governance. It would strengthen it.
Organizations would continue to make decisions about which technologies to deploy and how best to integrate them into their workflows. Clinicians would continue to exercise professional judgment. Patients would remain at the center of care.
What changes is that organizations would no longer learn in isolation.
The same principles that support aviation safety, drug surveillance, and public health can help healthcare AI mature responsibly.
The objective is not to slow innovation.
The objective is to ensure that oversight evolves at the same pace as innovation.
The Question Ahead
Anthropic’s warning is ultimately not about artificial intelligence.
It is about responsibility.
As AI capabilities accelerate, institutions must build the transparency, accountability, and verification infrastructure necessary to maintain public trust. The companies building AI are beginning to recognize this reality. Healthcare leaders should do the same.
The question is not whether AI will become more capable.
It will.
The question is whether our systems of oversight, transparency, and accountability will evolve quickly enough to ensure that capability translates into safer, higher-value care.
In my next article, I will explore what that evolution might look like and why healthcare’s approach to AI oversight must evolve as the technology itself evolves.
Sources:
Anthropic. (2026, June 4). Recursive self-improvement. Anthropic Institute. https://www.anthropic.com/institute/recursive-self-improvement
Joint Commission & Coalition for Health AI. (2025). The responsible use of AI in healthcare (RUAIH): Guidance document. Joint Commission. https://www.jointcommission.org/en-us/knowledge-library/news/2025-09-jc-and-chai-release-initial-guidance-to-support-responsible-ai-adoption
Chaiken, B. P. (2025). AI’s 1929 moment: Building HART for healthcare safety. Future-Primed Healthcare. https://barrychaiken.com/newsletter/ai-1929-moment-building-hart-for-healthcare-safety/
Chaiken, B. P. (2025). Beyond HART: Capitalism’s safety net for AI. Future-Primed Healthcare. https://barrychaiken.com/newsletter/beyond-hart-capitalisms-safety-net-for-ai/
