eudai
[email protected]Book a call
← Writing
Risk & Innovation · 3 min read

What We Learned From Agent Down

iluminr’s first Frontier AI Risk research microsimulation asked what happens when an AI capability financial services organizations depend on becomes unavailable. The main contingency was people.

This week also marked the release of iluminr’s first Frontier AI Risk microsimulation report, Agent Down. The exercise explored what happens when an AI capability that financial services organizations have come to depend on becomes unavailable.

One finding stood out to me. When participants considered how to maintain critical services during an AI disruption, the overwhelming contingency was people.

Alternative AI providers were rarely a practical solution within the timeframes required by critical service-level agreements. Organizations would need employees to step back into processes increasingly supported by automation.

A critical service depends on a single-provider AI capability that has failed. An alternative provider is rarely ready in time, so people step back into the process.
When the AI capability went down, people were the fallback.

The cumulative effect of strategic decisions

That raises an interesting question about the cumulative effect of strategic decisions. An AI investment might reduce operating costs. A workforce restructuring might improve margins. A decision to concentrate technology with one provider might simplify operations. Each could make sense independently.

Together, they may create an operating model that depends on capabilities the organization can no longer reproduce when circumstances change.

And disruption is only one dimension. A model provider might change its technology, pricing, or contractual terms. A capability that once differentiated a business might become widely available. The economics of a product or service could change without a security incident or technology failure.

These developments are connected to strategy, technology, workforce planning, procurement, and operational risk. Yet the decisions governing them are often made in different parts of the organization.

What it means for boards, adopters, and providers

That was also a theme in Vela’s AI governance workshop at Dorsey & Whitney in Dallas. Boards need to understand how management decisions interact, which assumptions underpin the business model, and what evidence exists that those assumptions will hold under different conditions.

For organizations adopting AI, it means testing the dependencies and human capabilities on which their operating models rely. For providers, it raises questions about how customers maintain continuity, manage concentration risk, and adapt when technology changes.

The longer-term challenge is developing organizations that can continue creating value as the technologies and assumptions behind their operating models evolve.

Disclosure: Eudai founder Paula Fontana serves as iluminr’s CMO and a board director.

Paula Fontana
Written byPaula Fontana
Founder & CEO, eudai

Paula works at the intersection of market opportunity, commercial strategy, and emerging technology. A three-time CMO with more than a decade of leadership in security and innovation, she has helped companies define new categories, commercialize complex technologies, and build the capabilities needed to grow. She advises founders, serves on multiple boards, has published research with Gartner on go-to-market strategy, and has been featured in The Wall Street Journal.

The Tropos

Get new writing in your inbox.

A weekly note from Eudai on the shifts shaping security, risk, and emerging technology.

Read next · Risk & InnovationWhen Efficiency Becomes a Strategic Liability →Nov 2026 · 6 min read

Want to pressure-test the dependencies behind your AI investments? Let’s talk.

Start a conversation →