The Infrastructure of Trust
Why AI needs the same supporting systems that make risk and security work.
Trust in a new system comes down to a few practical things: clear authority, understood dependencies, limits on what it can do, and a way to learn when something goes wrong.
Those are familiar ideas in risk and security. They are becoming central to AI adoption too.
Last week at GFMI Chicago, much of the conversation came back to the connections between teams, systems, suppliers, and decisions. The point was not that organizations need more plans. It was that they need to understand how work actually moves when the plan is under pressure. A playbook that does not show up in the operation has limited value.
That perspective makes this week’s developments easier to read.
What happened this week
OpenAI alerted more than 100 organizations about unauthorized activity linked to rogue AI agents. The details will continue to emerge, but the immediate lesson is clear. Once an agent can access data, connect to systems, or take action through other tools, its risk profile extends into every part of the business it can reach.
Google said monthly vulnerability disclosures have more than doubled. Its researchers reported an average of 10,740 disclosures a month between January and August, up from 5,045 in 2024. The concern is not only the volume. It is the faster weaponization of known, high-risk weaknesses. For internal teams, it is also a prioritization problem: when the queue doubles, deciding what to fix first matters as much as fixing it.
Microsoft warned that a China-linked group was exploiting a vulnerability in N-able security software. A trusted security product could become a ransomware route across a chain of customers. It is a sharp reminder that the tools used to manage risk are part of the risk environment too.
The Bank of England said AI developments could increase cyber and operational risk. Its Financial Policy Committee put that concern alongside risks from high debt levels and stretched asset prices. The message for financial institutions is that AI-related cyber risk is now part of the broader stability conversation.
The EU is reconsidering the timetable for removing high-risk telecom suppliers. The security rationale is clear. So are the costs and practical challenges of replacing critical infrastructure. Operators have put the potential replacement bill at up to €40 billion.
Insurers claim AI is already increasing healthcare costs. A Blue Cross Blue Shield Association analysis attributed an additional $942 million in spending over two years to hospitals’ use of AI tools when submitting claims. It found a sharp rise in patients documented with complex conditions, without a corresponding change in the care delivered. With hospitals and insurers both using AI, the harder question is whose system can show what it did and why.
None of these is solely an AI story. They are stories about access, concentration, supply chains, and the ability to keep operating when a dependency fails.
The model is one part of the system
In his essay, A frontier without an ecosystem is not stable, Satya Nadella argues that companies need to retain the learning that comes from using AI. The durable asset is not simply access to a model. It is the organizational knowledge that develops as people apply it, question it, improve it, and build it into their work.
That is a useful way to think about trust.
An AI agent changes character once it is connected to data, customer systems, internal workflows, or other tools. It has permissions, dependencies, and potential points of failure. The people responsible for product, security, operations, and risk may all see a different part of that picture. Trust depends on whether the organization can see the whole thing.
The same principle applies to a supplier decision. Replacing high-risk telecom equipment may reduce one exposure while introducing cost, implementation work, and continuity risk somewhere else. Those decisions need security, procurement, technology, finance, and operations looking at the same tradeoff.
What organizations learn is part of the control environment
“Learning loop” is a phrase that gets used a lot in AI. In practice, it is straightforward.
An AI feature behaves unexpectedly. A security incident reveals a dependency that was not well understood. A simulation shows that people are unclear about who can make a decision. The important question is what changes afterward.
A closed ticket or a circulated report may document the issue. The stronger response is a change that shows up in the business: a clearer approval path, a revised control, a supplier decision, an investment, or a different scenario to test next time.
That was one of the most useful themes at GFMI. An exercise should lead somewhere. It should change a decision about remediation, risk acceptance, investment, or preparedness. Otherwise, it is difficult to know whether the organization is becoming more capable or simply becoming more familiar with its own paperwork.
Trust is increasingly part of the product
For product teams, this means getting specific about how AI-enabled products work. Customers will want to understand what a system can access, how activity is monitored, what its boundaries are, and what happens when it produces an unexpected result.
For growth teams, those answers are increasingly part of the buying process. Buyers are looking past broad claims about responsible AI and enterprise security. They want to know whether the company has thought through the real operating conditions around the technology.
For leaders, the work is making tradeoffs visible before they become urgent. An AI deployment, a supplier change, or a security investment each involves choices about what the organization is willing to accept and what it needs to protect.
That is the infrastructure of trust: the systems and relationships that hold when the model, the market, or the threat landscape moves faster than expected.
On Eudai this week
We published three pieces related to these developments.
Gut feel at scale looks at how AI can make the Highest Paid Person’s Opinion harder to spot, and how to use it to test a decision before committing to it.
Why is everything an index now? notices a September cluster of research programs with the same name, and asks what happens to market language when everyone draws from the same models.
The dots are not harmless argues that AI’s friendly visual language shapes how much authority people give a system, which puts the interface inside the governance model.
One question for next week
What was the last exercise, incident or surprise that actually changed a decision in your organization?
Part of our work on security, risk, and innovation marketing.