Regulated industries will adopt AI testing first, not last

The standard forecast is that AI testing lands first in startups, then in big tech, and finally — after many committees — in insurance, healthcare, and the other rooms with rules. I think that forecast is backwards.

Regulated industries will put AI-in-the-loop testing into production before most consumer companies do, because they already have the machinery consumer companies keep skipping: evidence, signatures, and a priced cost of being wrong.

This is not a compliment. Those rooms can be slow for bad reasons too. It is a structural claim.

They already pay for being wrong

In a consumer app, a bad release is a bad week. In a policy admin system, a bad release is a letter to a customer, a reserve, a filing, or a claim that pays the wrong person. In a clinical-adjacent workflow, it is a skipped check. In a published ebook, it is a file a reader will keep.

When the cost is priced, “the model wrote 4,000 tests” is not a persuasive sentence. “Here is the oracle, here is the sample, here is who signed” is. That second sentence is already how those orgs ship, with or without models. Adding a model into a system that knows how to ask for evidence is easier than teaching a move-fast org to want evidence in the first place.

I have spent most of two decades in the first kind of room. The second kind of room talks about quality as a tax. The first kind talks about quality as the thing you will be asked to produce at an inconvenient time. Models do not change that. They make the inconvenient time more likely.

Audit trails are a feature, not a brake

People treat 21 CFR Part 11, SOX, GxP, state insurance filings, as reasons not to use a model. Sometimes they are. A model that silently edits a validated suite is a finding.

The same rules are also a design spec for how to use a model without being reckless.

You already version procedures. So version prompts and graders. You already require a signature above a threshold. So require a signature on agent-proposed changes. You already keep who-changed-what. So log model id, prompt hash, and the human who accepted the output.

Consumer orgs have to invent this. Regulated orgs have to map it. Mapping is faster than inventing, once someone in quality is willing to say the model is staff.

The pilots that stall in these orgs stall because someone tried to skip the map and buy a tool that looks like magic. Magic is what auditors are paid to disbelieve. A worker with a procedure is something they already know how to inspect.

The data is worse, and that is clarifying

Regulated data is a mess on purpose: access controls, masking, environments that do not look like production because they are not allowed to. This makes naive generation fail immediately, which is a gift.

In a consumer company, the demo can pretend. In a hospital system or a carrier, the demo dies on the first attempt to see a real claim file. You are forced into the right design: synthetic goldens, narrow jobs, oracles that do not require forbidden context, humans on the rest.

I would rather work in the place where the bad design fails fast.

What “first” means, operationally

It does not mean the most models, or the flashiest copilot. It means the first durable production pattern: an agent with a contract, an eval that is itself tested, a signature, and a measured reduction in work that used to be done by people who are now doing oracle work instead.

Publishing is my current instance of this, even though publishing is not “regulated” in the FDA sense. It is public, which is a cousin. Healthcare and insurance are the more obvious instances. I expect the pattern to be boring and documented before it is exciting and on stage.

If you are waiting for Silicon Valley to finish the playbook so you can copy it into a carrier: you may be waiting on a place that does not have your constraints, and so will not write your playbook. Write yours. You already know the shape of the evidence. The model is the new worker. It still needs a manager.