AI Inversion: Make the Model Ask the Questions

Most teams use AI the same way. You describe what you want. The model writes it. You review the result and ship it.

Call this the authoring pattern. The model writes and you edit. It works for a first draft, an email, a script. It fails on one kind of document: a document whose value comes from the decisions inside it, not from the prose around them.

I spent the last few months rebuilding one of those documents from scratch. The rebuild taught me a pattern I now call AI inversion.

AI inversion flips four roles.

You decide. The model never writes the content of a decision.
The model asks. It gathers the evidence, then interviews you one question at a time.
A program checks the model. Every claim the model makes quotes its source, and a deterministic checker refuses anything it fails to find.
The output becomes a spec. Your decisions land in machine-readable files that coding agents can build from directly.

The document at the end is a render of the spec. Nobody types it.

The setting

I work on carrier integrations in shipping. A shipping platform talks to dozens of carriers. Each carrier has its own API, label formats, errors and refund rules. Over years, each integration piles up small behaviours in code.

Here is a typical one, invented for this post. When the platform asks a carrier for a rate, it sends the shipment's internal ID as the package ID, cut to 20 characters. The carrier allows 35 on labels. Nobody recorded why the cut happens at 20.

Multiply this by 50 to 200 rules per carrier. When you move an integration to a new platform or a new carrier API, someone has to re-decide every rule. Keep it, change it, or drop it. Each answer reaches merchants. A wrong one means a failed label, a refund reported as complete when the carrier never heard about it, or a tracking event nobody maps.

The requirements document for one of these moves drives the whole project. Engineering, the test plan and the rollout all build on it. A mistake here carries into every later stage.

The authoring version

Our first AI-assisted process followed the authoring pattern. An agent read some evidence and wrote the requirements document. A product manager reviewed it.

Speed was never the problem. The documents arrived quickly and every gate landed on time. The cost arrived after the gate.

In an internal audit, every past project rewrote its specs after sign-off.
An older draft replaced an approved document, and nothing warned anyone.
Another reached sign-off on production figures that were months old.
Across a set of signed documents, only a minority of rules recorded a reason. For the rest, an engineer had no way to tell a decision from an accident.

None of these failures came from bad writing. The writing was fluent, and the fluency caused the problem. A well-written paragraph hides a missing decision. A reviewer reads the prose, finds nothing wrong with it, and approves. The model fills each gap with plausible text, and nothing points the reviewer at the gap.

The process also depended on the people who built it. They knew which paragraphs to distrust. A product manager new to the company had no way to know.

There was a second, quieter cost. The document was prose. Every engineer and every coding agent downstream had to interpret it, and each interpretation drifted a little from what the product manager meant.

The inverted version

We rebuilt the stage around one sentence: you decide the spec, you never write the document.

Inversion one: the person decides, the record renders

Every project now holds a decision record. One file says what the integration does today, as scenarios and rules, with no decision attached. A second holds every ruling and the reason behind it. The requirements document renders from both.

To change what the document says, you change a ruling and render again. The renderer stamps each file with a hash of its own text, so a hand edit shows up at the next check. If anything you signed changes, your signature reverts and you sign again. The overwritten-draft failure no longer happens in silence.

Inversion two: the model asks, you answer

The agent reads the code, production data, ticket history, the carrier's API and earlier documents. It compiles today's behaviour into rules. Then it interviews you.

Every question arrives as a card: a title, a statement of what happens today, a few facts, one question, and the answers the project type allows. Here is the package ID card:

Today the shipment's ID, cut to 20 characters, goes to the carrier as the package ID.

Nothing on record says why.
On labels, the carrier allows a package ID of up to 35 characters.
The carrier's rating example states no limit.

Should the new integration cut the ID to 20 characters?

Keep the 20-character cut. Merchants see no change.
Change the package ID. Needs your reason.
Stop sending a package ID. Needs your reason.
Tell me more.

The card carries no recommendation and no default. After you answer, the agent reads your answer back in one plain sentence about what it means for merchants. You confirm, and only then does the ruling go on record, with the read-back saved beside it.

"Tell me more" records nothing. The agent researches the question from the evidence and explains it. We built the process for a product manager on their first project, someone who knows how APIs work but not this company's code. The model teaches the domain while the person decides. In the authoring pattern, the person teaches the model through prompts. Inversion turns this around too.

Inversion three: a program checks the model

In the authoring pattern, a human checks the model's output. Humans skim. So we gave the checking job to a program.

Every claim on a card quotes its evidence, exactly as the source says it. The checker opens the file and looks for the quote. It refuses a claim with no quote, a quote the file does not hold, a number with no source, an internal ID or file path, and any wording which nudges toward one answer. Where the record is silent, the card says so in those words: "Nothing on record says why."

Production numbers come from recorded queries. A query runs against the warehouse, and the run is recorded with the day, the query and who ran it. A number that no recorded query or evidence file holds is refused.

The model still does most of the work. It reads thousands of lines of code, writes every card and drafts every read-back. It does the work under audit.

Inversion four: the product becomes a spec

The rendered document is the least important output. The real output is the decision record behind it, and the record is YAML.

A scenarios file holds every behaviour the integration has today. In simplified form, each scenario has an ID, a trigger, the evidence behind it, testable rules and acceptance criteria:

scenarios:
- id: SC-07
  title: Package ID on rate requests
  trigger: Every rate request
  evidence:
  - kind: code
    ref: rate request builder
  rules:
  - id: BR-07-01
    statement: The shipment ID, cut to 20 characters, goes to the carrier as the package ID.
  acceptance:
  - Given a shipment ID of 40 characters, the rate request carries its first 20.

A decisions file holds every ruling: keep, change, add or remove. Each one carries the reason in the product manager's words, who worded it, and the history of any earlier answer:

rules:
- rule: BR-07-01
  ruling: keep
  reason: Merchants match rates to shipments with this ID today.
  worded_by: pm
  history: []

A test file turns each ruling into an acceptance test a merchant would recognise. Risks and rollout steps live in their own files beside them.

This changes who the product manager writes for. A prose requirements document speaks to an engineer, who interprets it, and to a coding agent, which guesses. A YAML spec speaks to both with less left to interpret. The aim is that an agent reads the rule, the ruling and the acceptance test, builds the behaviour, and checks its work against the same test, so each line of code traces to a rule ID, and each rule ID to a decision and its evidence.

Rule IDs stay stable. When new evidence arrives and the record recompiles, every rule nothing moved keeps its ID. Rulings stay attached, and downstream code and tests keep their references.

This is where we expect the scale to come from. A prose document does not scale, because each one needs a person to translate it into tickets and code, and each translation loses something. A spec is meant to remove the translation step. Product decides in a format machines consume, and humans read a rendered view of the same source. Product management becomes a spec-driven discipline, and the spec is the contract between product and the agents which build.

Why shipping makes a good exhibit

Inversion fits some domains better than others. Carrier integrations meet every condition.

The decisions are discrete. A carrier move holds 50 to 200 rules. You count them, rule on each one, and prove none went missing. A coverage gate shows every behaviour found in the code reached a rule.

The evidence is checkable. Code, production data, carrier documentation and ticket history all live in files and tables. A quote either appears in the source or it does not. A domain built on opinion gives the checker nothing to check.

The behaviours map to code. A rule like "cut the package ID to 20 characters" names one code path and one test. A spec of 200 such rules describes the integration closely enough for an agent to build it, and for a test suite to prove the build matches the decisions.

The stakes are concrete. Each ruling changes what a merchant sees: a label, an error message, a refund, a tracking update. This gives the read-back weight. You confirm a sentence about merchants, not about code.

The situations repeat. Every carrier produces a value longer than its limit, an empty response, a refund request after the label went on a manifest. Earlier product managers answered these for other carriers. After you answer, the agent shows how the same situation plays out on other carriers and why. You keep your answer or change it, and you say what makes this carrier different. Over time the platform behaves one way everywhere, unless a carrier gives a reason to differ.

No single person knows the rules for dozens of carriers. The authoring pattern leaned on the few people who knew the most. Inversion moves their knowledge into the spec.

What changed

Every rule in a signed spec carries a ruling and a reason, in the product manager's own words.
The checks exist because of failures in early runs, and they do not catch everything yet. A review of an early generated document found false statements. Most were fixed by the next version of the process, and one was still open at our last check.
The handoff to engineering is a set of files with rule IDs, rulings, reasons and tests, not a document to interpret. The aim is for coding agents to build from those files directly. No project has been through engineering yet, so that part is still unproven.
Review comments come back through the same record. Each reviewer comment becomes a card, the answer changes a ruling, and the next version lists what changed and who asked for it.
The same record renders the full document, an executive summary, a rollout plan for the team, a version for merchants, and the acceptance tests.
Every run so far was run by the person who built the tool. The test that matters is a product manager who did not build it, running it from the written guide, and that run is next.

The tool grew from live use. Each run surfaced problems, and each problem went into a findings log with its cause and a test. The log runs to hundreds of entries. The work landed as dozens of stacked pull requests, each one a chapter built on the one below.

An illustrative run

The figures in this box are rounded and illustrative. They are drawn from our runs, not from any one project, and they show the shape of the work, not a measurement.

What inversion costs

Inversion is slower for the person. In the old process the product manager reviewed a finished document. In the new one they rule on every rule. The illustrative run above shows the scale: a working day of the product manager's time, and a model bill that stays small next to the cost of one wrong launch.

It also demands more judgement. Two hundred rulings tire anyone. We added a guided pace with one question at a time, an expert pace with up to four, and resume from any point, so nobody has to finish in one sitting.

The trade pays off when a wrong decision costs far more downstream than a careful one costs upstream. In carrier integrations, a wrong decision becomes a failed label in front of a merchant at peak season.

When to invert

Invert when:

the value of the output lies in the decisions inside it,
the decisions are discrete enough to count,
the evidence lives somewhere a program reads,
someone has to own each decision later,
agents or engineers build from the output,
a mistake carries into downstream work.

Keep the authoring pattern when fluency is the product: a first draft, a brainstorm, a summary nobody builds on.

The shift is simple to describe. Stop asking the model for answers. Ask it for questions, give it the evidence, put a checker between it and you, and keep your decisions in a format machines build from. You decide. The spec follows. The code follows the spec.