Optimising AI-assisted software integration: from 206 turns to 38 milliseconds

We tested a language model as a software integrator, not as a generator of new business logic.

The fixture represents a familiar integration job. A firm has bought Java components from several vendors — market data, pricing, liquidity, risk and capital. Their algorithms already exist and their source is unavailable. The integrator must select the appropriate implementations, determine what each component requires, connect them correctly and produce a working application.

We wanted to reduce the model turns and weighted usage needed to reach a correct assembly. Our hypothesis was that the model was spending too much effort rediscovering mechanical facts the component producers already knew.

Here, weighted usage means output tokens × 50 + input tokens × 10 + cache-read input tokens × 1. The 5:1:0.1 relative weighting mirrors the pricing proportions used during the experiments, but the result is a token-volume proxy — not money, and not a portable unit across different model tokenizers.

How much probabilistic work can be removed from software integration by turning mechanically knowable facts into an explicit contract?

In the early experiments, the model had to recover those facts from classes and documentation. We then attached a small machine-readable contract to each library:

Fluxtion-Provides:    adjusted=AdjustedApi, spread=SpreadApi
Fluxtion-Requires:    MidApi, DepthApi
Fluxtion-Constructor: (MidApi, DepthApi)

The metadata does not decide what the customer wants, choose between business policies or write application logic. It records what a component provides, what it requires and how it can be constructed — facts that would otherwise be inferred again by every downstream integrator, human or model.

With an open integration brief, the model took 206 turns, consumed 14.80M in our weighted usage measure and produced the wrong application. Progressively restructuring the information reduced that to 51 turns and a correct application — a 7.5× reduction in weighted usage.

That initially looked like a successful optimisation of the LLM workflow. Then we realised that, once the metadata existed, much of the remaining selection and wiring was a deterministic dependency problem. A small resolver reproduced the same assembly in 38 milliseconds using zero model tokens for that resolution step.

Last row: one run with a different model and the pre-fix component API, at then-current prices.

"The same assembly" is worth stating exactly, because it is stronger than a matching config file. The resolver selected the same components, emitted byte-identical XML, built green, and the resulting application produced an audit log matching the reference figure by figure — 12 scored events, 27 published figures, every compared value identical. The comparison runs through the analyser's own log reader, and the scorer caught all five mutations chosen for that run. That validates this particular comparison, not the scorer generally. Part 2 places the scorer's full false-pass history beside the 17/17 result that depends most visibly on it.

The lesson was not that the model had become better at wiring software. It was that wiring was the wrong work to give it. A component build can publish what is in its jar; a site can record its policy; a resolver can connect compatible components; and a compiler can derive dispatch from the resulting graph. The translation from a customer's language into a formal requirement remains judgement. The amortisation table near the end puts the reuse boundary around each step.

What "authoring" means here

Everything here concerns composing bought-in components: selecting and connecting things that already exist. The experiments target a Spring XML declaration because it is data that a resolver can emit; generating equivalent Java would be no harder. Designing a graph and writing its nodes is a different problem, unmeasured here. Part 2 owns the separate question of how Fluxtion derives execution from either declaration.

First, the optimisation that worked

Fifteen cells, one model, one problem. The only thing that varied was what the integrator was given: which documents, how the components described themselves, what scaffolding was supplied.

14.80M and wrong, to 1.98M and correct. Same model, same jars, same problem. Three findings came out of it that we still believe.

Discovery aids expire. A worked example produced the fewest builds in the entire study — then became actively harmful, costing 28 extra turns, once the components carried an index. It had been teaching the model to inspect classes for facts the manifest now answered. A document explaining how to find out X has a shelf life, and it ends when X is written down.

Guardrails are invisible to the runs that need them least. Every successful run called the "what you must not write" section unused — one wrote "I never considered writing one." The run with it removed produced a wrong answer. A successful run is, by definition, the one that did not need the guardrail. You cannot retire preventive text by asking; only by measuring.

Cost mostly followed round-trips rather than document length. That is partly mechanical: weighted usage accumulates on every turn, so its r = +0.955 correlation with turn count is not an independent discovery. The interesting observation is narrower: two cells with equal turns cost the same despite one carrying a manual 2.7× longer, so cutting words alone won little in this fixture. Prompt caching would explain that result, but we did not record cache-write or cache-read usage, and the shortest manual may have fallen below the cacheability floor. The mechanism remains a hypothesis.

Meanwhile, the pre-fix control run in plain Java

An obvious objection was that the framework might be doing nothing and this was merely a well-organised library. So we ran a control: equivalent components as ordinary Java classes, with an author writing a plain engine that calls them in dependency order. We deliberately gave that arm a substantially more capable model. This changed both architecture and model, so it is a comparison between two deployment choices, not an experiment that isolates the framework.

With the original component API, it produced a green build, passing tests and a running application — but a risk figure 88% understated. The cause was not plain Java and it was not a failure to understand the warning. The API exposed no way to recompute a derived figure without also advancing event counters. Adding one counter-free refresh() operation to the component library, then rerunning the same task, produced a perfect 17/17 score. The library changed; the framework did not. Part 2 carries that result and the scorer evidence behind it.

The correctness failure therefore provides no evidence that plain Java is generally unreliable, or that compiled execution would have prevented this particular error. What survives here is the cost measurement of the pre-fix run. Three ratios answer different questions:

When assembly becomes compilation

The manifest index shown above was the single biggest win. It made each component's provided capabilities, requirements, construction contract and consumed events available without another round of discovery.

Which prompts the question. If every component states what it needs and what it offers, and the business states which figures it wants — what exactly is the model deciding?

So we wrote a resolver: a small dependency-free prototype that reads the manifests, treats "cover these figures, satisfy every requirement" as a constraint problem, and emits the XML.

It produced the same components and the same wiring as the measured optimum. The build was green. The alerts were byte-identical to the reference — and so, figure by figure, was the audit log.

38 milliseconds. A zero-token resolution, repeatable on every rebuild and every vendor upgrade with no model cost.

Every component-composition cost figure in this series had been measuring a model doing a resolver's job. The 7.5× was real, and it was an optimisation of work that did not need doing.

What was left was judgement — and only judgement

The resolver's failures are the interesting part. We built a fixture where six pricing components have byte-identical declared surfaces — same provides, same requires, same constructor, same consumed events. They differ only in a sentence of prose:

PricingHedged     adds the spread including the desk hedging overlay
PricingNetted     adds the spread net of the standing inventory offset
PricingGross      adds the spread gross of execution fees
PricingCapped     adds the spread capped at the venue ceiling
PricingSmoothed   adds the spread with the smoothing factor applied
PricingFull       adds the spread

The resolver reports the ambiguity, lists the candidates, and stops. It does not guess.

The model, given the same fixture and a requirement saying "this desk hedges every position", chose correctly — and what it wrote about ruling out the plainest option is the right reason:

"Description says 'adds the spread' but provides no specification of hedge inclusion; unclear whether it includes hedging overlay."

Discrimination on the absence of a promise, not the presence of a keyword. It also cost eleven class inspections — the model stopped trusting prose where it had trusted the index.

And then that turned out to be memoisable too

If prose is expensive to read, declare the fact instead. One line per component, written once by the vendor who already knows the answer — and one line per installation, written once by the customer:

Fluxtion-Convention: spread=hedged     ← the vendor, once
--conventions        spread=hedged     ← the customer, once

One word decides the build. The fixture that cost eleven inspections and a paragraph of reasoning becomes a dictionary lookup.

It carries a rollout cost worth stating: a profile applied to a legacy fleet whose jars carry no conventions is unsatisfiable by construction. Vendors must label before customers can profile. That is a sequencing constraint, not a free lunch.

The pattern underneath all of it

Every step is the same move at a different scale: compute something once, where the information already is, and serialise the answer so nobody downstream re-derives it. Compiler people will recognise the pattern — move work upstream, resolve what can be known ahead of time, and serialise the result — though not every row is partial evaluation strictly; some is memoisation, some is declarative metadata, some is policy captured once.

On the third row: the memo is the resolved bean file, not the resolver. The resolver is the evaluator; what gets committed and rebuilt against is the artefact it emits.

The last row is worth stating carefully: a decision, once made, can be recorded in an artefact — that is exactly what a convention line is. What is not mechanically derivable is the translation from a human goal into a formal requirement.

Stated as a working rule: give the model the judgement, and give metadata the mechanics. Every fact a supplier can write down once is a fact no integrator has to infer, and every inference removed is a round-trip removed — which is the thing that actually costs money.

The judgement did not vanish. It moved — from an inference repeated on every build, to a one-line declaration written once. That is a better place for it: a line in a profile can be diffed, reviewed, and explained to an auditor. An inference buried in a model's reasoning cannot.

Things we got wrong on the way here

Published because the corrections are more instructive than the results, and a post reporting only its successes should not be trusted.

We asserted a cost without measuring it

Asked whether the framework's audit log allocated memory, we said "by design, yes" — having never checked. With event stringification disabled, an audit-enabled graph showed no detectable per-event allocation. That correction is about allocation only: compiling audit in at level NONE still raised dispatch time from 17.9 to 37.0 ns/event in the benchmark fixture.

We filed a defect that was a deliberate design decision

A profiler showed one call consuming 55% of dispatch cost and we labelled it a defect. The measurement was right: the default wall-clock read added 9.43 ns to a 7.76 ns baseline in the benchmark fixture. The classification was wrong. The clock deliberately gives every node a timestamp, and the remedy is configuration rather than a code fix: inject stream time for deterministic replay, where the clock abstraction itself measured 0.28 ns/event.

The scorer once agreed with an empty comparison

One comparison reported "12 of 12 events identical" while comparing twelve events against zero — an empty list yields no mismatches, and no mismatches read as agreement. It could have manufactured a perfect-looking result from no actual events.

We rebuilt it on a tested parser. The resolver comparison reported here then survived five chosen mutations, but that does not validate the instrument generally. Part 2 carries the ten-guard history, including five false-pass classes found by independent reviewers. The specific result survives; the scorer is still earning trust.

Where this goes next

The result draws a useful boundary, but it also leaves important work unmeasured.

Three things follow, and only one of them is comfortable.

The unmeasured half. No authoring instruction has ever been ablated here; every ablation was over assembly guidance. The ceiling of what a cheap model can author correctly is unfound, and we expect that experiment to be less flattering.

The step upstream of everything. Converting "a risk engine that alerts on breach" into the list of figures the business needs was never measured — every experiment was handed that list. It is the first place a model genuinely earns its keep and a blank spot in our data.

The uncomfortable one. Almost none of the catalogue idea is specific to this framework, and it is not new: OSGi has long used manifest-declared capabilities and requirements with a resolver. Ours is a thinner version of an idea that ecosystem worked out properly. The contribution is not the mechanism but a measurement of the trade-off against a language model, in this fixture — what a resolver is worth when the alternative is priced per token.

Resolution ends with a component graph; compilation starts from that graph and derives execution. Part 2 owns that distinction — and explains why neither mechanism would have repaired an API that offered no correct operation.

What this evidence does not support

• It is one headline fixture: five jars, nine entry points, one problem domain. A later ambiguity extension expanded the catalogue to fourteen entry points. The ablation cells are n = 1; only the optimum was replicated.

• The comparison arm is a single run, and it has since passed a task it previously failed once its library was changed. The ~13× should be read as one measurement, not a settled multiple.

• Cross-model cost comparison is genuinely messy. The Opus 5 tokenizer used in the control produced roughly 30% more tokens for identical text than the Haiku 4.5 tokenizer used in the structured arm, so a token is not a portable unit across these models. We quoted one of three defensible numbers for months without saying which, and it is easy to apply that inflation twice by accident.

• The resolver has no test suite. Its evidence is that its output builds and produces byte-identical results on one fixture, which is not the same thing.

• Nothing here measures free-form authoring at all. The strongest claim in this post is scoped to composition, and stops there.

• The resolver prototype has known gaps. Independent review found it permitted only one selected entry point per jar, silently chose between duplicate providers, accepted constructor cycles, and ignored declared event consumption. A later review found cycle validation sitting too late — the CLI refused a cycle while the machine-readable path still declared it resolved — and identifier collisions between distinct jar names. Those paths are now repaired and the fixture still reproduces byte-identically. Declared event consumption is not fixed: it remains parsed but unused, documented as outside the solve rather than repaired. One successful fixture does not establish a general resolver.

• The caching mechanism is uninstrumented. Equal-turn cells costing the same despite very different document lengths is consistent with prompt caching, but cache-write and cache-read usage were never recorded, so the mechanism remains a hypothesis.

The answer, in this fixture

The metadata improved the LLM integration process before it replaced any of it. Better-structured information reduced the run from 206 turns and 14.80M weighted usage with a wrong result to 51 turns and 1.98M weighted usage with a correct result — a 7.5× reduction in weighted usage.

The plain-Java control establishes something different. Its pre-fix failure came from the component API: both available actions were wrong. Adding one counter-free recompute method allowed the same task to score 17/17. That result does not show that vanilla Java was unreliable or that a compiler would have prevented the failure; it shows that API shape determines whether a correct behaviour is available to an integrator at all.

The instruction-ablation cells are also single runs, so they do not establish a success probability or the minimum model tier required. Their defensible result is narrower: explicit guidance removed discovery and inference from the task, and the observed structured run reached a correct result with fewer turns and much lower weighted usage. It is reasonable to hypothesise that reducing that burden improves reliability and permits less-capable models, but quantifying either claim requires repeated cross-model trials.

That creates an architectural and economic choice about where the guidance should live. It can be repeated as application-specific instructions for each integration, or its stable component facts can be expressed once in a pre-generated contract. This experiment does not establish one universal balance, but it shows why the placement matters: it determines how much model and conventional compute is consumed, how often the same work is repeated, and who pays for it. When the component vendor publishes and validates the stable information once, that production cost can be amortised across many customers, applications and rebuilds instead of being paid again by every integrator.

Once the same mechanical facts formed an explicit component contract, the resolver replaced that remaining 51-turn selection-and-wiring step. It chose the same components, emitted byte-identical configuration and reproduced the same scored output in 38 milliseconds with zero model tokens for resolution.

That is the direct answer: for this fixture, the mechanically specified component discovery, selection and wiring could be removed from the model entirely. What remained outside the resolver was the work that was not mechanically derivable — interpreting the customer's goal, defining the metadata contract and deciding policy when several technically valid components expressed different business meanings.

The result is not a general claim that software integration no longer needs judgement. It is evidence that model cost and iteration fall when stable facts are declared once, and disappear from the stage whose remaining decisions can be expressed as constraints.

Give the model the judgement. Give metadata the mechanics.

Experiment archive: this is a working repository, not a frozen snapshot of the evidence at publication. Files and results may move or change as the experiments and independent review continue. It holds the principal fixtures, predictions committed before each run, generated sources, notes recording what each round got wrong, and the resolver's committed output — which one reviewer reproduced byte-identically after deriving the inputs independently. It is not yet a one-command reproduction package: the composition round lacks its executable jar workspace, and the runtime benchmark lacks a run script, pinned dependency provenance and raw output for several published tables. That is outstanding work, not a claim.