Four times the specification asked for a model the data could not support

A specification is written before anyone has looked at the data. That is not a criticism of specifications — it is what they are for. But it means a specification routinely asks for a model that the data it will run on cannot support, and the moment where you discover that is the moment the engineering actually starts.

There are three things you can do at that moment. You can build what was asked and let the numbers come out meaningless. You can refuse and escalate. Or you can build the version the data does support, and measure the gap between that and what was asked.

The third one is the only one that produces a number. Here are four cases from a corporate treasury system, each with the measurement that made the decision defensible rather than merely opinionated.

W-10: the label was the thing the model was replacing

The specification asked for a gradient-boosted classifier that scores counterparty credit risk.

There was a label available. Every counterparty carried a risk level, set manually by an analyst. It was populated, it was categorical, it was right there in the table.

It was also the exact field the agent existed to replace. A model fitted on it would not measure credit risk; it would measure how well a gradient-boosted tree reproduces an analyst’s guess. It would score beautifully — analysts are consistent, and consistency is what supervised learning is good at — and the better it scored, the more precisely it would have learned to make the same mistakes.

That is a circular label, and the thing to understand about circular labels is that they pass every offline test you would normally run. Cross-validation is clean. The AUC is high. Feature importances look sensible. Nothing in a standard model-evaluation pipeline flags it, because nothing in a standard pipeline knows what the label means.

So the label was changed to something with an outside referent: onset of an adverse credit event within a forecast horizon, read from the observation panel. Harder, rarer, noisier, and actually about the world.

The measurement that makes this more than an assertion is what happened to the thing it replaced. The additive scoring prior — the plausible weighted-sum formula of the kind that sits in a lot of production risk systems — was kept as a cold-start fallback for counterparties with too little history. And it was scored on the out-of-time test block, where it ranks adverse events at AUC 0.47.

Below chance. Not weak: worse than a coin.

It ships, because a new counterparty needs a band at all, and it is labelled cold_start everywhere it appears. But the number is published alongside it, because a fallback whose measured performance is unknown is indistinguishable from a fallback that works.

W-12: two of the three required inputs did not exist

The specification asked for an ARIMA-plus-ML ensemble over spot rates, forward curves and implied volatility surfaces.

There is no forward market in this system and no options market to imply a surface from. That leaves spot.

And on spot, the thing worth knowing is that short-horizon FX point forecasting is close to unbeatable by a random walk. Meese and Rogoff established that in 1983 and it has survived forty years of people trying to overturn it. In this particular deployment it is sharper still, because the rate feed is geometric Brownian motion — in logs it is a random walk with drift. No model can extract a signal that was never put in.

Fitting an ARIMA there would have produced coefficients, a plausible chart, and no information. The coefficients would have been real numbers computed correctly from real data, and they would have meant nothing.

So the forecaster estimates the two things that are actually estimable — drift and volatility — and every run is scored against the random walk. It ties it: RMSE ratio ≈ 1.00, which is the expected result on a near-martingale. That number is reported in the same breath as the forecast, on every run, not in a footnote from a bake-off six months ago.

The useful output turns out to be the interval rather than the point. Downstream, the netting stage does not need a direction — eight conversions cost more than one whichever way the rate goes — and the timing stage needs the spread of outcomes, because how much worse things could get by waiting is what makes waiting a decision instead of a coin toss. So interval calibration is measured too: the share of realised moves landing inside the 80% and 95% bands. A band that covers 60% of outcomes while claiming 95% is worse than no band at all.

There is a second measurement in this one, and it is the more embarrassing and more useful of the two. A first cut estimated drift and volatility as annual rates. The simulation compresses time — the feed advances the market by a scale factor for every real second — so annualising against the wall clock produced a euro drift of 70,374% a year and a timing “improvement” of $2.87M on an $8.6M book.

Both figures were arithmetically perfect. Every step was correct. They were faithful readings of a clock that does not mean what the units claimed, and the only thing that caught them was that 70,374% is absurd on its face. A 7% figure produced the same way would have shipped.

W-09: the score has to survive a regulator asking why

The specification asked for an entity-matching model — a bi-encoder, fine-tuned on sanctions data.

Fine-tuning needs labelled pairs: names that are the same party, and names that are not. This system has none. A bi-encoder trained on nothing would embed names according to whatever its base model happened to encode, and its similarity scores would be uninterpretable at exactly the point where interpretation matters — the threshold that decides whether a payment is blocked.

But the data problem is not the main reason, and this is the case where the specification was wrong about something deeper than availability.

Sanctions screening in practice — in the vendor products, in the deployed systems — runs on deterministic string comparison, phonetic keys and rules. Not because nobody has tried embeddings. Because a compliance function has to be able to tell a regulator why a name matched, and “cosine 0.94” is not an answer. It is not a worse answer than a decomposition; it is not an answer at all. The requirement is not accuracy, it is accountability, and those select different model classes.

So every score decomposes into named components: which comparison method carried it, and which alias it matched through. Three specific problems shape it. Word order, because “Demirci, Volkan” and “Volkan Demirci” are one person and edit distance says otherwise, so comparison is token-set based. Transliteration, because Muhammad / Mohammed / Mohamed / Muhammed are one name under four conventions, so a phonetic pass runs alongside the literal one. And legal-form noise, because “Ltd”, “GmbH”, “OJSC” and “FZE” carry no identifying information — stripping them stops one company from scoring as two, and stops two unrelated “… Limited” companies from scoring as similar because they share a suffix.

Scores are capped rather than summed. A match score is the strongest single piece of evidence, not an accumulation — combining a weak literal match with a weak phonetic match into a strong total lets two independent near-misses manufacture a hit.

The general point: in a regulated domain, a model that cannot explain itself cannot be deployed, whatever it scores. That is a constraint on the hypothesis class, and it belongs in the specification rather than in the postmortem.

W-06: the formula was dimensionally wrong

The specification asked for IRB-style PD × LGD × EAD to set counterparty exposure limits.

That expression is an expected loss. A limit is an exposure cap. They are different quantities in different units, and using one as the other is not a modelling choice — it is a dimensional error that happens to produce a number.

What a credit line actually answers is the inverse question: given the expected loss the treasury is willing to carry on one name, how much exposure does that buy?

EL_budget = PD × LGD × EAD     →     EAD_max = EL_budget / (PD × LGD)

The recommendation is then the smaller of that and what the business flow actually needs, and which one bound is recorded — because the remedies differ. A business-need limit is a commercial fact. A credit-capacity limit is a risk decision someone may want to argue with, and they cannot argue with it if the system does not say which one it was.

The second correction is about a borrowed constant. Every counterparty here is a supplier, not a borrower. If one fails, the treasury does not lose what it owes them — it loses the value in flight and pays to re-source at short notice. So LGD is replacement loss, which sits far below the 45% a senior unsecured lender assumes. Importing the lender’s figure would have halved every limit on the book while looking rigorous, and rigour is exactly how it would have looked: a named industry-standard parameter, correctly applied, to the wrong kind of exposure.

One structural rule falls out of this and is worth stating separately. Nothing that sizes a limit may depend on the limit. Utilisation is exposure ÷ limit, so any input derived from it closes a loop — lower the limit, utilisation rises, the name looks riskier, lower the limit again. The inputs are therefore credit rating, CDS-implied default probability, sovereign and sector risk, settled dispute history and observed payment volume. None of them move when the limit moves.

What the four have in common

None of these is a case of the specification being careless. Each one asks for something reasonable, and three of the four ask for the industry-standard approach to the problem. PD × LGD × EAD is in every credit textbook. A fine-tuned bi-encoder is the modern answer to entity matching. Gradient boosting on the labelled field is what you would do. The specifications were wrong given this data and this deployment, which is the only way specifications are ever wrong.

What made each refusal defensible was not the argument. It was the number attached to it:

• AUC 0.47 — the replaced prior is worse than chance, measured on held-out time
• RMSE ratio ≈ 1.00 — the forecast ties a random walk, reported every run
• A decomposition — every match score names the method and alias that carried it
• An inverted identity — with the binding constraint recorded per decision

A refusal without a measurement is an opinion, and opinions do not survive the first person who wants the feature. A refusal with a measurement is a finding, and it survives because anyone can check it and nobody wants to be the person who shipped the below-chance model after being shown that it was below chance.

There is a cost to this and it is worth being honest about. It makes demonstrations worse. “Our gradient-boosted model achieves 0.91 AUC” is a better sentence than “our model predicts a real outcome at a modest AUC, and here is the below-chance baseline we removed.” The first sentence wins meetings. The second one is the one that is still true in production, and the gap between them is most of what goes wrong when machine learning meets a regulated process.

The standard I would want held to any model in a system that moves money: state what the data can support, build that, and measure the distance to what was asked. All three parts. The measuring is the part that converts a judgement call into something a reviewer can disagree with on evidence, which is the only kind of disagreement worth having.