What to Build First When Everything Is Broken

The brief usually arrives as a mood rather than a problem statement. The numbers in the board pack do not agree with the numbers in the operations report. A pipeline fails on Sunday and nobody notices until Tuesday. Finance is asking why the platform bill moves without the roadmap moving. Someone has asked for an AI assistant over documents that nobody trusts. And you own all of it, plus the budget that has to cover it.

The instinct in that position is to buy a clean sheet: new platform, new modelling layer, new orchestration, migrate everything, come out the other side with a coherent story. That instinct is not stupid — the current estate genuinely is incoherent — but it converts a set of concrete, separable failures into one long undifferentiated programme with a single payoff date. If the programme slips, nothing gets better in the meantime. And the political capital that funded it is spent before the first defensible number appears.

Sequencing is the actual deliverable here. Not a target architecture.

What does the complaint actually decompose into?

“Everything is broken” is a summary of unrelated failures that happen to share an owner. Pull them apart before you price anything, because they have different fixes, different costs, and different consequences for delay.

The useful property of this split is that only some of these failures compound. Disputed meaning compounds: every new report built on an ambiguous definition adds to the reconciliation debt, and every reconciliation meeting teaches the business that the platform cannot be trusted. Unsafe change compounds: each unprotected dependency raises the cost of the next edit, which suppresses editing, which grows a backlog. Freshness and access friction are painful but broadly static — they cost you the same amount each week until you fix them. Cost sits in between; it compounds only if workloads are being added faster than they are being understood.

Fix the compounding failures first. That is the whole heuristic.

Which failure do you fix first?

The ordering is deliberate. A semantic layer centralises metric definitions so separate reports agree, which is the shortest available path from “the data is broken” to “these seven numbers are now the numbers”. It is narrow enough to finish, visible enough to be believed, and it produces an artefact the rest of the sequence can lean on: once definitions live in one place, tests have something to assert against and freshness work has a clear target.

Resist the urge to make the first slice representative. It should be load-bearing, not representative. Pick the metrics that appear in the meeting you dread.

models:
- name: fct_revenue_daily
  meta:
    owner: finance-analytics
    definition_source: metrics/revenue.yml
  columns:
    - name: recognised_revenue_aud
      tests:
        - not_null
        - accepted_range:
            min: 0
  tests:
    - reconciles_to:
        source: gl_revenue_summary
        grain: [date, entity]

Why does the rebuild look cheaper than it is?

Because the estimate covers the migration and not the parallel run. A rebuild does not replace the old platform on a cutover date; it runs beside it while trust transfers, and trust transfers report by report, with reconciliation at each step. During that period you are funding two platforms, two sets of on-call, and the reconciliation labour between them — and the reconciliation labour is the part nobody budgets, because it looks like a rounding error until the definitions turn out to disagree.

There is a legitimate case for the rebuild: when the constraint is the substrate itself, not the modelling on top of it. If the current engine cannot express incremental processing, or cannot enforce row-level security, or cannot be observed at all, then remediation on top of it is throwing good money after bad. Test that claim hard, though. “Cannot” is a much stronger word than “is unpleasant to”.

Build or buy for the first slice?

Build-versus-buy turns on whether the capability is a differentiator or a commodity. The trap is that almost everything in a remediation programme is a commodity, and remediation programmes are staffed by people who enjoy building.

Ingestion from well-known SaaS sources is a commodity. Orchestration is a commodity. Alerting is a commodity. Catalogue and lineage are commodities. None of these will ever be the reason your organisation wins anything, and each one you build becomes a component you now maintain with a team that has no slack. Buy them, accept the seams, and spend your build capacity on the layer where your business logic actually lives — the transformations and metric definitions that encode how your organisation counts things. That layer is genuinely yours: nobody sells your revenue recognition rules, your customer hierarchy, or your definition of an active account.

The honest exception is when a bought tool would sit in the middle of the compounding failure. If a managed transformation product cannot express the tests you need to stop unsafe change, buying it does not remove work, it relocates it somewhere you cannot instrument. Tools that sit on the critical path of trust get evaluated on observability first and features second.

What team shape does the first slice need?

Not the shape of the end state. The first slice needs someone who can decide what a metric means and make the decision stick, and someone who can build the model and the tests. Those are two different people and only one of them needs to be full-time.

The decision-making role is the one that gets under-resourced, because it looks like governance and governance sounds deferrable. It is not: without a person who can adjudicate between finance’s definition and operations’ definition, the engineering work produces a second competing model rather than a single authoritative one. This is the case where a fractional Head of Data & AI genuinely fits — the work is decisions, escalation and sequencing, which is high-leverage and low-hours, and it does not require a permanent hire to be defensible.

What you should not do is hire to the target architecture. Hiring a platform team for a platform that does not exist yet produces engineers who build the platform to stay busy, and you will get a beautifully engineered substrate underneath the same disputed numbers.

What should you deliberately defer?

Self-service enablement, until there is something worth serving. A catalogue, until there are assets stable enough to describe. A governance forum, until there are decisions queued for it. And the AI layer, until the documents or tables it retrieves from are ones you would defend in a meeting — retrieval-augmented generation grounds a model’s output in supplied documents, which means it faithfully propagates whatever is wrong with them.

Deferral is not the same as denial. Write the deferred items down with the condition that releases them: catalogue starts when the first slice is in production and a second consumer asks where a metric comes from. A deferral with a trigger survives budget season. A deferral without one gets re-litigated every fortnight.

How do you tell the sequence is working?

Not by counting pipelines, datasets, dashboards or models. Those numbers go up whether or not anything improved.

The signals worth watching are behavioural and mostly negative — things that stop happening. Reconciliation meetings that fall off the calendar because there is nothing left to reconcile. Change requests that ship without a preceding “can I check this won’t break anything” thread. Failures reported by a test rather than by a user. A finance query about the platform bill that gets answered from a tag rather than from an investigation. When those stop being events, the compounding failure you targeted has actually stopped compounding, and you have earned the right to pick the next slice.

Then pick it the same way: find what is still compounding, fix that, and leave the clean sheet alone until someone can name the capability it would unlock.