Build Versus Buy When It Isn't a Differentiator

If you own the platform and the budget, build-versus-buy is not an architecture question. It is a question about where your engineering attention compounds and where it merely accumulates. The architecture follows.

What is actually being decided?

Not whether your team is capable of building the thing. On a data platform, almost everything is buildable: ingestion, scheduling, lineage capture, data quality assertions, a catalogue, a chunking-and-retrieval layer. The decision is which of those you want to still be maintaining when the person who wrote it has moved on and the requirement has drifted.

Build-versus-buy turns on whether the capability is a differentiator or a commodity. That framing is correct and also slippery, because a capability is rarely uniformly one or the other. Ingestion from a dozen SaaS sources is commodity. Ingestion from the one internal system that encodes how your business actually books revenue is not. The useful move is to stop assessing the capability and start assessing its parts, then buy the commodity body and build the differentiating sliver.

Why does “we could build that in a sprint” mislead?

Because a sprint buys you the first release, and the first release is the cheapest thing you will ever own. The cost that matters arrives later: the credential rotation nobody documented, the source that silently changes a column type, the schema drift handler, the backfill path, the on-call rota, the person who has to be interrupted because they are the only one who understands the retry semantics.

A platform owner evaluating a build should price the second year, not the first. Ask who carries the pager. Ask what happens when that person takes long leave. Ask what the cost is of the capability being down for a day, and whether anyone outside the team would notice or care. A capability that nobody notices when it works and everybody notices when it breaks is the archetype of something to buy.

The shape above is the whole argument in one picture. Buying is cheap up to the vendor’s ceiling and then flat — you cannot spend your way past it. Building is expensive from the start and gets steeper as the long tail arrives. The question is whether your requirement lives under the vendor’s ceiling. If it does, building is a decision to pay more for the same outcome. If it does not, buying is a decision to be permanently almost-right.

Which parts of the capability are genuinely commodity?

Decompose before you decide. Take a capability you might otherwise treat as a single line item and split it into the plumbing, the semantics, and the interface.

Run the split and the answer usually resolves itself. Connectors and scheduling are commodity — the difficulty is real but it is not your difficulty. Entity resolution, grain, and history are where your business logic lives. A semantic layer centralises metric definitions so separate reports agree; that definition set is yours and should never sit inside a vendor’s proprietary format you cannot read without a licence. Row-level security filters a model’s data by who is asking, and the rules encode your org structure, which is also yours.

What does a good seam look like?

The seam is the thing you are really designing. Buy the plumbing, own the meaning, and put a contract between them that you control.

The contract test is not ceremony. It is the thing that makes the vendor replaceable. If the bought tool writes into your storage in a shape you assert against, swapping it out is a migration of one component rather than a rebuild of the platform. If the bought tool writes into its own storage and your models read from its API, you have bought the plumbing and sold the exit.

type LandedContract = {
entity: string;
loadedAt: string;
watermark: string;
columns: Record<string, "string" | "number" | "timestamp" | "boolean">;
};

const orders: LandedContract = {
entity: "orders",
loadedAt: "iso-8601",
watermark: "updated_at",
columns: {
  order_id: "string",
  customer_id: "string",
  updated_at: "timestamp",
  net_amount: "number",
},
};

// Assert before the transformation layer runs, not after.
export function assertContract(actual: LandedContract, expected: LandedContract) {
return Object.entries(expected.columns).every(
  ([name, type]) => actual.columns[name] === type,
);
}

What does buying badly look like?

Buying badly is rarely about price. It is about where the vendor puts the semantics. A tool that lands data into storage you control, in an open format, with your transformation logic downstream, is a tool you can leave. A tool that also wants to hold your metric definitions, your access rules, and your orchestration graph is a tool that will hold your roadmap.

Note the last row of that comparison. Building a commodity capability teaches your engineers a great deal about something that does not distinguish you. That is an opportunity cost paid in the only currency you cannot top up.

How should the spend be sequenced?

Buy the boring middle first and buy it early, because it is the part that unblocks everything else and the part with the clearest exit if you get it wrong. Defer the purchases that touch semantics until your models are stable enough that you know what you are asking for. Buying a catalogue before you have agreed what an entity is produces an expensive, well-indexed record of disagreement.

Build in the order of irreversibility. Grain decisions, history handling, and metric definitions are the expensive things to change later; they deserve your senior people and your time. A connector is cheap to change later, so it should not be consuming either.

What does this do to team shape?

Buying a commodity capability does not reduce headcount; it changes what the headcount does. You still need someone who understands the tool well enough to diagnose it at three in the morning, and you still need the contract tests maintained. What you get back is the ability to staff for modelling and semantics rather than for plumbing — and modelling and semantics are where a platform earns its keep.

There is a specific risk worth naming. A team that has bought most of its plumbing can lose the ability to reason about it. Keep at least one person who can read what the tool is doing at the storage layer, and make sure the runbook for the bought component is written by your team rather than pasted from vendor documentation.

What should be deferred?

Defer anything whose requirements you cannot yet write down. Defer the second tool that overlaps the first. Defer optimisation work on a pipeline whose shape is still changing — incremental refresh processes only changed partitions rather than a full dataset, which is worth doing, but not worth doing to a model you are about to re-grain.

Defer, too, the temptation to standardise across a platform that has not settled. Standardisation is a way of locking in decisions; lock in the ones you are confident about and leave the rest loose. Tooling choices at the plumbing layer — Microsoft Fabric, Databricks, Azure Synapse Analytics, dbt for transformation, Power BI for consumption — are easier to live with when the seams between them are explicit and the semantics sit in a layer you can lift.

When do you revisit the decision?

On two triggers, not on a calendar. First, when the vendor’s ceiling starts costing you — when you are writing workarounds rather than configuration, and the workarounds are accumulating. Second, when the capability stops being a commodity, which happens when the business starts asking for something in that layer that no vendor can supply because it only makes sense inside your organisation.

Until one of those fires, leave the decision alone. Re-litigating a settled build-versus-buy call is one of the cheapest ways to spend a quarter with nothing to show for it. The platform owner’s job here is not to find the optimal answer; it is to find a defensible one, wrap it in a seam that permits a change of mind, and then spend the attention somewhere it compounds.