Skip to content
Architecture

The Data Estate as Competitive Infrastructure

Every competitor holds roughly the same data. The advantage is the decision, and how fast and confidently it can be made. That makes the data estate infrastructure to build, not a cost to minimise.

7 min read
THE DATA ESTATE · INFRASTRUCTURE, NOT A COST CENTRERETURNS ARE NON-LINEARPlatformLAKEHOUSE · WAREHOUSEPipelinesINDUSTRIALISED, NOT HAND-RUNModelsONE DEFINITION OF EVERY METRICDecisionsPRICING · CREDIT · ROUTINGGOVERNANCE AND DEFINITIONS RUN THROUGH EVERY LAYERThe Monday testONE QUESTION, TWO ESTATESWITH THE ESTATEAsked Monday · answered MondayWITHOUT ITReconciled by hand, debated until FridayAND THE WHOLE SEQUENCE RUNS AGAIN NEXT TIMETHE FIRST ESTATE PAYS FOR DATA EVERY TIME IT ASKS A QUESTION · THE SECOND PAID ONCE AND DRAWS THE RETURN

Every competitor in your industry holds roughly the same data. The same transactions, the same customers in a different CRM, the same suppliers, the same sensor readings from the same class of equipment. Whatever edge the data itself once offered, it has long since evened out.

What has not evened out is the decision. Two companies with the same data can take a week or an hour to answer the same question, and can answer it with a shrug or with confidence. That gap is where competitive advantage now lives, and it is a property of the data estate: the platform, the models, the definitions and the governance that sit between the systems that record the business and the people who run it.

That reframes what the estate is for. It is not a cost centre to be minimised, nor a set of tools to be selected and licensed. It is infrastructure, in the same sense as the ERP or the network: the thing that determines whether a question asked on Monday is answered on Monday, or debated until Friday.

The Monday-to-Friday test

The simplest diagnostic of a data estate is to pick a question an executive asked recently and trace what happened next.

In an organisation with reporting but no real estate, the question goes to an analyst, who pulls an extract from two systems, reconciles them in a spreadsheet, discovers that finance and sales count customers differently, asks around, builds a view, and presents it on Friday with caveats. The next time a similar question is asked, the whole sequence runs again, because nothing from the first time was kept.

In an organisation with an estate, the question is answered from a governed model that already holds the agreed definition of a customer, with a lineage back to the source systems, and the answer is available the same day, to anyone with the right access, without an analyst in the loop. The next question is cheaper than the last, because the model, the definitions and the trust were built once and are reused.

The first organisation is paying for data every time it asks a question. The second paid once and is drawing the return. That is the difference between a cost and an asset, and it is why returns on data are non-linear.

Returns are non-linear, and most firms stop before the curve bends

There are three recognisable stages, and the value does not grow evenly across them.

Reporting. The estate exists to produce reports. Numbers are reconciled by people, definitions live in their heads, and every report is a small project. Most organisations have been here for years and have made peace with the cost. The return is modest and flat: the reports get produced, decisions get made on them, and nothing compounds.

Self-service with governance. Definitions are agreed and implemented in one place, quality is measured at source, and the business can answer its own questions within the guardrails. Analysts move from reconciling to analysing. This is where the return starts to climb, because every question answered adds to a shared model rather than to a private spreadsheet.

Decision-native operations. Data is not consulted before a decision; it is part of how the decision is made. A replenishment order is placed by a model and reviewed by exception. A case is routed by its predicted risk. A price is set from the margin model that finance and sales both trust. Here the return is steep, because the estate is no longer informing the business but running parts of it, and the quality of every one of those decisions improves as the model learns from the outcomes.

The curve bends between the second and third stages, and that is where most organisations stop. They reach governed self-service, which is a real achievement, and declare the data programme complete. The estate becomes a reporting utility again, and the firms that pushed on, into decisions, pull away.

Technical debt with a business interest rate

What stops organisations reaching the third stage is rarely ambition and usually debt: the accumulated shortcuts in the estate that made the second stage possible but make the third one unsafe.

Free-hand SQL in the load path, written to get a report out and never industrialised, so that nobody can say with confidence what the numbers went through on the way. Extracts that leave the warehouse into spreadsheets and come back as uploads, undocumented. The same metric defined three times in three tools because reconciling them was somebody else’s problem. A warehouse that is technically a single platform but holds three sets of books.

Each of these was a rational decision at the time. Together they carry interest, paid every month in reconciliation effort, in decisions delayed by doubt, and in AI initiatives that stall because nobody will certify the training data. We worked with an oil and gas major whose sales, finance and procurement functions each ran on their own numbers; the interest on that debt was a management team that spent its monthly review arguing about which figure was right rather than what to do about it. Industrialising the load path and putting the three functions on one governed model did not add data; it removed the argument.

The debt is not paid down by buying a new platform. It is paid down by deciding, domain by domain, what the business actually needs to trust and then building that properly.

Sequencing: decisions first, then data, then platform

Building the estate well is a sequencing problem, and the sequence matters more than the technology.

Start from the decisions that matter. Not “what data do we have” but “which decisions, made better, would move the business”. Pricing, replenishment, credit, maintenance, onboarding, allocation of sales effort. Five or six, ranked by value at stake and by how far their data is from being trustworthy.

Model the data those decisions need. For each decision, the entities, the definitions and the sources. This is where the owner of the decision and the owner of the data meet, and where the definition of “customer” finally gets settled, because now there is a decision riding on it.

Make that data trustworthy end to end. Governed definitions in a semantic layer the tools read from, quality rules at source, lineage from the decision back to the record. Done for the first domain properly, this is the template for every domain after it.

Then extend, domain by domain. The second domain onboards faster than the first because the patterns exist. The fifth is routine.

The platform choices, lakehouse or warehouse, which cloud, which BI tool, which orchestration, follow from the decisions and the data rather than leading them. They matter, and they are easier to get right once you know what the estate is for. The organisations that choose the platform first usually end up with an excellent platform holding the same three sets of books.

What “built well” looks like

An estate built as infrastructure has a few recognisable properties, and they are worth writing down because they are what the investment is for.

One view across applications and functions. Sales, finance, procurement and operations read from the same governed model, with the same definitions, and a figure means the same thing in every room.

Self-service, with governance underneath it. People build their own views, and the views are trustworthy, because the definitions and the quality are controlled where the data is produced rather than where it is consumed.

AI-ready as a property, not a product. Data that is defined, governed and lineaged is data a model can be trained on and a regulator can ask about. “AI-ready” is not something you buy; it is what a well-built estate already is.

A platform that grows with the business rather than against it. A new product, a new region, an acquisition: each extends the model instead of forcing a rebuild, because the patterns for a domain were settled on the first one.

Someone accountable for running it. Infrastructure is operated. Whether that is an internal team or a managed analytics arrangement on a fixed monthly model, the estate needs an owner who keeps the pipelines, the models and the definitions aligned as the business changes. The estates that decay are the ones everyone assumed somebody else was looking after.

Where to start

The work begins with the decisions, which is why it starts as data and AI strategy rather than as a platform project: which decisions, what they are worth, what data they need, and what it would take to trust it. The governance to make that data trustworthy is the first workstream, and the data engineering to build the platform follows from both.

If you want a one-line test of where your own estate stands, use the Monday-to-Friday one. Pick last week’s hardest question and count the days. Then ask what it would be worth to make it hours.

  • Architecture
  • Data & AI Strategy
  • Data engineering
  • Operating model

Related perspectives

All articles
GOVERNANCE · WHO DECIDESOWNERS DECIDE · STEWARDS MAINTAIN · ONE FORUM, MONTHLYDOMAINS · A BUSINESS OWNER ON EACHCustomerOWNER · HEAD OF SALESSupplierOWNER · HEAD OF PROCUREMENTProductOWNER · HEAD OF OPERATIONSSTEWARDS KEEP DEFINITIONS AND QUALITY CURRENTFour decisionsBUSINESS DECISIONS, NOT DATA DECISIONS1Who defines a metric2Who fixes quality at source3Who approves a new use4Who answers the regulatorOne forumMONTHLY · 30–60 MINSelf-servicebecomes safeAIbecomes assurableRegulatory questionstake hours, not weeksGOVERNANCE OWNED BY THE PEOPLE WHO RUN THE BUSINESS PRODUCES TRUST · DELEGATED TO A FUNCTION, IT PRODUCES DOCUMENTATION

Governance

Governance Is a Strategy Problem, Not a Checkbox

OPEN ORDERS · SCORED WHILE THEY RUNOrder receivedCredit checkPick & packShippedDelivered#20471 · Order · 2-day promiseAT CREDIT CHECK · 3.2 DAYS82%→ Release hold, expedite#20458 · Order · partial stockAT PICK & PACK · 1.1 DAYS41%→ Split shipment#20463 · OrderAT SHIPPED · 0.4 DAYS12%On trackLATE-DELIVERY PROBABILITY · MODEL ON ORDER ATTRIBUTESLATE DELIVERIES · LAST 12 WEEKSAGENT ACTIONS · LOGGED TO THE EVENT LOGRELEASE HOLD · SPLIT SHIPMENT · RE-ROUTE CARRIER

Process intelligence

From Seeing the Process to Running It: Process Intelligence, Execution Intelligence, and the Context AI Needs

EVENT LOG · THREE FIELDSCASE IDACTIVITYTIMEORD-20471Order received09:14ORD-20471Credit check09:31ORD-20458Order received09:40ORD-20471Credit check11:52ORD-20458Pick & pack13:05ORD-20471Pick & pack16:20ORD-20458Shipped17:48ORD-20471 · CREDIT CHECK TWICE → A REWORK LOOPDISCOVERED PROCESS · DAY 5ReceivedCredit checkPick & packShippedDeliveredRework loop · 31% of cases · +2.4 days

Process intelligence

What Your Event Log Needs Before Process Mining Will Work