Product
Data warehouse
The dependable option for structured finance and operations data. SQL-first, mature governance, predictable cost, and a dimensional model that still reproduces last year's numbers.
Stage 01
Sources - most estates have more than they think
The inventory usually turns up systems nobody listed: a legacy instance still feeding a report, a supplier file drop, a departmental database holding something month-end depends on.
- ERP finance, supply chain and HCM modules
- Relational databases across the estate
- Extracts from reporting and consolidation tools
- Supplier and partner file feeds
- APIs where a direct database route does not exist
- Statutory reporting feeds where they touch the same data
Stage 02
Ingest - three routes, chosen per source
Not everything needs the same treatment. A nightly file does not need replication, and a high-volume transactional table does not want a full reload. We pick per source and document why.
- Set-based ELT running inside the warehouse itself
- Change data capture where low latency genuinely matters
- External tables to read object storage files in place, without loading them
- Bulk transfer for historical migration
- Every load restartable, watermarked and logged
- Extraction logic that handles backdated and late-arriving transactions
Layer 01
Staging - a landing layer with no opinions
Source structures preserved exactly, with load metadata attached. No transformation, no filtering, no cleverness. Staging exists so anything downstream can be rebuilt without going back to the source systems for another extract.
- Tables mirror the source, including its inconsistencies
- Load id, source system and extract timestamp on every row
- History retained, so any prior state can be reconstructed
- External tables where a file can simply be read where it sits
- Restricted access - this layer is plumbing, not reporting
Layer 02
Conformed - where the data becomes trustworthy
Data types enforced, business keys established, duplicates resolved, reference data conformed across systems, and quality rules applied with exceptions kept visible. The reconciliation back to source lives here too.
- Deduplication and survivorship rules on real business keys
- Conformed reference data: one customer, one product, one cost centre
- Quality rules with exceptions routed to a review table, never discarded
- Automated source-to-target counts and checksums per load
- Standardised dates, currencies and units across every source
- Documented rules, so every number has a written explanation behind it
Layer 03
Dimensional - modelled for the questions asked
Conformed dimensions and fact tables with surrogate keys and slowly changing history, partitioned and aggregated for the way the business actually queries. This is the layer reporting points at.
- Star schemas with dimensions shared across subject areas
- Surrogate keys and type 2 history, so last year's report still reproduces
- Fact partitioning by period, with indexing strategy set deliberately
- Materialised aggregates for the queries dashboards hit repeatedly
- One agreed definition per metric, held here rather than inside reports
- A governed semantic layer where it earns its place
Stage 06
Serve - and what the platform already includes
Reporting is the usual front door, but a modern warehouse platform bundles capabilities people often buy separately. Worth knowing what you already own before adding anything.
- Dashboards and semantic models over the dimensional layer
- Natural-language querying that generates SQL against your own schemas
- Retrieval over your own documents, using built-in vector search
- In-database machine learning, so data is never extracted to train
- Governed data sharing to partners without copying files around
- Low-code application layers where a small internal tool is the real need
The idea
Same discipline, expressed in schemas
A warehouse and a lakehouse separate their layers for the same reason. The names differ and the technology differs; the argument for keeping them apart does not.
Staging answers what arrived
A one-to-one landing of each source with load metadata attached, so downstream can always be rebuilt without another extract.
Conformed answers what is true
Typed, deduplicated and agreed, with exceptions visible in a review table and reconciliation counts produced on every run.
Dimensional answers what it means
Facts and conformed dimensions with proper history, built for the questions the business asks rather than the way the source stored them.
Cost is predictable
Sizing is known in advance and scales on a schedule you control, which matters more than peak performance for most finance reporting.
Fit
When this is right, and when it is not
A good fit when
- Your reporting runs on structured finance and operations data
- The team is strong in SQL and wants to stay there
- Predictable cost matters more than elastic scale
- Governance, auditability and stable definitions are the priority
- You need last year's report to reproduce exactly, years later
Probably not when
- You have significant streaming, document or unstructured data
- Machine learning at scale is central to the plan
- Volumes are large and spiky, making fixed sizing wasteful
- A lakehouse would serve you better - we will say so
Tools
Platforms we work on: Oracle Autonomous Data Warehouse, SQL Server, Snowflake and the major cloud warehouses.
Ready to build a warehouse?
We can size and build one from nothing, or take on a warehouse that is already running and bring it up to standard.