ChistaDATA · Data strategy · Updated September 2026
Data Strategy: How to Build One That Holds Up in Production, Under Regulation and for AI
A data strategy is a small set of binding decisions: which business outcomes data must serve, who owns each data product, what service levels it keeps, and which controls it obeys. Technology comes after those decisions, never before.
Most organisations have a data platform. Far fewer have a data strategy that a CFO, a regulator and an on-call engineer would all recognise. This guide is how we build one at ChistaDATA: seven steps, each with a concrete deliverable, a reference architecture, the 2026 regulatory calendar, and the SQL we use to measure whether the strategy is being kept.
01
What this guide decides
- Outcomes, KPIs and data products
- Owners and decision rights
- Freshness, latency and uptime SLOs
- Retention, deletion and cost budgets
02
What it engineers
- Sources, CDC and streaming ingest
- Hot, warm and open lakehouse tiers
- Row policies, RBAC and erasure
- SLO telemetry from system tables
03
What is current in 2026
- EU Data Act and AI Act Omnibus dates
- India DPDP Rules phase-in
- Apache Iceberg v3 and open catalogs
- ClickHouse 26.8 LTS capabilities
04
How it is measured
- p95 latency per data product
- Freshness lag in seconds
- Cost per query and per terabyte
- Restore drills passed per quarter
Definition
What a data strategy is, and what it is not
A data strategy is the plan that connects business objectives to the way data is collected, stored, governed, served and retired. It answers five questions in writing, with named owners: what outcomes the data must move, which data products deliver those outcomes, what service levels each product keeps, which rules constrain it, and what it is allowed to cost.
It is not a technology shortlist, a slide of vendor logos, or a data lake project. Those are implementation choices. A data strategy that starts with “we will adopt platform X” has already skipped the decisions that determine whether platform X will succeed.
It is also not a one-off document. The strategy is a living contract between the business and the platform team, reviewed quarterly against measured results. When the measurements drift, either the platform changes or the strategy does.
| Decision | The question it answers | Evidence it is working |
|---|---|---|
| Outcomes | Which revenue, risk or cost KPI does this data move? | KPI movement attributed to a data product |
| Data products | What is delivered, to whom, with which owner? | Catalogued product with one accountable owner |
| Service levels | How fresh, how fast, how available? | Freshness lag, p95 latency, uptime vs SLO |
| Controls | Who may see it, how long is it kept, how is it erased? | Access reviews, TTL coverage, erasure SLA met |
| Economics | What may it cost to store and to query? | Cost per TB stored and per 1,000 queries |
Why it matters
Why a data strategy matters more in 2026 than it did five years ago
Three forces have turned the data strategy from an architecture exercise into a board-level control.
1. AI has made data quality a delivery risk, not a reporting nuisance
Gartner’s February 2025 research put it bluntly: through 2026, organisations will abandon 60% of AI projects that are not supported by AI-ready data, and 63% of organisations either do not have, or are unsure whether they have, the right data management practices for AI. A model trained or grounded on stale, duplicated or unlabelled data fails in ways that are expensive to detect. A data strategy defines the lineage, freshness and quality gates that make data AI-ready before the first model is trained.
2. Regulation now reaches into architecture
The EU Data Act has applied since 12 September 2025, and from 12 September 2026 new connected products must be designed so users can access their data by default. The EU AI Act was amended by the Digital Omnibus, in force since 27 July 2026, which moves stand-alone high-risk obligations to 2 December 2027. India’s Digital Personal Data Protection Rules were notified on 13 November 2025, with the bulk of fiduciary duties arriving in May 2027. Each of these requires capabilities such as retention, erasure, access logging and data portability that must be built into the platform. They cannot be retrofitted with a policy PDF.
3. Real-time is now the default expectation
Fraud scoring, ad bidding, observability and customer-facing analytics all expect answers in under a second on data that is seconds old. Columnar engines make that economical at very large scale: in our own lab, ClickHouse aggregated one billion rows in 3.34 seconds on two vCPUs, reading 985 MiB of a table that holds 14 GiB compressed (27 GiB uncompressed). That capability only pays off when the data strategy states which products need seconds-fresh data and which can tolerate a nightly batch. Otherwise the whole estate gets engineered, and billed, for the strictest case.
We document the full measurement in Columnar Databases vs Row-Based Databases: Measured on 1 Billion Rows, and the execution model behind it in ClickHouse vectorized query processing.
Reference architecture
A data strategy reference architecture: four layers, two planes
The diagram below is the architecture we use to turn a data strategy into a running platform. The strategy layer on top is decided first. Everything underneath exists to keep its promises, and every component should trace back to an outcome, an owner or an SLO in that top band.
01 · SOURCES
Inventory before integration
OLTP databases, event streams, SaaS and ERP APIs, machine data. Each source gets an owner, a contract and a classification (personal, confidential, public) before it is ingested.
02 · INGEST
Contracts at the edge
Debezium CDC into Kafka, the ClickHouse Kafka engine or async inserts for streams, Parquet on object storage for batch. Schema registry and quality gates reject bad data before it lands.
03 · STORE
Tier by access pattern
Hot ReplicatedMergeTree on NVMe for real-time, TTL moves to S3-backed disks for warm data, Apache Iceberg v3 tables behind a REST catalog for open, multi-engine access.
04 · SERVE
Products, not tables
Dashboards, customer-facing APIs, ML features, vector search and RAG retrieval, plus federated SQL through Trino. Each is a data product with its own SLO.
Method
How to build a data strategy in seven steps
Each step below ends with a deliverable you can review. If a step cannot produce its deliverable, the next step is premature. For a CTO-level view of how we run this with leadership teams, see how ChistaDATA partners with CTOs.
Anchor the data strategy to business outcomes
Start with the three to five outcomes the business will fund this year: reduce fraud loss, raise conversion, cut infrastructure cost, meet a regulatory deadline. For each, name the KPI, its current baseline and the target. Interview the executives who own those KPIs, not only the data team.
Discard every data initiative that does not map to one of these outcomes. That single filter removes most of the scope creep that sinks data programmes.
Assess the current data landscape with evidence
Inventory sources, pipelines, stores and consumers. Measure rather than survey: row counts and growth rates, pipeline failure rates, query latency distributions, storage cost per terabyte, and the age of the freshest record in each critical table. On ClickHouse estates we pull this from system.parts, system.query_log and system.merges; on PostgreSQL from pg_stat_statements and pg_stat_user_tables.
Define data products and owners
Group data by the decision it supports, not by the system it came from. “Customer 360”, “Payment risk features” and “Ad delivery metrics” are products. Each has one accountable owner, documented consumers, a schema contract and a classification. Ownership without decision rights is decoration, so write down what the owner may approve.
Set service levels and budgets
For every product, agree freshness (for example, 95% of events queryable within 10 seconds), query latency (p95 under 800 ms at 200 concurrent queries), availability and recovery objectives (RPO and RTO). Agree a cost envelope too. SLOs are what turn a data strategy from aspiration into something an SRE team can defend at 3 a.m.
Write governance as enforceable controls
Translate policy into mechanisms: role-based access, row and column policies, retention classes enforced by TTL, an erasure procedure with a measured SLA, audit logging, and lineage from source to product. Map each control to the regulation it satisfies. A control that exists only in a document will fail its first audit.
Design the architecture to the SLOs
Now choose technology. Pick the platform pattern that meets the strictest SLO in each product family at the lowest sustainable cost. Real-time products usually justify a columnar OLAP engine such as ClickHouse. Transactional products stay on PostgreSQL or MySQL. Open table formats such as Apache Iceberg keep historical data portable across engines. Design replication, backup and disaster recovery at this stage, not after go-live.
Deliver incrementally and review quarterly
Ship the highest-value product first, measure it against its SLOs and KPI, then expand. Review the whole data strategy each quarter: which KPIs moved, which SLOs held, what cost more than planned, and which regulatory dates are approaching. Build skills alongside the platform; our ChistaDATA University programmes exist for exactly that handover.
Platform patterns
Choosing the platform pattern your data strategy actually needs
We are vendor-neutral by principle, and that includes recommending against ClickHouse when it is the wrong fit. The pattern follows the SLOs set in step four.
| Pattern | Best fit | Watch out for | Typical engines |
|---|---|---|---|
| Real-time OLAP | Sub-second analytics on seconds-fresh events at billions of rows: observability, ad tech, fraud, customer-facing dashboards | Frequent single-row updates, multi-row ACID transactions, highly normalised joins across many large tables | ClickHouse |
| Cloud data warehouse | Governed BI across many business domains, elastic batch workloads, heavy SQL transformation | Per-query or per-credit cost at high concurrency; latency for customer-facing use | Snowflake, BigQuery, Redshift |
| Open lakehouse | Long retention, multi-engine access, ML training sets, avoiding format lock-in | Catalog governance, small-file compaction, query latency without a caching layer | Apache Iceberg or Delta on object storage, queried by Spark, Trino or ClickHouse |
| Transactional + HTAP | Systems of record, strict consistency, moderate analytics on current state | Analytical scans competing with OLTP for buffer pool and I/O | PostgreSQL, MySQL, SQL Server |
| Federated query | Joining across stores without copying data, exploration, migration bridges | Performance is bounded by the slowest source; not a substitute for a serving layer | Trino |
Most mature estates combine at least three of these. The data strategy decides which products live where and how data moves between them. That is also where lock-in is decided: ClickHouse has read Iceberg since 23.3 and added catalog integrations for Unity, AWS Glue, REST and Polaris through 2025. The 26.8 LTS release of 27 August 2026 added Iceberg writes to S3 Tables and Puffin statistics with deletion vectors, so an Iceberg-first storage tier no longer forces a single query engine.
If you are moving an existing warehouse, our ClickHouse migration practice covers Redshift, Snowflake, BigQuery, Druid, Vertica and Elasticsearch sources.
Governance · current as of September 2026
The 2026 regulatory calendar every data strategy must plan for
Dates change. The table reflects published positions at the time of writing; confirm each against the primary source and your counsel before committing a roadmap.
| Regulation | Key dates | What it demands from the data platform |
|---|---|---|
| EU GDPR | In force since 25 May 2018 | Lawful basis, purpose limitation, right to erasure and access, breach notification. Fines up to €20 million or 4% of worldwide annual turnover. |
| EU Data Act | Applicable 12 Sep 2025; access-by-design for new connected products from 12 Sep 2026; cloud switching charges removed from 12 Jan 2027 | Export product and service data to users and third parties on request; portable formats; switching between data processing services without punitive egress fees. |
| EU AI Act (as amended by the Digital Omnibus) | Omnibus in force 27 Jul 2026; stand-alone high-risk (Annex III) from 2 Dec 2027; product-embedded high-risk (Annex I) from 2 Aug 2028 | Training, validation and test data governance; documented provenance; logging; data quality measures for high-risk systems. |
| India DPDP Act 2023 and DPDP Rules 2025 | Rules notified 13 Nov 2025; consent manager provisions from Nov 2026; remaining fiduciary obligations from May 2027 | Itemised notice and consent, purpose-bound retention and erasure, breach reporting, children’s data safeguards. Penalties up to ₹250 crore per instance. |
The architectural implication is consistent across all four: every data product needs a retention class, an erasure path that can be demonstrated, access logging, and lineage to its source. We build these into the data strategy at step five so they are tested controls rather than promises. Our Data SRE practice runs them as part of day-to-day operations.
Engineering
Encoding the data strategy in SQL: retention, access and erasure
A strategy that cannot be expressed as configuration will not survive staff turnover. The examples below show how we encode three data strategy decisions directly in ClickHouse. They are written for ClickHouse 25.8 LTS and later. Test them on a non-production cluster first, and keep a tested backup and DR posture in place before applying any change to production.
1 · Retention classes as table TTL, with hot and warm tiers
-- Prerequisite: a storage policy named 'hot_warm' with volumes 'hot' (NVMe) and 'warm' (S3-backed disk)
CREATE TABLE IF NOT EXISTS analytics.customer_events ON CLUSTER '{cluster}'
(
event_time DateTime64(3, 'UTC') CODEC(Delta(8), ZSTD(1)),
event_date Date MATERIALIZED toDate(event_time),
tenant_id UInt32,
customer_id UInt64,
region LowCardinality(String),
event_type LowCardinality(String),
amount Decimal(18, 2),
payload String CODEC(ZSTD(3))
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/analytics/customer_events', '{replica}')
PARTITION BY toYYYYMM(event_date)
ORDER BY (tenant_id, event_type, event_time)
TTL event_date + INTERVAL 90 DAY TO VOLUME 'warm',
event_date + INTERVAL 25 MONTH DELETE
SETTINGS storage_policy = 'hot_warm',
index_granularity = 8192,
ttl_only_drop_parts = 1;The 90-day and 25-month values are placeholders for the retention class your data owner and legal team sign off. ttl_only_drop_parts = 1 drops whole monthly parts instead of rewriting them, which keeps retention cheap at billions of rows. Adding a DELETE TTL to an existing table is irreversible for expired data, so run it through change control with a verification count first.
2 · Least-privilege access with roles, column grants and row policies
CREATE ROLE IF NOT EXISTS role_eu_analyst ON CLUSTER '{cluster}';
GRANT ON CLUSTER '{cluster}'
SELECT(event_time, tenant_id, region, event_type, amount)
ON analytics.customer_events
TO role_eu_analyst;
CREATE ROW POLICY IF NOT EXISTS rp_customer_events_eu ON CLUSTER '{cluster}'
ON analytics.customer_events
FOR SELECT
USING region = 'EU'
TO role_eu_analyst;
-- Users authenticate with credentials from your secret store, never inline:
-- CREATE USER analyst_01 IDENTIFIED WITH sha256_password BY '${CH_ANALYST_PASSWORD}';The analyst role never sees customer_id or payload, and only reads EU rows. The access decision from step five becomes a reviewable object in system.row_policies and system.grants.
3 · A right-to-erasure request, with gates before and after
-- STEP A: verification (read-only). Record the count in the erasure ticket.
SELECT count() AS rows_to_erase
FROM analytics.customer_events
WHERE customer_id = {customer_id:UInt64};
-- STEP B: CONFIRMATION GATE. Proceed only after the data owner approves the ticket
-- and the count above matches the request scope. This operation is irreversible.
DELETE FROM analytics.customer_events ON CLUSTER '{cluster}'
WHERE customer_id = {customer_id:UInt64};
-- STEP C: validation. Must return 0.
SELECT count() AS rows_remaining
FROM analytics.customer_events
WHERE customer_id = {customer_id:UInt64};DELETE hides rows immediately and removes them physically as parts merge. If your erasure SLA requires physical removal by a deadline, schedule the purge (for example with ALTER TABLE ... APPLY DELETED MASK on supported versions) and log the completion time. Backups and Iceberg snapshots need their own expiry policy, or erased data survives there.Measurement
Measuring whether your data strategy is working
Every recommendation we make is anchored to a metric. These are the ones we put on the data strategy scorecard, with the system table or view that produces each number.
| Metric | Why it matters | Where it comes from |
|---|---|---|
| Freshness lag | Proves the product meets its freshness SLO | system.parts (latest active part per table) or max event time vs now() |
| Query p95 / p99 | User-facing latency SLO | system.query_log, pg_stat_statements |
| Failed query rate | Error budget consumption | system.query_log where exception_code != 0 |
| Bytes read per query | Cost and schema quality signal | system.query_log.read_bytes |
| Compressed TB and growth | Capacity and storage cost | system.parts, system.disks |
| Restore drill result | Only a tested backup is a backup | Quarterly drill log with measured RTO |
Freshness and latency SLOs from ClickHouse system tables
-- Freshness: seconds since the newest part was written, per table
SELECT
database,
table,
max(modification_time) AS last_write,
dateDiff('second', max(modification_time), now()) AS freshness_lag_s
FROM system.parts
WHERE active
AND database = 'analytics'
GROUP BY database, table
ORDER BY freshness_lag_s DESC;
-- Latency and errors: hourly p95 and failures for one data product
SELECT
toStartOfHour(event_time) AS hour,
quantile(0.95)(query_duration_ms) AS p95_ms,
countIf(exception_code != 0) AS failed_queries,
count() AS total_queries
FROM system.query_log
WHERE event_date >= today() - 1
AND type IN ('QueryFinish', 'ExceptionWhileProcessing')
AND has(tables, 'analytics.customer_events')
GROUP BY hour
ORDER BY hour;Feed these into the same dashboard the business uses for its KPIs. When a data strategy review can show that the payment-risk product held a 600 ms p95 all quarter and fraud loss fell against baseline, funding the next phase is an easy decision.
Roadmap
A data strategy roadmap for the first 365 days
This is the cadence we use with enterprise clients. Timelines compress for focused programmes and stretch for multi-country estates, but the order holds.
Days 0–30
Decide
- Outcome register signed by executives
- Baseline report from system telemetry
- Top three data products named with owners
Days 31–90
Design
- SLO sheets and cost envelopes agreed
- Control matrix mapped to GDPR, DPDP, Data Act
- Architecture decisions and DR design
Days 91–180
Deliver
- First product live against its SLOs
- CDC and streaming ingest in production
- First quarterly restore drill passed
Days 181–365
Scale
- Products two and three live
- Iceberg tier for long retention
- AI and ML consumers on governed data
Lessons
Why most data strategy programmes stall
The failure patterns are remarkably consistent across industries. Each one is avoidable if it is named early.
Technology before outcomes
A platform is procured, then a use case is sought. Without an outcome register there is no way to prioritise, and the platform is judged on adoption instead of business impact.
Ownership without authority
Data owners are named but cannot approve schema changes, access or retention. Decisions escalate to committees and stall for weeks.
No service levels
“Real-time” is promised but never defined. Engineering over-builds for the strictest imagined case, and costs grow faster than value.
Governance on paper
Policies exist in documents but not in TTLs, grants and row policies. The first audit or erasure request exposes the gap.
Untested recovery
Backups run nightly, but nobody has restored one at production scale. The first real incident becomes the first drill.
Strategy as a one-off
A document is approved and filed. Without quarterly reviews against measurement, it drifts from reality within two quarters.
How ChistaDATA helps
Turning a data strategy into a production platform with ChistaDATA
ChistaDATA is a full-stack ClickHouse infrastructure operations company, running 24×7×365 on 100% open-source ClickHouse with zero vendor lock-in. We work at the point where data strategy meets production: architecture, performance engineering, scalability, high availability, and the operating discipline that keeps SLOs green.
- ClickHouse consulting for architecture reviews, schema and sort-key design, and capacity models tied to your SLOs.
- 24×7 ClickHouse support with a 15-minute Severity 1 response, 12 hours for S2, 24 hours for S3 and 48 hours for S4.
- ClickHouse managed services for teams that want the platform run to agreed SLOs, including upgrades, backups and DR drills.
- Break-fix engineering when a production cluster needs expert hands immediately.
New to the engine? Start with our guide to understanding ClickHouse for real-time analytics, or read our companion piece on crafting the right data strategy.
FAQ
Data strategy: frequently asked questions
What is the difference between a data strategy and a data architecture?
A data strategy defines outcomes, data products, owners, service levels, controls and budgets. A data architecture is the technical design that delivers them: sources, pipelines, stores and serving layers. Architecture should be derived from the strategy, and the reference architecture above shows how each layer traces back to a strategy decision.
How long does it take to build a data strategy?
The decision work in steps one to five typically takes 30 to 90 days for a focused organisation. The first data product can usually be in production within six months. A data strategy is never finished; it is reviewed every quarter against measured KPIs and SLOs.
Who should own the data strategy?
An executive sponsor, often the CDO, CTO or CIO, owns the strategy as a whole. Each data product has its own accountable owner in the business. Platform and SRE teams own the service levels. What fails is shared ownership with no single name attached.
Do we need ClickHouse to execute a data strategy?
No. ClickHouse is the right engine when products need sub-second analytics on large, fast-growing, append-heavy data. Transactional products belong on PostgreSQL or MySQL, and broad governed BI may suit a cloud warehouse. We recommend the pattern that meets your SLOs at the lowest sustainable cost, including when that is not ClickHouse.
How does a data strategy prepare us for AI?
AI needs data that is documented, fresh, lineage-tracked, quality-checked and lawfully usable. Those are exactly the controls a data strategy puts in place. Gartner expects 60% of AI projects without AI-ready data to be abandoned through 2026, so the data foundation comes before the model.
Which regulations should a 2026 data strategy account for?
At minimum: GDPR, the EU Data Act (applicable since September 2025), the EU AI Act as amended by the Digital Omnibus (high-risk obligations from December 2027), and India’s DPDP Rules (main obligations from May 2027), plus sector rules such as HIPAA, PCI DSS or SOX where they apply. Confirm current positions with counsel.
Sources
References and further reading
- Gartner: Lack of AI-ready data puts AI projects at risk (Feb 2025)
- Gartner: Data quality, why it matters and how to achieve it
- IDC via Network World: 175 zettabytes of data worldwide by 2025
- Cloudflare: HTTP analytics for 6M requests per second using ClickHouse
- European Commission: The Data Act
- White & Case: EU AI Omnibus enters into force, amending the AI Act
- Government of India (PIB): DPDP Rules, 2025 notified
- ClickHouse: Release 26.8 LTS
- ClickHouse: ClickHouse is data lake ready
- Apache Iceberg table specification (v3)
- DAMA International: DAMA-DMBOK 3.0 project
- Salesforce: Data strategy guide
Next step
Build a data strategy your platform can keep
Bring us your outcome list and your current telemetry. In a first working session we will map them to data products, SLOs and an architecture that fits, and tell you plainly where ClickHouse helps and where it does not.
All guidance on this page is general engineering advice. Test every change on a non-production environment before applying it to production, and maintain a robust backup and disaster recovery posture. Regulatory dates reflect published sources as of September 2026 and are not legal advice.