The metrics that actually matter

A benchmark is only as meaningful as the metrics it captures, and ClickHouse rewards measuring a specific, interrelated set. Throughput and latency are the headline figures, but they must be read alongside the efficiency metrics that determine cost. A cluster that hits its latency target while burning every available core has no headroom; one that meets throughput targets while reading ten times more data than necessary is a tuning opportunity disguised as a capacity problem.

The metrics below form the core of any serious ClickHouse benchmark. Capture all of them together, because optimizing one in isolation almost always moves another.

Note the emphasis on percentiles rather than averages. An average latency hides the tail, and for customer-facing analytics the tail is the experience users remember. Always benchmark and report p95 and p99, and treat data read per query as the leading indicator it is: in a columnar engine, latency is overwhelmingly a function of how many granules a query touches.

One metric deserves promotion above the rest: data read per query, expressed as a fraction of the table. It is the closest thing ClickHouse offers to a universal explanation of performance, because nearly every latency and throughput outcome traces back to how much data the engine had to touch. When a query is slow, the first question is not “how many cores do we have” but “what fraction of the table did this read, and why” — and a benchmark that captures this number for every query gives you the answer directly rather than forcing you to infer it from wall-clock time alone.

Real-world workload profiles

The single biggest mistake in ClickHouse benchmarking is using data and queries that do not resemble production. ClickHouse behaves very differently across workload types, so a benchmark is only valid if it mirrors the profile you actually run. Four profiles cover the large majority of production deployments, and each stresses the engine in a distinct way.

Before writing a single benchmark query, classify your workload against these profiles and build the test data and query mix to match. Generate data with realistic cardinality and distribution rather than uniform random values, which compress and prune unrealistically well and flatter the engine. The closer the test data resembles production in shape and skew, the more the benchmark predicts real behavior.

A benchmarking methodology that holds up

A defensible benchmark is reproducible, isolates variables, and measures the system under conditions that resemble production. The single most useful tool is clickhouse-benchmark, which runs a stream of queries at a configurable concurrency and reports the full latency distribution along with rows and bytes processed per second. For standardized cross-system comparison, the ClickBench suite provides a well-defined analytical workload, though you should still complement it with your own queries.

Method discipline matters more than tool choice. Separate cold-cache and warm-cache runs and report both, because the first run pays I/O the cache later hides. Run each test multiple times and report the distribution, not a single number. Change one variable at a time — schema, setting, hardware, or concurrency — so that any difference is attributable. And always capture rows and bytes read from system.query_log alongside wall-clock time, because that is the metric that explains why a number changed.

# Drive a realistic concurrency level and read the full distribution
clickhouse-benchmark --concurrency 16 --iterations 1000 \
  --query "SELECT tenant_id, count() FROM events
           WHERE event_date >= today() - 7 GROUP BY tenant_id"

# Ground truth: what each query actually read
clickhouse-client --query "SELECT query_duration_ms, read_rows,
  formatReadableSize(read_bytes) FROM system.query_log
  WHERE type='QueryFinish' ORDER BY event_time DESC LIMIT 20"

Repeatability deserves particular attention because it is what makes a benchmark evidence rather than anecdote. Fix the ClickHouse version, the server settings, and the dataset, and record them alongside every result, so a number can be reproduced months later or by a colleague. Quiesce the environment — no competing workloads on the host, no background merges mid-run — or, where you cannot, measure with them present and say so. Warm the cache deliberately for warm-run figures and drop it deliberately for cold-run figures, rather than letting cache state vary uncontrolled between runs. A result you cannot reproduce is a result you cannot defend in a capacity plan or a vendor negotiation.

Ingestion throughput analysis

Ingestion is where ClickHouse newcomers most often leave performance on the table, and the cause is almost always insert pattern rather than raw capacity. ClickHouse is built to ingest data in large batches: each insert creates a part, and a flood of tiny inserts produces a flood of tiny parts that the engine must then merge, consuming I/O and CPU and degrading both ingestion and queries. The same hardware can differ by orders of magnitude in sustained throughput depending purely on how rows are batched.

The chart below illustrates the representative shape of this relationship — the absolute numbers depend entirely on row width, hardware, and schema, but the pattern is universal.

The lesson is to batch aggressively. Where the application cannot batch — many independent producers writing small payloads — asynchronous inserts let the server accumulate rows and write them in efficient batches, recovering most of the throughput without changing the client. Tune insert block size, parallelism, and the number of concurrent insert streams, and watch part counts in system.parts to confirm you are not creating merge pressure. ClickHouse’s own guidance on bulk inserts is the right starting point.

When you benchmark ingestion, measure sustained throughput rather than a momentary peak. A cluster can absorb a brief burst that it cannot maintain, because background merges eventually compete with incoming writes for I/O and CPU; the number that matters for capacity planning is the rate the cluster holds for hours without the merge queue growing without bound. Watch the merge backlog and part count throughout the run, and treat a steadily rising part count as the signal that you have exceeded sustainable ingestion regardless of what the instantaneous rows-per-second figure claims. Sustainable throughput, not peak throughput, is what your pipeline must be sized against.

Query latency and concurrency

Query benchmarking has two dimensions that are easy to conflate: single-query latency and behavior under concurrency. A query that returns in 80 milliseconds in isolation may degrade sharply when fifty of them run at once, because they contend for CPU, memory, and I/O bandwidth. For customer-facing analytics, the figure that matters is the concurrency level the cluster sustains while holding p99 under the SLA — not the best-case latency of a single query on an idle system.

Benchmark latency across the percentiles and across realistic concurrency levels, increasing concurrency until p99 breaches your target to find the cluster’s true capacity ceiling. The chart below shows the representative effect of proper indexing and pre-aggregation on tail latency — the kind of improvement a structured tuning engagement targets.

The takeaway is that latency is rarely a hardware problem first. The largest gains come from reducing the data each query reads — through the sorting key, data-skipping indexes, projections, and pre-aggregation — after which concurrency scales because each query consumes less of the shared resource pool. Benchmark before and after each change so the improvement is attributable and defensible.

Hardware and cost efficiency

Throughput numbers are meaningless without the hardware context that produced them, and cost efficiency is increasingly the metric executives actually care about. ClickHouse scales well with CPU cores because query execution is heavily parallelized, benefits enormously from fast local NVMe storage for hot data, and uses memory as the cache that keeps the working set off disk. A benchmark should always record the instance type, core count, memory, and storage class, because a throughput figure without them is uninterpretable.

The modern decision is also architectural: local-disk clusters versus the separation of storage and compute, including object storage tiers and managed offerings such as ClickHouse Cloud. Each has a different cost-and-latency profile, and the right choice depends on whether your workload is latency-critical or cost-critical and how bursty it is. ClickHouse publishes sizing and hardware recommendations that make a sound baseline. The metric to optimize is cost per query or cost per ingested terabyte at the required latency — a figure that turns a benchmark into a budget.

Benchmark scaling explicitly rather than assuming it. ClickHouse parallelizes a single query across cores, so adding cores often reduces latency for large scans, but the benefit tapers once a query is bound by memory bandwidth or I/O rather than CPU. Adding nodes to a cluster increases aggregate throughput and concurrency but introduces network and coordination costs that a single-node test will not reveal. Measure how your specific workload responds to more cores, more memory, and more nodes, because the shape of that scaling curve — not a single data point — is what tells you whether to scale up, scale out, or tune first. Frequently the most cost-effective move is to make each query read less before adding any hardware at all.

Common benchmarking mistakes

Most ClickHouse benchmarks fail in predictable ways, and recognizing the patterns saves weeks of misdirected effort. The most common is unrealistic data: uniformly random values that compress and prune far better than the skewed, correlated data of production, producing throughput numbers that evaporate on real workloads. Close behind is benchmarking at concurrency one, which measures a scenario that never occurs and ignores the contention that defines real capacity.

Other recurring mistakes include comparing a cold-cache run against a warm one, benchmarking a single query in isolation rather than a representative mix, ignoring the merge activity that small inserts generate, and reporting averages that conceal a punishing tail. Perhaps the most consequential is failing to record rows and bytes read, which leaves you unable to explain why a number changed and reduces the exercise to guesswork. A benchmark that cannot explain its own results cannot guide a decision.

Turning benchmarks into decisions

A benchmark is a means, not an end. Its purpose is to answer concrete questions: will this cluster meet our latency SLA at projected growth, how much headroom remains before we must scale, which hardware delivers the lowest cost per query at our target latency, and what is the return on a tuning engagement. Frame every benchmark around the decision it must inform, and the methodology follows naturally from the question.

The discipline that connects measurement to decision is the same throughout this guide: benchmark against representative data and concurrency, capture the full set of metrics including data read, change one variable at a time, and report distributions rather than averages. Build a baseline you re-run as the workload evolves, and a regression becomes visible the moment it appears rather than after users complain. This is precisely the telemetry our engineers establish at the start of an engagement and re-measure at the end, so that every recommendation is backed by a number rather than an opinion.

Done well, benchmarking also reframes the cost conversation. Once you can express performance as cost per query and cost per ingested terabyte at a required latency, infrastructure spend becomes an engineering variable you can optimize rather than a fixed tax you absorb. A tuning engagement that halves the data each query reads does not merely make dashboards faster; it defers the next hardware purchase, lowers the cloud bill, and raises the concurrency ceiling at the same time. That is why we treat benchmarking as the opening move of every performance engagement — it converts a vague sense that the cluster is slow or expensive into a precise, prioritized list of the changes that will pay back fastest.