Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Monitoring Query Latency due to Wait and Latch Events in ClickHouse

Shiv Iyer

Introduction In ClickHouse, you can monitor query latency and waits/latches using the system.query_log and system.metrics tables. Here is an example query that can be used to monitor query latency and wait/latch events: This query selects […]

How NULL Values affect ClickHouse Query Performance

ClickHouse Troubleshooting: How NULL Values affect Query Performance

Shiv Iyer

Introduction NULL values in ClickHouse can affect performance in several ways. One way is that they can increase the size of data, which can lead to slower query execution times as more data needs to […]

ClickHouse Troubleshooting: How to Identify Blocks Causing Latch Contention

Shiv Iyer

Introduction Latch contention in ClickHouse can have a significant impact on performance. A latch is a synchronization mechanism that allows multiple threads to access a shared resource, such as a data block, in a controlled […]

ClickHouse Live Views to Compute Real-time Moving Averages

ClickHouse Materialized Views: Using Live Views to Compute Real-time Moving Averages

Shiv Iyer

Introduction ClickHouse Live Views can be used to compute real-time moving averages efficiently. Here’s how: Conclusion By using ClickHouse Live Views and defining the appropriate query, settings, and subscription, you can efficiently compute real-time moving […]

ClickHouse Performance: Implementing Dynamic Disks in ClickHouse 23.2

Shiv Iyer

Introduction Dynamic disks in ClickHouse 23.2 refer to a feature that allows for more efficient use of disk space and improved performance when working with large amounts of data. Traditionally, ClickHouse used a file-based storage […]

Fatal Background Processes in ClickHouse

Shiv Iyer

Introduction ClickHouse is an open-source columnar database management system that is designed to handle high volumes of data and support high concurrency. However, like any complex system, it is possible for background processes to fail, […]

How does ClickHouse Merge Sort Algorithm work?

Shiv Iyer

Introduction ClickHouse uses a merge-sort algorithm to sort data internally. The merge-sort algorithm works by dividing the data into smaller blocks, sorting each block individually, and then merging the sorted blocks together to produce the […]

How to implement correlated columns in ClickHouse?

Shiv Iyer

Introduction Correlated columns are useful in many situations where you need to retrieve data from multiple tables and use data from one table to filter data in another. Some common use cases include: How are […]

How to Configure ClickHouse for Physical & Logical I/O Performance

Shiv Iyer

Introduction ClickHouse is a high-performance column-oriented database management system. It uses a unique approach to both physical and logical I/O that is optimized for performance and scalability. In addition to the physical and logical I/O […]

ClickHouse Caches: Configuring Buffer Cache for High Performance

Shiv Iyer

Introduction The ClickHouse Buffer Cache works by caching frequently accessed data in memory. This cache reduces disk I/O operations and speeds up query performance. The buffer cache is organized as a pool of memory blocks, […]

ChistaDATA Inc.

Enterprise-class 24*7 ClickHouse Consultative Support and Managed Services

Articles by Shiv Iyer

Monitoring Query Latency due to Wait and Latch Events in ClickHouse

ClickHouse Troubleshooting: How NULL Values affect Query Performance

ClickHouse Troubleshooting: How to Identify Blocks Causing Latch Contention

ClickHouse Materialized Views: Using Live Views to Compute Real-time Moving Averages

ClickHouse Performance: Implementing Dynamic Disks in ClickHouse 23.2

Fatal Background Processes in ClickHouse

How does ClickHouse Merge Sort Algorithm work?

How to implement correlated columns in ClickHouse?

How to Configure ClickHouse for Physical & Logical I/O Performance

ClickHouse Caches: Configuring Buffer Cache for High Performance