Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Troubleshooting Disk I/O Performance in ClickHouse

Shiv Iyer

Introduction Disk I/O performance is an absolutely critical element of high performance of ClickHouse servers & systems. Troubleshooting disk I/O performance upon observation of any anomalies is helpful in sustaining performance of your system. Runbook […]

How Regular Expressions Are Implemented in ClickHouse

How are Regular Expressions (RegEx) implemented in ClickHouse?

Shiv Iyer

Introduction In ClickHouse, regular expressions are implemented using the match function and the like operator. The match function is used to check if a string matches a specified regular expression. The syntax for the function […]

Monitoring Load on ClickHouse Server

Shiv Iyer

Introduction Load can significantly impact the performance of a ClickHouse server. The load on a server is a measure of how busy it is, and it can be represented by several different metrics such as […]

Rules for Writing Optimal SQL for Performance

ClickHouse SQL Engineering: Rules for Writing Optimal SQL for Query Performance

Shiv Iyer

Introduction Rules for Writing Optimal SQL are essential for achieving high performance in any database system, as they directly impact how quickly and efficiently businesses can access and process data. Strong SQL engineering ensures that […]

ClickHouse Monitoring: Python Script to Monitor Linux Disk I/O Matrix

Shiv Iyer

Introduction Optimizing and monitoring the performance of your Linux kernel is absolutely key to achieving the highest performance from your ClickHouse cluster. In this article we explore how to use a simple python script to […]

ClickHouse Monitoring: Python Script to Monitor Linux Process Memory Matrix

Shiv Iyer

Runbook for Troubleshooting ClickHouse Replication

Shiv Iyer

Introduction Replication is a critical part of horizontally scaling ClickHouse to meet growing user traffic and data size. Issues in replication are critical to resolve to maintain data integrity and ensure system scalability. In this […]

Runbook for Troubleshooting Excessive Logical IOs in ClickHouse

Monitoring and Troubleshooting Excessive Logical I/Os in ClickHouse

Shiv Iyer

Introduction Excessive logical I/Os in ClickHouse is a key reason for slow, sub-optimal, and expensive query performance and compromised scalability. In this detailed guide, we share tips & tricks to monitor and troubleshoot logical I/Os […]

How to implement Real-time Stream Processing in ClickHouse with Kafka

Shiv Iyer

Introduction Kafka is the ideal open source platform to implement real-time stream processing in ClickHouse for real-time analytics. At ChistaDATA Inc., we have extensive experience in working with Kafka in the context of ClickHouse for […]

How data compression is implemented in ClickHouse?

How are Data Compression Algorithms implemented in ClickHouse?

Shiv Iyer

Introduction ClickHouse implements data compression in several ways to reduce storage space and improve query performance. Here are a few examples of how data compression is implemented in ClickHouse: (1) Dictionary encoding ClickHouse uses dictionary […]