Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Optimizing Indexes in ClickHouse for High-Velocity, High-Volume Data Ingestion

Shiv Iyer

1. Choose the Right Index Type 2. Utilize Merge Tree Indices 3. Properly Define Primary Keys 4. Use Low Cardinality for Secondary Indices 5. Batch Insertions 6. Monitor System Resources 7. Tune Buffer Settings 8. […]

ClickHouse Memory: Tuning max_memory_usage and max_bytes_before_external_group_by

Shiv Iyer

Introduction “Memory is the golden key to performance; unlock its potential with precision tuning in ClickHouse, and transform how your large database handles complex join operations.” – ChistaDATA ClickHouse Performance Engineering Memory management is crucial […]

Mastering Concurrency in ClickHouse by Optimizing ClickHouse Thread Performance

Shiv Iyer

Introduction “Unlock unparalleled efficiency and speed in your data operations; this runbook is your key to mastering ClickHouse thread performance and conquering high concurrency with confidence.” – ChistaDATA Performance Engineering ClickHouse is a high-performance columnar […]

Linux Swaps in ClickHouse: Peak Performance in Memory-Intensive Operations

Shiv Iyer

Introduction “Swap wisely, perform optimally: Fine-tuning Linux swap is the silent key to unleashing ClickHouse’s full potential.” Excessive Linux swaps can significantly degrade the performance of memory-intensive applications, including databases like ClickHouse. When the system […]

ClickHouse Functions: Deep Dive into ClickHouse User Defined Functions (UDFs)

Shiv Iyer

Introduction Empower, Elevate, Automate – With ChistaDATA, transcend the limits of performance and automation in your data infrastructure. User Defined Functions (UDF) in ClickHouse allow users to extend the native capabilities of the database by […]

Comprehensive Guide for ClickHouse Storage Infrastructure Optimization

Shiv Iyer

Introduction Mastering ClickHouse storage optimization is the key to unlocking unparalleled data analytics performance. Optimizing ClickHouse storage infrastructure is vital to ensure the database performs at its best, especially for large-scale data analytics tasks. ClickHouse […]

Demystifying Data Compression in ClickHouse

Shiv Iyer

Introduction “Compressing data is like putting it on a diet; it becomes leaner and faster.” – Unknown Data compression plays a pivotal role in optimizing storage and query performance in any database system, and ClickHouse […]

Troubleshooting High CPU Usage in ClickHouse

Shiv Iyer

Introduction “Don’t watch the clock; do what it does. Keep going.” – Sam Levenson High CPU usage in ClickHouse can be a common challenge, especially when dealing with large datasets and complex queries. It can […]

Materialized Views in ClickHouse for Optimal Server Performance

Shiv Iyer

Introduction ClickHouse, an open-source columnar database management system, is renowned for its ability to handle massive volumes of data while delivering high-performance analytics. However, in real-time analytics scenarios, even ClickHouse can face challenges when it […]

Mastering Index Selection and Troubleshooting in ClickHouse

Shiv Iyer

Introduction Index selection is a critical aspect of database optimization in ClickHouse. Efficient indexes can significantly speed up query execution, but what happens when your queries aren’t using indexes as expected? In this guide, we’ll […]

ChistaDATA Inc.

Enterprise-class 24*7 ClickHouse Consultative Support and Managed Services

Articles by Shiv Iyer

Optimizing Indexes in ClickHouse for High-Velocity, High-Volume Data Ingestion

ClickHouse Memory: Tuning max_memory_usage and max_bytes_before_external_group_by

Mastering Concurrency in ClickHouse by Optimizing ClickHouse Thread Performance

Linux Swaps in ClickHouse: Peak Performance in Memory-Intensive Operations

ClickHouse Functions: Deep Dive into ClickHouse User Defined Functions (UDFs)

Comprehensive Guide for ClickHouse Storage Infrastructure Optimization

Demystifying Data Compression in ClickHouse

Troubleshooting High CPU Usage in ClickHouse

Materialized Views in ClickHouse for Optimal Server Performance

Mastering Index Selection and Troubleshooting in ClickHouse