Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Enhancing ClickHouse Query Efficiency: The Power of Materialized Columns in Practice

Shiv Iyer

Introduction Incorporating materialized columns into ClickHouse for managing complex filtering conditions represents a strategic optimization that significantly boosts database performance. This technique revolves around pre-calculating and storing the results of expressions directly within the table, […]

Long integer query optimization in ClickHouse

Optimizing Long Integer Queries in ClickHouse: Strategies for High-Speed Data Analysis

Shiv Iyer

Introduction Navigating the terrain of querying long integer data (64-bit integers) in ClickHouse is a journey of optimizing performance through strategic choices in data types, leveraging indexing, and fine-tuning query operations. ClickHouse’s architecture is primed […]

RocksDB + ClickHouse for High-Velocity Workloads

Enhancing Data Ingestion: Integrating RocksDB with ClickHouse for High-Velocity Workloads

Shiv Iyer

Introduction Integrating RocksDB with ClickHouse for high-velocity, high-volume data ingestion leverages the strengths of both systems to address specific challenges. RocksDB, a high-performance, embedded key-value store optimized for fast storage on flash and high-speed disk, […]

Optimizing Performance: Inside ClickHouse’s Thread Pool Management

Shiv Iyer

Introduction ClickHouse manages its thread pool operations with a keen focus on maximizing performance and efficiently handling concurrent queries and tasks. The internal handling of thread pools in ClickHouse is designed to optimize the usage […]

Troubleshooting inadequate system resources error ClickHouse

Troubleshooting Inadequate System Resources error in ClickHouse

Shiv Iyer

Introduction : ClickHouse System Resources error Troubleshooting “Inadequate System Resources” errors in ClickHouse often involves dealing with settings like max_memory_usage and max_bytes_before_external_group_by. These settings are crucial for managing memory usage, especially in environments where resources […]

ClickHouse v/s PostgreSQL & MySQL for Real-time Analytics

Shiv Iyer

“ClickHouse, with its columnar storage approach, is inherently suited for high-velocity data ingestion and real-time analytics, offering superior performance, scalability, and cost-effectiveness, making it a strategic choice for organizations seeking real-time insights and decision-making.” Introduction […]

ClickHouse Troubleshooting - Profiling Queries for Performance

ClickHouse Query Profiling for Performance Monitoring

Shiv Iyer

Introduction Creating a Python CLI (Command-Line Interface) application for custom profiling of ClickHouse Server queries involves several components. The application will allow users to input queries via the command line, execute them on a ClickHouse […]

ClickHouse JOIN: Understanding the Internal Mechanics of JOIN operations

Shiv Iyer

Introduction ClickHouse JOIN operations are executed differently compared to traditional SQL databases, primarily due to its columnar storage architecture and distributed data processing capabilities. Understanding how JOINs work internally in ClickHouse involves looking at the […]

ClickHouse Performance: Optimizing HASH GROUP BY and ORDER BY Queries

Shiv Iyer

Introduction In the realm of database management, particularly with ClickHouse, query optimization is a critical aspect of ensuring efficient data processing and retrieval. Among the various queries, HASH GROUP BY and ORDER BY stand out […]

ClickHouse Troubleshooting: Understanding Estimated I/O and CPU Costs

Shiv Iyer

Introduction In ClickHouse, like many database systems, performance troubleshooting and query optimization often involve understanding various metrics, including estimated I/O (Input/Output) costs and estimated CPU (Central Processing Unit) costs. These metrics are crucial for identifying […]

ChistaDATA Inc.

Enterprise-class 24*7 ClickHouse Consultative Support and Managed Services

Articles by Shiv Iyer

Enhancing ClickHouse Query Efficiency: The Power of Materialized Columns in Practice

Optimizing Long Integer Queries in ClickHouse: Strategies for High-Speed Data Analysis

Enhancing Data Ingestion: Integrating RocksDB with ClickHouse for High-Velocity Workloads

Optimizing Performance: Inside ClickHouse’s Thread Pool Management

Troubleshooting Inadequate System Resources error in ClickHouse

ClickHouse v/s PostgreSQL & MySQL for Real-time Analytics

ClickHouse Query Profiling for Performance Monitoring

ClickHouse JOIN: Understanding the Internal Mechanics of JOIN operations

ClickHouse Performance: Optimizing HASH GROUP BY and ORDER BY Queries

ClickHouse Troubleshooting: Understanding Estimated I/O and CPU Costs