Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Why Delta Updates Are Not Recommended in OLAP Databases: A Performance and Efficiency Perspective

Shiv Iyer

Why Delta Updates Are Not Recommended in OLAP Databases: A Performance and Efficiency Perspective Delta Updates in OLAP (Online Analytical Processing) databases are not recommended due to the fundamental design and architecture of these systems, […]

ClickHouse Search: Manticore Full Text Search with Plain Index

Mastering User Management in ClickHouse: A Complete Guide to Authentication, Authorization, and Future Security Enhancements

Shiv Iyer

User Management in ClickHouse: A Comprehensive Guide Introduction User management is a critical aspect of any analytical application, as it ensures secure access to data while maintaining flexibility for various users. In ClickHouse, user management […]

Integrating Parquet File Ingestion into ClickHouse Using Kafka: A Step-by-Step Guide

Shiv Iyer

Unlock the Power of Data: Seamlessly Integrate Parquet File Ingestion into ClickHouse with Kafka – Your Ultimate Step-by-Step Guide to Optimized Performance! ClickHouse Parquet Ingestion enables a seamless, high-performance workflow for moving data from distributed […]

ClickHouse Data Compression Techniques for Time-series Datasets

Optimizing Non-SARGable Predicates in ClickHouse for Improved Query Performance

Shiv Iyer

ClickHouse Query Optimization Non-SARGable (Search ARGument ABLE) predicates are conditions in SQL queries that prevent the database engine from using indexes efficiently, leading to full table scans and degraded query performance. Implementing and handling Non-SARGable […]

Tuning ClickHouse for High-Velocity Data Ingestion in Distributed Tables

Implementing Tiered Storage in ClickHouse: Leveraging S3 for Efficient Data Archival and Compliance

Shiv Iyer

ClickHouse S3 Archival is a common strategy for handling large volumes of data efficiently, particularly for compliance purposes where data must be retained but is queried infrequently. Using tiered storage like S3 for archiving data […]

When to Avoid Indexing in ClickHouse for Optimal Performance

Implementing Custom Access Policies in ClickHouse: A Comprehensive Guide

Shiv Iyer

Implementing access policies in ClickHouse similar to SQL Server’s Purview Access Policies requires combining ClickHouse’s built-in access control mechanisms with additional scripting and possibly external tools. ClickHouse does not have a direct equivalent to SQL […]

Implementing an Oracle RMAN-Like Backup and Recovery Toolkit for ClickHouse

Shiv Iyer

Implementing a comprehensive backup and recovery toolkit for ClickHouse is essential to ensure data integrity, consistency, and reliability, forming the core of ClickHouse Data Reliability Engineering. While ClickHouse lacks a built-in tool as comprehensive as […]

ClickHouse Redo Operations for Data Reliability

Efficient Strategies for Purging Data in ClickHouse: Real-Life Use Cases and Detailed Implementation

Shiv Iyer

Efficiently purging data from ClickHouse is crucial for maintaining performance and managing storage costs, especially when dealing with large, real-life datasets. Here are some detailed strategies, complete with real-life data sets and use cases: 1. […]

Troubleshooting High CPU Usage in ClickHouse

Securing ClickHouse Data at Rest: A Guide to Implementing Filesystem-Level Encryption

Shiv Iyer

ClickHouse does not directly support Transparent Data Encryption (TDE) in the same way that some other database systems do, such as Oracle or SQL Server, which provide built-in TDE capabilities to automatically encrypt database files. […]

Enhancing Data Processing Workflows with Chained Materialized Views in ClickHouse

Shiv Iyer

Chaining materialized views in ClickHouse is a powerful feature that can significantly enhance data processing workflows by creating layers of data transformation and aggregation. This technique involves creating a series of materialized views where each […]

ChistaDATA Inc.

Enterprise-class 24*7 ClickHouse Consultative Support and Managed Services

Articles by Shiv Iyer

Why Delta Updates Are Not Recommended in OLAP Databases: A Performance and Efficiency Perspective

Mastering User Management in ClickHouse: A Complete Guide to Authentication, Authorization, and Future Security Enhancements

Integrating Parquet File Ingestion into ClickHouse Using Kafka: A Step-by-Step Guide

Optimizing Non-SARGable Predicates in ClickHouse for Improved Query Performance

Implementing Tiered Storage in ClickHouse: Leveraging S3 for Efficient Data Archival and Compliance

Implementing Custom Access Policies in ClickHouse: A Comprehensive Guide

Implementing an Oracle RMAN-Like Backup and Recovery Toolkit for ClickHouse

Efficient Strategies for Purging Data in ClickHouse: Real-Life Use Cases and Detailed Implementation

Securing ClickHouse Data at Rest: A Guide to Implementing Filesystem-Level Encryption

Enhancing Data Processing Workflows with Chained Materialized Views in ClickHouse