How ClickHouse wire protocol is implemented?

How ClickHouse wire protocol is implemented?


The ClickHouse wire protocol is a binary protocol that is used to communicate between a client and a ClickHouse server. The protocol is implemented in the ClickHouse server and client libraries, and it uses the TCP/IP protocol for communication.

The protocol consists of several types of packets, such as:

  1. Handshake Packet: The first packet sent by the client to the server to initiate the connection. It contains the client version and the client capabilities.
  2. Query Packet: The packet sent by the client to the server to execute a query. It contains the query text, query parameters, and the query settings.
  3. Data Packet: The packet sent by the server to the client to return the query results. It contains the data in a binary format, which is compressed and encoded.
  4. Progress Packet: The packet sent by the server to the client to return the progress of the query execution. It contains the number of rows processed, the number of bytes processed, and the estimated time remaining.
  5. Exception Packet: The packet sent by the server to the client when an exception occurs during the query execution. It contains the exception code and the exception message.
  6. Profile Packet: The packet sent by the server to the client to return the query profile. It contains the query execution plan, the time taken to execute each step of the plan, and the memory used by each step.

The communication between the client and the server is initiated by the client, and it consists of several steps:

  1. The client sends the Handshake Packet to the server to initiate the connection
  2. The server sends the Handshake Packet to the client to confirm the connection
  3. The client sends the Query Packet to the server to execute the query
  4. The server sends the Progress Packet to the client to inform about the progress of the query execution
  5. The server sends the Data Packet to the client to return the query results
  6. The client sends the Query Packet to the server to execute the next query

In summary, the ClickHouse wire protocol is a binary protocol that is used to communicate between a client and a ClickHouse server. The protocol consists of several types of packets, such as Handshake Packet, Query Packet, Data Packet, Progress Packet, Exception Packet and Profile Packet. These packets are used to initiate the connection, execute the query, return the query results, report the progress of the query execution, report the exception and profile of the query execution. The communication between the client and the server is initiated by the client and the communication is done using TCP/IP protocol.

About Shiv Iyer 56 Articles
Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.