ClickHouse Network Security Guide

In this article we will talk about how to secure your ClickHouse network and will enrich article with examples.

To boost your ClickHouse network, you have to monitor your network traffic and user who is connecting your database.

Boosting ClickHouse network is follow the steps below;

• Enable TLS
• Encrypt Cluster Communications

Enable TLS

ClickHouse allows for both encrypted and unencrypted network communications. To harden network communications, unencrypted ports should be disabled and TLS enabled.

TLS encryption required a Certificate, and whether to use a public or private Certificate Authority (CA) is based on your needs.

• Public CA: Recommended for external services or connections where you can not control where they will be connecting from.
• Private CA: Best used when the ClickHouse services are internal only and you can control where hosts are connecting from.
• Self-signed certificate: Only recommended for testing environments.

Whichever method is used, the following files will be required to enable TLS with CLickHouse:

• Server X509 Certificate: Default name server.crt
• Private Key: Default name server.key
• Diffie-Hellman parameters: Default name dhparam.pem

Generate Files

The instructions below require the use of openssl, and was tested against version OpenSSL 1.1.1j

Generate the private key

openssl genrsa -aes256 -out server.key 2048

Generate dhparam.pem to create a 4096 encrypted file

openssl dhparam -out dhparam.pem 4096

Create the Certificate Signing Request (CSR) from the generated private key

openssl req -new -key server.key -out server.csr

Store the files server.key, server.csr, and dhparam.pem in a secure location, typically /etc/clickhouse-server/

Create a Private CA

Create the Certificate Private Key

openssl genrsa -aes256 -out internalCA.key 2048

Create the self-signed root certificate from the certificate key

openssl req -new -x509 -days 3650 -key internalCA.key \
    -sha256 -extensions v3_ca -out internalCA.crt

Store the Certificate Private Key and the self-signed root certificate in a secure location.

Sign the server.csr file with the self-signed root certificate:

openssl x509 -sha256 -req -in server.csr -CA internalCA.crt \
    -CAkey internalCA.key -CAcreateserial -out server.crt -days 365

openssl req -subj "/CN=my.host.name" -new -key server.key -out server.crt

<config>
<openSSL>
    <client>
        <invalidCertificateHandler>
            <name>AcceptCertificateHandler</name>
        </invalidCertificateHandler>
    </client>
</openSSL>

Enable TLS in ClickHouse

To enable TLS and disable unencrypted ports:

Review the /etc/clickhouse-server/config.d files. Comment out unencrypted ports, including http_port and tcp_port:

<!-- <http_port>8123</http_port> -->
<!-- <tcp_port>9000</tcp_port> -->

Enable encrypted ports. To more details please visit ClickHouse documentation.

<https_port>8443</https_port>
<tcp_port_secure>9440</tcp_port_secure>

Specify the certificate files to use:

<openSSL>
    <server>
        <!-- Used for https server AND secure tcp port -->
        <certificateFile>/etc/clickhouse-server/server.crt</certificateFile>
        <privateKeyFile>/etc/clickhouse-server/server.key</privateKeyFile>
        <dhParamsFile>/etc/clickhouse-server/dhparams.pem</dhParamsFile>
        ...
    </server>
...
</openSSL>

Encrypt Cluster Communications

If your organization runs ClickHouse as a cluster, then cluster-to-cluster communications should be encrypted. This includes distributed queries and inter service replication. To harden cluster communications:

Create a user for distributed queries. For example, if the cluster is contained in a subdomain named test1,test2, etc. This internal user be set with or without a password:

CREATE USER IF NOT EXISTS internal ON CLUSTER 'my_cluster'
    IDENTIFIED WITH NO_PASSWORD
    HOST REGEXP 'test[47]!'

Enable TLS for interservice replication and comment out the unencrypted interserver port by updating the /etc/clickhouse-server/config.d files:

<!-- <interserver_http_port>9009</interserver_http_port> -->
<interserver_https_port>9010</interserver_https_port> -->

Set an the interserver_http_credentials in the /etc/clickhouse-server/config.d files, and include the internal username and password:

<interserver_http_credentials>
    <user>internal</user>
    <password></password>
</interserver_http_credentials>

Enable TLS for distributed queries by editing the file /etc/clickhouse-server/config.d/remote_servers.xml

Clickhouse version 20 and later You have to do it for each shard

<remote_servers>
    <my_cluster>
    <shard>
        <secret>shared secret text</secret> <!-- Update here -->
        <internal_replication>true</internal_replication>
        <replica>
            <host>test1</host> <!-- Update here -->
            <port>9440</port> <!-- Secure Port -->
            <secure>1</secure> <!-- Update here, sets port to secure -->
        </replica>
    </shard>