ChistaDATA – ClickHouse Shard Config Generator

We developed a shard config xml generator for ClickHouse. All you have to do download and run the python script and enter your cluster informations to generate an example config file.

Download file via GitHub

git clone https://github.com/ChistaDATA/Shard-Generator.git

Run pyhton script and enter Data Volume, Replica, Shard and Port informations. Script will generate a xml file.

root@c54ec094a31b:/Shard-Generator# python3 shard-generator.py 
Welcome to ClickHouse Sharding Configuration Generator!
Enter the total data volume (in GB): 100   
Enter the replication factor (number of replicas per shard): 2
Enter the number of shards: 3
Enter details for Shard 1, Replica 1:
Enter the host for the replica: 127.0.0.1
Enter the port for the replica: 9000
Enter details for Shard 1, Replica 2:
Enter the host for the replica: 127.0.0.2
Enter the port for the replica: 9001
Enter details for Shard 2, Replica 1:
Enter the host for the replica: 127.0.0.3
Enter the port for the replica: 9002 
Enter details for Shard 2, Replica 2:
Enter the host for the replica: 127.0.0.4
Enter the port for the replica: 9003
Enter details for Shard 3, Replica 1:
Enter the host for the replica: 127.0.0.5
Enter the port for the replica: 9004
Enter details for Shard 3, Replica 2:
Enter the host for the replica: 127.0.0.6
Enter the port for the replica: 9005
Sharding configuration has been generated and saved to 'clickhouse_sharding_config.xml'.

clickhouse_sharding_config.xml has been created. Now lets look at the file:

root@c54ec094a31b:/Shard-Generator# cat clickhouse_sharding_config.xml

<yandex>
    <profiles>
        <!-- Define your query profiles here -->
    </profiles>
    <remote_servers>
        <!-- Define remote servers here if needed -->
    </remote_servers>
    <clickhouse_remote_servers>
        <!-- Define ClickHouse remote servers here if needed -->
    </clickhouse_remote_servers>
</yandex>

<databases>
    <default>
        
        <shard>
            
            <replica>
                <host>127.0.0.1</host>
                <port>9000</port>
            </replica>

            <replica>
                <host>127.0.0.2</host>
                <port>9001</port>
            </replica>

        </shard>

        <shard>
            
            <replica>
                <host>127.0.0.3</host>
                <port>9002</port>
            </replica>

            <replica>
                <host>127.0.0.4</host>
                <port>9003</port>
            </replica>

        </shard>

        <shard>
            
            <replica>
                <host>127.0.0.5</host>
                <port>9004</port>
            </replica>

            <replica>
                <host>127.0.0.6</host>
                <port>9005</port>
            </replica>

        </shard>

    </default>
</databases>

Your example xml file is ready.