Managing ClickHouse Configuration & Checking Restart Requirements

ClickHouse server configuration comprises two main components: server settings (config.xml) and user settings (users.xml). By default, these files reside in the /etc/clickhouse-server/ directory as config.xml and users.xml.

To streamline configuration management and ensure smooth ClickHouse upgrades, we recommend avoiding direct modifications to vendor-supplied configuration files. Instead, adopt a structured approach:

1. Separation of Custom Configurations:

• Place custom configuration changes into separate .xml files stored in designated sub-folders.
• This separation enhances clarity and simplifies maintenance.

2. Folder Structure:

• /etc/clickhouse-server/users.d: Sub-folder for user settings.
• /etc/clickhouse-server/config.d: Sub-folder for server settings.
• /etc/clickhouse-server/conf.d: Sub-folder for any configuration, encompassing both user and server settings.

3. File Naming and Application Order:

• File names for custom XML files can be arbitrary but are applied in alphabetical order.
• This ensures predictable application of configurations.

listen_host

<clickhouse>
  <listen_host>::</listen_host>
</clickhouse>

macros.xml

<clickhouse>
  <macros>
    <cluster>test</cluster>
    <replica>host22</replica>
    <shard>0</shard>
    <server_id>41295</server_id>
    <server_name>host22.server.com</server_name>
  </macros>
</clickhouse>

access_management

<clickhouse>
  <users>
    <default>
      <access_management>1</access_management>
    </default>
  </users>
</clickhouse>

memory_usage

<clickhouse>
    <profiles>
        <default>
            <max_bytes_before_external_group_by>40000000000</max_bytes_before_external_group_by>
            <max_memory_usage>40000000000</max_memory_usage>
        </default>
    </profiles>
</clickhouse>

When making changes to ClickHouse configurations, it’s essential to understand when a server restart is necessary for those changes to take effect. Here’s a guide to help you navigate this process efficiently:

1. Server Settings vs. User Settings:

• Server settings are typically stored in config.xml and files within the config.d directory.
• User settings reside in users.xml and files within the users.d directory.
• Generally, changes to server settings require a server restart, while user settings can often be applied dynamically without restarting.

2. Exceptions to Restart Rules for Server Configurations:

• Certain sections within config.xml do not require a server restart when modified. These include:
  • max_server_memory_usage
  • max_server_memory_usage_to_ram_ratio
  • max_table_size_to_drop (since version 19.12)
  • max_partition_size_to_drop (since version 19.12)
  • max_concurrent_queries (since version 21.11)
  • macros
  • remote_servers
  • dictionaries_config
  • user_defined_executable_functions_config
  • models_config
  • keeper_server
  • zookeeper (although automatic reconnection doesn’t occur)
  • storage_configuration
  • user_directories
  • access_control_path
  • encryption_codecs
  • logger (since version 21.11)
• Additionally, sections like dictionaries, functions, and models are housed in separate files and do not require a restart when modified.

3. User Settings Requiring Restart:

• Most user setting changes can be applied dynamically, but certain settings do require a server restart. These include:
  • background_buffer_flush_schedule_pool_size
  • background_pool_size
  • background_merges_mutations_concurrency_ratio
  • background_move_pool_size
  • background_fetches_pool_size
  • background_common_pool_size
  • background_schedule_pool_size
  • background_message_broker_schedule_pool_size
  • background_distributed_schedule_pool_size
  • max_replicated_fetches_network_bandwidth_for_server
  • max_replicated_sends_network_bandwidth_for_server

By understanding these rules and exceptions, administrators can efficiently manage ClickHouse configurations, minimizing downtime and maximizing system availability.