chproxy/config/README.md at e47a098d41acda52aad31bd6ae0140e59ad3746b · ContentSquare/chproxy

Possible types used in configuration:

<bool>: a boolean value true or false
<addr>: string value consisting of a hostname or IP followed by an optional port number
<scheme>: a string that can take the values http or https
<duration>: a duration matching the regular expression ^([0-9]+)(w|d|h|m|s|ms|µs|ns)
<networks>: string value consisting of IP, IP mask or named group, for example "127.0.0.1" or "127.0.0.1/24".
<host_name>: string value consisting of host name, for example "example.com"
<byte_size>: string value matching the regular expression /^\d+(\.\d+)?[KMGTP]?B?$/i, for example "100MB"
<map[label_name]label_value>: а map consisting of string keys and string values for optional custom labels in metrics

Global configuration consist of:

# Whether to print debug logs
log_debug: <bool> | default = false [optional]

# Whether to ignore security warnings
hack_me_please: <bool> | default = false [optional]

# Allow ping server
allow_ping: <bool> | default = false [optional]

# Named list of cache configurations
caches:
  - <cache_config> ...

# Named list of parameters to apply to each query
param_groups:
  - <param_groups_config> ...

# Named network lists
network_groups: <network_groups_config> ... [optional]

# Maximum total size of fail reason of queries. Config prevents large tmp files from being read into memory, affects only cachable queries
# The default value is set to 1 Petabyte.
# If error reason exceeds limit "unknown error reason" will be stored as a fail reason
max_error_reason_size: <byte_size>

# Settings for connection pool to ClickHouse
connection_pool:
  max_idle_conns: 100
  max_idle_conns_per_host: 2
  # Connection idle timeout must be < Clickhouse `keep_alive_timeout`.
  idle_conn_timeout: 9

server:
  <server_config> [optional]

# List of allowed users
# which requests will be proxied to ClickHouse
users:
  - <user_config> ...

clusters:
  - <cluster_config> ...

<network_groups_config>

# Name of group
name: "office"

# List of networks access is allowed from
# Each list item could be IP address or subnet mask
networks: <networks> ...

<file_system_cache_config>

# Cache name, which may be passed into `cache` option on the `user` level.
#
# Multiple users may share the same cache.
name: <string>

mode: "file_system"

file_system:
    # Path to directory where cached responses will be stored.
    dir: <string>

    # Maximum cache size.
    max_size: <byte_size>

# Expiration time for cached responses.
expire: <duration>

# DEPRECATED: default value equal to `max_execution_time` should be used.
#             New configuration parameter will be provided to disable the protection at will.
# When multiple requests with identical query simultaneously hit `chproxy`
# and there is no cached response for the query, then only a single
# request will be proxied to clickhouse. Other requests will wait
# for the cached response during this grace duration.
# This is known as protection from `thundering herd` problem.
#
# By default `grace_time` is 5s. Negative value disables the protection
# from `thundering herd` problem.
grace_time: <duration>

# Maximum total size of request payload for caching. The default value
# is set to 1 Petabyte.
# The default value set so high is to allow users who do not use response size limitations virtually unlimited cache.
max_payload_size: <byte_size>

# Whether a query cached by a user can be used by another user
shared_with_all_users: <bool> | default = false [optional]

<distributed_cache_config>

# Cache name, which may be passed into `cache` option on the `user` level.
#
# Multiple users may share the same cache.
name: <string>

mode: "redis"

# Applicable for cache mode: redis
redis:
  # list of addresses to redis nodes
  # you should use multiple addresses only if they all belong to the same redis cluster.
  addresses:
    - <string> # example "localhost:6379"
  username: <string>
  password: <string>
  pool_size: <int>
  db_index: <int> | default = 0 [optional] # This option is only applicable for non-clustered Redis instance.

# Expiration time for cached responses.
expire: <duration>

# DEPRECATED: default value equal to `max_execution_time` should be used.
#             New configuration parameter will be provided to disable the protection at will.
# When multiple requests with identical query simultaneously hit `chproxy`
# and there is no cached response for the query, then only a single
# request will be proxied to clickhouse. Other requests will wait
# for the cached response during this grace duration.
# This is known as protection from `thundering herd` problem.
#
# By default `grace_time` is 5s. Negative value disables the protection
# from `thundering herd` problem.
grace_time: <duration>

# Maximum total size of request payload for caching. The default value
# is set to 1 Petabyte.
# The default value set so high is to allow users who do not use response size limitations virtually unlimited cache.
max_payload_size: <byte_size>

# Whether a query cached by a user can be used by another user
shared_with_all_users: <bool> | default = false [optional]

<param_groups_config>

# Group name, which may be passed into `params` option on the `user` level.
- name: <string>
# List of key-value params to send
params:
  - key: <string>
    value: <string>

<server_config>

# HTTP server configuration
http: <http_config> [optional]

# HTTPS server configuration
https: <https_config> [optional]

# Metrics handler configuration
metrics: <metrics_config> [optional]

<http_config>

# TCP address to listen to for http
listen_addr: <addr>

# List of networks or network_groups access is allowed from
# Each list item could be IP address or subnet mask
allowed_networks: <network_groups>, <networks> ... | optional

# ReadTimeout is the maximum duration for reading the entire
# request, including the body.
read_timeout: <duration> | optional | default = 1m

# WriteTimeout is the maximum duration before timing out writes of the response.
# Default is largest MaxExecutionTime + MaxQueueTime value from Users or Clusters
write_timeout: <duration> | optional

// IdleTimeout is the maximum amount of time to wait for the next request.
idle_timeout: <duration> | optional | default = 10m

<https_config>

# TCP address to listen to for https
listen_addr: <addr> | optional | default = `:443`

# List of networks or network_groups access is allowed from
# Each list item could be IP address or subnet mask
allowed_networks: <network_groups>, <networks> ... | optional

# ReadTimeout is the maximum duration for proxy to reading the entire
# request, including the body.
read_timeout: <duration> | optional | default = 1m

# WriteTimeout is the maximum duration for proxy before timing out writes of the response.
# Default is largest MaxExecutionTime + MaxQueueTime value from Users or Clusters
write_timeout: <duration> | optional

// IdleTimeout is the maximum amount of time for proxy to wait for the next request.
idle_timeout: <duration> | optional | default = 10m

# Certificate and key files for client cert authentication to the server
# If you change the cert & key files while chproxy is running, you have to restart chproxy so that it loads them.
# Triggering a SIGHUP signal won't work as for the rest of the configuration.
cert_file: <string> | optional
key_file: <string> | optional

# Autocert configuration via letsencrypt
autocert: <autocert_config> | optional

<autocert_config>

# Path to the directory where autocert certs are cached
cache_dir: <string>

# List of host names to which proxy is allowed to respond to
# see https://godoc.org/golang.org/x/crypto/acme/autocert#HostPolicy
allowed_hosts: <host_name> ... | optional

<metrics_config>

# List of networks or network_groups access is allowed from
# Each list item could be IP address or subnet mask
allowed_networks: <network_groups>, <networks> ... | optional

# Prometheus metric namespace
namespace: <string> | optional

# Labels that should be added to each sent prometheus metrics
constant_labels: <map[label_name]label_value>

<user_config>

# User name, will be taken from BasicAuth or from URL `user`-param
name: <string>

# User password, will be taken from BasicAuth or from URL `password`-param
password: <string> | optional

# Must match with name of `cluster` config,
# where requests will be proxied
to_cluster: <string>

# Must match with name of `user` from `cluster` config,
# whom credentials will be used for proxying request to CH
to_user: <string>

# Maximum number of concurrently running queries for user.
# By default there is no limit on the number of concurrently
# running queries.
max_concurrent_queries: <int> | optional | default = 0

# Maximum duration of query execution for user
# By default there is a 120 sec limit the query duration.
max_execution_time: <duration> | optional | default = 120s

# Maximum number of requests per minute for user.
# By default there are no per-minute limits.
# A negative value would effectively block the user.
requests_per_minute: <int> | optional | default = 0

# The burst of request packet size token bucket for user
# By default there are no request packet size limits
request_packet_size_tokens_burst: <byte_size> | optional | default = 0

# The request packet size tokens produced rate per second for user
# By default there are no request packet size limits
request_packet_size_tokens_rate: <byte_size> | optional | default = 0

# Maximum number of requests waiting for execution in the queue.
# By default requests are executed without waiting in the queue
max_queue_size: <int> | optional | default = 0

# Maximum duration the request may wait in the queue.
# By default 10s duration is used
max_queue_time: <duration> | optional | default = 10s

# Whether to deny http connections for this user
deny_http: <bool> | optional | default = false

# Whether to deny https connections for this user
deny_https: <bool> | optional | default = false

# Whether to allow `CORS` requests for this user.
# Such requests are needed for `tabix`.
allow_cors: <bool> | optional | default = false

# List of networks or network_groups access is allowed from
# Each list item could be IP address or subnet mask
allowed_networks: <network_groups>, <networks> ... | optional

# Optional response cache name from <cache_config>
# By default responses aren't cached.
cache: <string> | optional

# Optional group of params name to send to ClickHouse with each proxied request from <param_groups_config>
# By default no additional params are sent to ClickHouse.
params: <string> | optional

# The user is wildcarded
# Name matches prefix* or *suffix or *
# Name and password to ClickHouse are obtained
# from original request, not from cluster user
is_wildcarded: <bool> | optional | default = false

<cluster_config>

# Name of CH cluster, must match with `to_cluster`
name: <string>

# Scheme: `http` or `https`; would be applied to all nodes
scheme: <scheme> | optional | default = "http"

# Node addresses. Requests would be balanced among them.
#
# Either nodes or replicas may be configured, but not both.
nodes: <addr> ...

# The cluster may contain multiple replicas instead of flat nodes.
#
# Chproxy selects the least loaded node among the least loaded replicas.
#
# Either nodes or replicas may be configured, but not both.
replicas:
    - <replica_config>

# List of ClickHouse cluster users
users:
    - <cluster_user_config> ...

# KillQueryUser - user configuration for killing timed out queries.
# By default timed out queries are killed from `default` user.
kill_query_user: <kill_query_user_config> | optional

# HeartBeat - user configuration for heart beat requests.
heartbeat: <heartbeat_config> | optional

# RetryNumber - user configuration for query retry when one host cannot respond.
# By default there are no query retries.
# Note: Retrying may impact performance if there are many large-payload requests (such as inserts),
# because it requires copying the request body for reuse.
retry_number: 0

<replica_config>

# Replica name
name: <string>

# Node addresses in the replica. Requests are balanced among them.
nodes: <addr> ...

<cluster_user_config>

# User name in ClickHouse `users.xml` config
name: <string>

# User password in ClickHouse `users.xml` config
password: <string> | optional

# Maximum number of concurrently running queries for user
# By default there is no limit on the number of concurrently
# running queries.
max_concurrent_queries: <int> | optional | default = 0

# Maximum duration of query execution for user
# By default there is a 120 sec limit the query duration.
max_execution_time: <duration> | optional | default = 120s

# Maximum number of requests per minute for user.
# By default there are no per-minute limits
requests_per_minute: <int> | optional | default = 0

# The burst of request packet body size token bucket for user
# By default there are no request packet size limits
request_packet_size_tokens_burst: <byte_size> | optional | default = 0

# The request packet body size tokens produced rate per second for user
# By default there are no request packet size limits
request_packet_size_tokens_rate: <byte_size> | optional | default = 0

# Maximum number of requests waiting for execution in the queue.
# By default requests are executed without waiting in the queue
max_queue_size: <int> | optional | default = 0

# Maximum duration the request may wait in the queue.
# By default 10s duration is used
max_queue_time: <duration> | optional | default = 10s

<kill_query_user_config>

# User name to access CH with basic auth
name: <string>

# User password to access CH with basic auth
password: <string> | optional

<heartbeat_config>

# An interval for checking all cluster nodes for availability
interval: <duration> | optional | default = 5s

# A timeout of wait response from cluster nodes
timeout: <duration> | optional | default = 3s

# The parameter to set the URI to request in a health check
request: <string> | optional | default = `/?query=SELECT%201`

# Reference response from clickhouse on health check request
response: <string> | optional | default = `1\n`

# Credentials to send heartbeat requests
# for anything except '/ping'.
# If not specified, the first cluster user' creadentials are used
user: <string> | optional
password: <string> | optional

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Possible types used in configuration:

Global configuration consist of:

<network_groups_config>

<file_system_cache_config>

<distributed_cache_config>

<param_groups_config>

<server_config>

<http_config>

<https_config>

<autocert_config>

<metrics_config>

<user_config>

<cluster_config>

<replica_config>

<cluster_user_config>

<kill_query_user_config>

<heartbeat_config>

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Possible types used in configuration:

Global configuration consist of:

<network_groups_config>

<file_system_cache_config>

<distributed_cache_config>

<param_groups_config>

<server_config>

<http_config>

<https_config>

<autocert_config>

<metrics_config>

<user_config>

<cluster_config>

<replica_config>

<cluster_user_config>

<kill_query_user_config>

<heartbeat_config>