diff --git a/.github/styles/base/Dictionary.txt b/.github/styles/base/Dictionary.txt index 095289ff87..192dd196b3 100644 --- a/.github/styles/base/Dictionary.txt +++ b/.github/styles/base/Dictionary.txt @@ -910,6 +910,7 @@ workingDir workqueue workspace workspace's +workspaced xai xbox xcode diff --git a/app/_support/code-10-error-invalid-unique-name.md b/app/_support/code-10-error-invalid-unique-name.md new file mode 100644 index 0000000000..2631ab6f03 --- /dev/null +++ b/app/_support/code-10-error-invalid-unique-name.md @@ -0,0 +1,83 @@ +--- +title: "Code 10 \"invalid unique name\" error when an entity name matches a reserved Admin API endpoint" +content_type: support +description: "This error is returned because of a pre-existing Admin API endpoint named 'auth'." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do I get a Code 10 \"invalid unique name\" error when creating an entity named auth?" + a: | + This error is returned because of a pre-existing Admin API endpoint named `auth`. Route + matching prefers the existing Admin API endpoint over a newly created entity, so entity + names that collide with reserved endpoints (such as `auth`, `services`, `routes`, `consumers`, + and others listed in the error) must be avoided when creating new entities. +related_resources: [] +--- + +## Problem + +Using a fresh `kong` image in DB mode after running `kong migrations bootstrap`, you try to get an Upstream named `auth` by making the request: + +```bash +curl http://localhost:8001/upstreams/auth +``` + +The following error is returned: + +```json +{"code":10,"name":"invalid unique name","message":"must not be one of: workspaces, consumers, certificates, services, routes, snis, upstreams, targets, consumer_groups, plugins, tags, ca_certificates, clustering_data_planes, parameters, vaults, key_sets, keys, filter_chains, files, legacy_files, workspace_entity_counters, consumer_reset_secrets, credentials, audit_requests, audit_objects, rbac_users, rbac_roles, rbac_user_roles, rbac_role_entities, rbac_role_endpoints, admins, developers, document_objects, applications, application_instances, groups, group_rbac_roles, login_attempts, keyring_meta, keyring_keys, event_hooks, licenses, consumer_group_plugins, consumer_group_consumers, rbac_user_groups"} +``` + +## Cause + +This error is returned because of a pre-existing Admin API endpoint named `auth`. + +## Solution + +Along with the referenced list in the error, avoid the following entity names when creating new entities, because Route matching prefers the existing Admin API endpoint over the newly created one: + +``` +/keyring, +/timers, +/userinfo, +/endpoints, +/auth, +/status, +/cache, +/config, +/metrics, +/acme, +/ca_certificates, +/tags, +/vaults, +/certificates, +/consumers, +/keys, +/services, +/routes, +/admins, +/groups, +/oauth2, +/jwts, +/graphql_ratelimiting_advanced_cost_decoration, +/sessions, +/acme_storage, +/oauth2_tokens, +/acls, +/degraphql_routes, +/plugins, +/workspaces, +/snis, +/upstreams, +/targets, +/files, +/licenses, +/consumer_groups, +/developers, +/document_objects, +/applications, +/login_attempts, +``` diff --git a/app/_support/configuration-does-not-fit-in-lmdb-database-error.md b/app/_support/configuration-does-not-fit-in-lmdb-database-error.md new file mode 100644 index 0000000000..c3d3c6fc88 --- /dev/null +++ b/app/_support/configuration-does-not-fit-in-lmdb-database-error.md @@ -0,0 +1,50 @@ +--- +title: "HTTP 413 \"Configuration does not fit in LMDB database\" error when pushing config to data planes" +content_type: support +description: Pushing configuration to data planes fails with an HTTP 413 "Configuration does not fit in LMDB database" error; raise `lmdb_map_size` to fix it. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do configurations fail to push to data planes with an HTTP 413 \"Configuration does not fit in LMDB database\" error?" + a: | + The default LMDB size is `128m`, and configurations that exceed it fail to get pushed to data + planes. Increase the size by raising `lmdb_map_size` (for example, `KONG_LMDB_MAP_SIZE=256m`, + or `lmdb_map_size: "256m"` on Kubernetes, along with a larger `prefixDir` `sizeLimit`). +related_resources: [] +--- + +## Problem + +We see the following error in our Kong logs: + +``` +time="2023-01-31T11:12:33Z" level=error msg="could not update kong admin" error="posting new config to /config: HTTP status 413 (message: \"Configuration does not fit in LMDB database, consider raising the \\\"lmdb_map_size\\\" config for Kong\")" subsystem=dataplane-synchronizer +``` + +We see configurations failing to get pushed to our data planes. + +## Solution + +The default size of the LMDB is `128m`. To increase the size, make the following changes to your configuration. + +General: + +```bash +KONG_LMDB_MAP_SIZE=256m +``` + +Kubernetes: + +```yaml +env: + lmdb_map_size: "256m" +``` + +```yaml +deployment: + prefixDir: + sizeLimit: 1Gi +``` diff --git a/app/_support/error-attempt-to-concatenate-field-authority-a-nil-value.md b/app/_support/error-attempt-to-concatenate-field-authority-a-nil-value.md new file mode 100644 index 0000000000..6ce07fbb7f --- /dev/null +++ b/app/_support/error-attempt-to-concatenate-field-authority-a-nil-value.md @@ -0,0 +1,60 @@ +--- +title: "Error: attempt to concatenate field 'authority' (a nil value) after upgrading to 3.X" +content_type: support +description: Resolve the "attempt to concatenate field 'authority' (a nil value)" error after upgrading to {{site.base_gateway}} 3.X by setting a valid KONG_ADMIN_GUI_URL value. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do I get \"attempt to concatenate field 'authority' (a nil value)\" after upgrading to {{site.base_gateway}} 3.X?" + a: | + The `KONG_ADMIN_GUI_URL` parameter holds a value that was valid in 2.X but breaks in 3.X — + either an asterisk (`KONG_ADMIN_GUI_URL="*"`) or a value missing the protocol + (`KONG_ADMIN_GUI_URL=127.0.0.1:8002`). Set it to a valid value using the format + `://(:)`, for example `KONG_ADMIN_GUI_URL=http://127.0.0.1:8002`. +related_resources: + - text: "{{site.base_gateway}} `admin_gui_url` configuration" + url: /gateway/configuration/#admin-gui-url +--- + +## Problem + +After upgrading to {{site.base_gateway}} 3.x, I am getting the following error on start up: + +``` +Error: attempt to concatenate field 'authority' (a nil value) +``` + +## Cause + +The `KONG_ADMIN_GUI_URL` parameter holds a value that was valid in 2.X but breaks in 3.X — either an asterisk (`KONG_ADMIN_GUI_URL="*"`) or a value missing the protocol (`KONG_ADMIN_GUI_URL=127.0.0.1:8002`). + +## Solution + +1. Check whether you have an asterisk in your `KONG_ADMIN_GUI_URL` parameter: + + ```bash + KONG_ADMIN_GUI_URL="*" + ``` + + This value needs to be set with the following format: + + ``` + ://(:) + ``` + + Items within the parenthesis are optional. + +2. Check whether you have not written the protocol in your `KONG_ADMIN_GUI_URL` parameter as below: + + ```bash + KONG_ADMIN_GUI_URL=127.0.0.1:8002 + ``` + + Fix it by adding the protocol as below: + + ```bash + KONG_ADMIN_GUI_URL=http://127.0.0.1:8002 + ``` diff --git a/app/_support/error-nginx-kong-lua-9-expected-near-customformat-when-using-a-custom-log-format.md b/app/_support/error-nginx-kong-lua-9-expected-near-customformat-when-using-a-custom-log-format.md new file mode 100644 index 0000000000..d3ba9fe50a --- /dev/null +++ b/app/_support/error-nginx-kong-lua-9-expected-near-customformat-when-using-a-custom-log-format.md @@ -0,0 +1,76 @@ +--- +title: "Error: `nginx_kong.lua:9: '=' expected near 'customformat'` when using a custom `log_format`" +content_type: support +description: This can occur when attempting to use `log_format` outside the http context. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do I get `nginx_kong.lua:9: '=' expected near 'customformat'` when using a custom `log_format`?" + a: | + This occurs when `log_format` is placed outside the `http` context in a custom nginx template — + `log_format` can only be used within the `http` context. When using a custom template, properly + add the contents of `nginx_kong.lua` inside `http {}`, with the `log_format` directive defined + there. +related_resources: [] +--- + +## Problem + +When attempting to use a custom `log_format` in an nginx template, for example when using the Correlation ID plugin to log IDs, the following error appears on startup or reload: + +``` + +ERROR: /usr/local/share/lua/5.1/luarocks/loader.lua:104: error loading module 'kong.templates.nginx_kong' from file '/usr/local/share/lua/5.1/kong/templates/nginx_kong.lua': + /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua:9: '=' expected near 'customformat' +stack traceback: + /usr/local/share/lua/5.1/luarocks/loader.lua:104: in function + [C]: in function 'require' + /usr/local/share/lua/5.1/kong/cmd/utils/prefix_handler.lua:9: in main chunk + [C]: in function 'require' + /usr/local/share/lua/5.1/kong/cmd/reload.lua:8: in main chunk + [C]: in function 'require' + /usr/local/share/lua/5.1/kong/cmd/init.lua:71: in function + /usr/local/bin/kong:9: in function 'file_gen' + init_worker_by_lua:46: in function + [C]: in function 'xpcall' + init_worker_by_lua:53: in function +``` + +Where `customformat` is the name you've given to the `log_format`. + +## Cause + +This can occur when attempting to use `log_format` outside the `http` context. The `log_format` directive can only be used within the `http` context. + +## Solution + +When using a custom template be sure to follow the instructions as documented, properly adding the contents of the `nginx_kong.lua` within `http {}`, for example: + +```bash + +worker_processes ${{NGINX_WORKER_PROCESSES}}; +daemon ${{NGINX_DAEMON}}; +pid pids/nginx.pid; + +events { + use epoll; # a custom setting + multi_accept on; +} + +http { + +log_format customformat '$remote_addr - $remote_user [$time_local] ' + '"$request" $status $body_bytes_sent ' + '"$http_referer" "$http_user_agent" ' + 'Kong-Request-ID="$sent_http_Kong_Request_ID"'; + + # contents of the nginx_kong.lua template follow: + + access_log ${{PROXY_ACCESS_LOG}} customformat; + + ... # etc +} +``` diff --git a/app/_support/error-require-resty-http-not-allowed-within-sandbox.md b/app/_support/error-require-resty-http-not-allowed-within-sandbox.md new file mode 100644 index 0000000000..e29a416a04 --- /dev/null +++ b/app/_support/error-require-resty-http-not-allowed-within-sandbox.md @@ -0,0 +1,58 @@ +--- +title: "\"require 'resty.http' not allowed within sandbox\" error in serverless or Exit Transformer plugins" +content_type: support +description: Plugins that execute arbitrary Lua code run in a sandbox that blocks require of certain modules; allow the module or disable the sandbox to resolve it. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do I get \"require 'resty.http' not allowed within sandbox\" when using a plugin that runs Lua code?" + a: | + Plugins that execute arbitrary Lua code (such as serverless plugins or the exit transformer) + run in a sandbox with restricted access to the global environment, so they cannot `require` + certain modules like `resty.http`. To resolve it, either set `untrusted_lua_sandbox_requires` + to include the allowed modules (the preferred option, e.g. `untrusted_lua_sandbox_requires = resty.http`), + or turn off the sandbox by setting `untrusted_lua` to `on`. Allowing modules or disabling the + sandbox can create opportunities to escape it, so proceed with caution. +related_resources: + - text: "`untrusted_lua_sandbox_requires`" + url: /gateway/configuration/#untrusted-lua-sandbox-requires + - text: "`untrusted_lua`" + url: /gateway/configuration/#untrusted-lua +--- + +## Problem + +When attempting to use a plugin that allows execution of arbitrary Lua code (that is, serverless plugins, exit transformer, and so on) you receive the message: + +```json +{"message":"An unexpected error occurred"} +``` + +A review of the Kong error log shows something similar to the below: + +``` +2023/02/10 12:49:34 [error] 2107#0: *2582 [kong] init.lua:317 [pre-function] /usr/local/share/lua/5.1/kong/tools/kong-lua-sandbox.lua:171: /usr/local/share/lua/5.1/kong/tools/sandbox.lua:88: require 'resty.http' not allowed within sandbox, client: 192.168.64.1, server: kong, request: "GET /echo HTTP/1.1", host: "localhost:8000" +``` + +## Cause + +These types of plugins, by default, operate in a sandboxed environment. The sandbox function has restricted access to the global environment and only has access to standard Lua functions that will generally not cause harm to the {{site.base_gateway}} node. This means you cannot arbitrarily "require" certain modules, in our example here `resty.http`. + +## Solution + +There are two options to resolve this, however you should be aware of the potential risks. + +Warning: + +Allowing certain modules may create opportunities to escape the sandbox. For example, allowing `os` or `luaposix` may be unsafe. Turning off the sandbox will allow unchecked access that can cause severe damage. Proceed with extreme caution and ensure any such changes are thoroughly vetted in lower environments. + +1. The preferred option is to set the `untrusted_lua_sandbox_requires` parameter to include the modules allowed to be loaded with "require" inside the sandboxed environment. This will give you more control allowing you to explicitly and consciously define a list. + + ``` + untrusted_lua_sandbox_requires = resty.http + ``` + +2. Turn off the sandbox environment by setting `untrusted_lua` to on. This allows functions to have unrestricted access to the global environment and can load any Lua modules. diff --git a/app/_support/gateway-faqs.md b/app/_support/gateway-faqs.md new file mode 100644 index 0000000000..7d10804d14 --- /dev/null +++ b/app/_support/gateway-faqs.md @@ -0,0 +1,63 @@ +--- +title: "{{site.base_gateway}} FAQs" +content_type: support +description: "Answers to common questions about {{site.base_gateway}} plugins, workspaces, custom plugin caching, and DNS resolver configuration." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: Where can I find answers to common {{site.base_gateway}} questions? + a: | + This page collects frequently asked questions about {{site.base_gateway}}: plugin scope and + workspaces, retrieving secrets in the Request Transformer Advanced plugin, cache `hit_level` + definitions for custom plugins, and the configurable DNS resolver options. +related_resources: [] +--- + +## Plugins + +### Can a global plugin be applied to all workspaces? + +A plugin that isn't associated with any Service, Route, or Consumer is considered global and runs on every request, but only within its own workspace. There is no way to set up a plugin that automatically applies to all workspaces. + +### How do I retrieve a secret with Kong Secret Management in the Request Transformer Advanced plugin? + +You can fetch a token from Kong's Secret Management and pass it to the upstream service in the `Authorization` header as `Bearer `: + +1. Enable `KONG_UNTRUSTED_LUA="on"` in your {{site.base_gateway}} configuration. +2. Add the following under the plugin's `config.add_headers`: + + ```lua + Authorization:$((function() local value, err = + kong.vault.get("{vault://env/kong-token}") if value then return "Bearer " + .. value end end)()) + ``` + +Make sure you have made the required changes to the Vault reference. This configuration dynamically retrieves the token from Kong's Secret Management (Vault) and appends it to the `Authorization` header as a Bearer token. + +### What are the `hit_level` definitions for caching in custom plugins? + +When you implement caching in a custom plugin, `hit_level` indicates which cache level a value was fetched from. The cache level hierarchy is: + +- **L1**: Least-Recently-Used Lua VM cache using `lua-resty-lrucache`. Provides the fastest lookup if populated, and uses LRU eviction to avoid exhausting the workers' Lua VM memory. +- **L2**: `lua_shared_dict` memory zone shared by all workers. This level is only accessed if L1 was a miss, and prevents workers from requesting the L3 cache. +- **L3**: a custom function that is only run by a single worker to avoid the dog-pile effect on your database or backend (via `lua-resty-lock`). Values fetched via L3 are set to the L2 cache for other workers to retrieve. + +This is also defined in the `lua-resty-mlcache` repo on GitHub. + +## Networking + +### What options can be configured with the {{site.base_gateway}} DNS resolver? + +You can use the `RES_OPTIONS` environment variable to override the following DNS resolver options: + +``` +rotate +ndots +timeout +attempts +``` + +For additional details on these settings, refer to the [`resolv.conf` documentation](https://man7.org/linux/man-pages/man5/resolv.conf.5.html). See also the [DNS resolver configuration](/gateway/configuration/#dns-resolver-section). diff --git a/app/_support/getting-this-instance-contains-workspaced-entities-that-need-a-custom-migration-in-helm-upgrade.md b/app/_support/getting-this-instance-contains-workspaced-entities-that-need-a-custom-migration-in-helm-upgrade.md new file mode 100644 index 0000000000..3efde6df3d --- /dev/null +++ b/app/_support/getting-this-instance-contains-workspaced-entities-that-need-a-custom-migration-in-helm-upgrade.md @@ -0,0 +1,51 @@ +--- +title: "Getting `This instance contains workspaced entities that need a custom migration` in Helm upgrade" +content_type: support +description: "This error during a Helm upgrade usually means you are using an older chart version; update the Helm repo and rerun the upgrade." +products: + - kic +works_on: + - on-prem + - konnect +tldr: + q: "Why does my Helm upgrade fail with `This instance contains workspaced entities that need a custom migration`?" + a: | + This error during the `helm upgrade` is usually because you are using an older chart version. + Update the Helm repo locally with `helm repo update`, then rerun the upgrade, for example: + + ```bash + helm upgrade kong-test -f values.yaml kong/kong --set migrations.preUpgrade=true --set migrations.postUpgrade=false + ``` +related_resources: [] +--- + +## Problem + +After running the `helm upgrade` command, the `wait-for-db` container fails and prints: + +``` +This instance contains workspaced entities that need a custom migration. +please use the provided helpers to migrate them: +kong migrations upgrade-workspace-table vaults_beta +Error: nginx not running in prefix: /tmp/tmp.okiPjN Run with --v (verbose) or --vv (debug) for more details +``` + +## Cause + +This error during the upgrade is usually because you are using an older chart version. + +## Solution + +1. Make sure the Helm repo is updated locally: + + ```bash + helm repo update + ``` + +2. Then proceed with the Helm upgrade: + + ```bash + helm upgrade kong-test -f values.yaml kong/kong --set migrations.preUpgrade=true --set migrations.postUpgrade=false + ``` + +You can find more information about the Helm upgrade in the Kong Helm Chart documentation. diff --git a/app/_support/how-can-i-insert-a-field-from-the-request-body-into-the-header-for-rate-limiting-purposes-in-kong.md b/app/_support/how-can-i-insert-a-field-from-the-request-body-into-the-header-for-rate-limiting-purposes-in-kong.md new file mode 100644 index 0000000000..400bfb3bbe --- /dev/null +++ b/app/_support/how-can-i-insert-a-field-from-the-request-body-into-the-header-for-rate-limiting-purposes-in-kong.md @@ -0,0 +1,55 @@ +--- +title: Insert a field from the request body into the header for rate limiting in Kong +content_type: support +description: Use the pre-function plugin with the Kong PDK to extract a field from the request body and insert it into a request header so you can rate limit on that field. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How Can I Insert a Field from the Request Body into the Header for Rate Limiting Purposes in Kong? + a: | + Use the `pre-function` plugin together with the Kong Plugin Development Kit (PDK) to manipulate + the request before it reaches the upstream service. Add a Lua script to the plugin's + `config.access` parameter that uses `kong.request.get_body()` to extract the desired field and + `kong.service.request.add_header()` to insert it into a request header (for example, + `x-contact-number`). You can then configure rate limiting based on that header. +related_resources: [] +--- + +## Overview + +How Can I Insert a Field from the Request Body into the Header for Rate Limiting Purposes in Kong? + +## Steps + +To insert a field from the request body directly into the header so you can rate limit on a specific field in Kong, use the Pre-function plugin together with the Kong Plugin Development Kit (PDK). This approach lets you manipulate the request before it reaches the upstream Service, so you can extract the desired field from the request body and insert it into the request header. + +Here is a practical example of how you can accomplish this task: + +1. First, ensure that the pre-function plugin is enabled on your Kong instance. You can check the official Kong documentation for guidance on how to enable plugins if you're not familiar with this process. + +2. Next, use the following Lua script as a template for your pre-function plugin configuration. This script extracts the `number` field from a JSON object in the request body and adds it as a custom header (`x-contact-number`) to the request. + +```lua +-- Example request body +-- {"contacts": [{"phoneNumber": {"number": "123456789"}}]} + +local rl_header_name = kong.request.get_body() +if (rl_header_name.contacts and #rl_header_name.contacts == 1) then + if (rl_header_name.contacts[1].phoneNumber and + rl_header_name.contacts[1].phoneNumber.number) then + kong.service.request.add_header("x-contact-number", + rl_header_name.contacts[1].phoneNumber.number) + end +end +``` + +3. Add this Lua script to the `config.access` parameter of the pre-function plugin configuration. This step is crucial as it tells Kong to execute your custom logic during the access phase of the request processing. + +4. Once the pre-function plugin is correctly configured with this script, Kong will automatically insert the `number` field from the request body into the request header as `x-contact-number`. You can then set up rate limiting based on this header as per your requirements. + +Please note, the provided code snippet is a basic example meant to illustrate the concept. Depending on your specific use case, you might need to extend this script to handle various edge cases, such as requests with multiple contacts or missing fields. Implementing proper error handling and validation logic will ensure that your configuration is robust and can handle unexpected inputs gracefully. + +This method provides a flexible way to manipulate request headers based on the content of the request body, enabling more granular control over rate limiting and other policies in Kong. diff --git a/app/_support/how-to-audit-route-configuration-changes-via-kong-audit-log-apis.md b/app/_support/how-to-audit-route-configuration-changes-via-kong-audit-log-apis.md new file mode 100644 index 0000000000..77b917f66d --- /dev/null +++ b/app/_support/how-to-audit-route-configuration-changes-via-kong-audit-log-apis.md @@ -0,0 +1,173 @@ +--- +title: Auditing Route configuration changes via Kong audit log APIs +content_type: support +description: Use the {{site.ee_product_name}} audit log APIs to trace who changed a Route and to detect Routes created with empty or missing paths. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How to audit Route configuration changes via Kong Audit Log APIs? + a: | + {{site.ee_product_name}} provides two audit log endpoints: `/audit/requests` (captures who did what, + including HTTP method, user, and payload) and `/audit/objects` (captures the data entity that + was created or updated, with full snapshots). Ensure audit logging is enabled with + `KONG_AUDIT_LOG: "on"`, then use the provided `kong-audit-query.sh` script to identify the RBAC + user who created a specific route, or to detect routes with empty or missing `paths` that can + cause 409 Conflict collisions. +related_resources: [] +--- + +## Overview + +In large-scale {{site.base_gateway}} deployments, teams often automate Route provisioning via CI/CD. A common issue arises when Routes are created with missing or empty `paths`, leading to 409 Conflict errors during deployment due to Route collisions. + +## Steps + +In environments where multiple teams or pipelines interact with {{site.base_gateway}}, it is vital to trace configuration changes, especially for objects like Routes. {{site.ee_product_name}} provides two audit log endpoints: + +1. `/audit/requests`: captures who did what, including HTTP method, user, and payload. +2. `/audit/objects`: captures what data entity was created/updated, with full snapshots. + +The script below was built to: + +- Help customers identify the RBAC user who made a route change. +- Pinpoint changes to a specific route. +- Enable SRE or Platform teams to perform accountable debugging based on audit trails. + +### Script + +This script helps identify who created a specific route, or detect routes with empty/missing paths (which can cause conflicts). Output includes the RBAC user, method, workspace, and full route payload. + +```bash +#!/bin/bash + +BASE="http://localhost:8081" +TOKEN="password" +ROUTE_NAME="$1" +NEXT="/audit/requests" + +log() { + local ts="$1" + local msg="$2" + local dt + dt=$(date -d "@$ts" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || date -r "$ts" '+%Y-%m-%d %H:%M:%S') + echo "[$dt] $msg" +} + +validate_json() { + echo "$1" | jq empty >/dev/null 2>&1 +} + +get_next_page() { + echo "$1" | jq -r '.next // "null"' +} + +process_route_audit() { + echo "$1" | jq -r --arg name "$2" ' + .data[]? + | select(.method | test("POST|PUT|PATCH|DELETE")) + | select(.path | test("/routes")) + | select(.payload != null) + | (.payload | fromjson? // {}) as $pl + | select($pl.name == $name) + | { + summary: "User: \(.rbac_user_name // "UNKNOWN") | Workspace: \(.workspace // "N/A") | Request Source: \(.request_source // "N/A") | Method: \(.method) | Path: \(.path) | Route name: \($name) | At: \(.request_timestamp)", + payload: $pl + } + | .summary, (.payload | @json) + ' +} + +process_empty_paths_audit() { + echo "$1" | jq -r ' + .data[]? + | select(.method | test("POST|PUT|PATCH|DELETE")) + | select(.path | test("/routes")) + | select(.payload != null) + | (.payload | fromjson? // {}) as $pl + | select(($pl.paths? == null) or ($pl.paths | length == 0)) + | { + summary: "User: \(.rbac_user_name // "UNKNOWN") | Workspace: \(.workspace // "N/A") | Request Source: \(.request_source // "N/A") | Method: \(.method) | Path: \(.path) | Changed Route has EMPTY or MISSING paths | At: \(.request_timestamp)", + payload: $pl + } + | .summary, (.payload | @json) + ' +} + +pretty_print_json() { + local line="$1" + if echo "$line" | jq empty >/dev/null 2>&1; then + echo "Payload:" + echo "$line" | jq . + echo "---" + else + echo "$line" + fi +} + +main() { + log "$(date +%s)" "Starting Kong audit query (route filter: '${ROUTE_NAME:-}')" + + while [ "$NEXT" != "null" ]; do + URL="${BASE}${NEXT}" + RESP=$(http --body "$URL" "kong-admin-token:$TOKEN") + + if ! validate_json "$RESP"; then + echo "[ERROR] Invalid JSON response from $URL" >&2 + break + fi + + if [ -n "$ROUTE_NAME" ]; then + process_route_audit "$RESP" "$ROUTE_NAME" | while IFS= read -r line; do + pretty_print_json "$line" + done + else + process_empty_paths_audit "$RESP" | while IFS= read -r line; do + pretty_print_json "$line" + done + fi + + NEXT=$(get_next_page "$RESP") + done + + log "$(date +%s)" "Audit query completed." +} + +main +``` + +Note: Replace the `BASE` and `PASSWORD` in the script with the Admin API endpoint and the RBAC password/token. + +### Setup instructions + +1. Save the script as `kong-audit-query.sh`. + +1. Make it executable: + + ```bash + chmod +x kong-audit-query.sh + ``` + +1. Ensure Kong audit logging is enabled in your Kong configuration: + + ```bash + KONG_AUDIT_LOG: "on" + ``` + + Without this, Kong will not emit any audit log events, and the script will return empty results. + +### Usage + +- To check who created a specific route: + + ```bash + ./kong-audit-query.sh + ``` + +- To scan all routes with missing/empty paths: + + ```bash + ./kong-audit-query.sh + ``` diff --git a/app/_support/how-to-configure-consumer-groups-rate-limiting-policy-after-upgrading-to-latest-kong-version.md b/app/_support/how-to-configure-consumer-groups-rate-limiting-policy-after-upgrading-to-latest-kong-version.md new file mode 100644 index 0000000000..76c2b068f7 --- /dev/null +++ b/app/_support/how-to-configure-consumer-groups-rate-limiting-policy-after-upgrading-to-latest-kong-version.md @@ -0,0 +1,63 @@ +--- +title: Configuring Consumer Groups Rate Limiting policy after upgrading to the latest Kong version +content_type: support +description: After upgrading Kong gateway to latest version, the method for configuring the Rate Limiting Policy / Rate limiting plugin for Consumer Groups has changed. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I configure the Consumer Groups Rate Limiting policy after upgrading to the latest Kong version? + a: | + After upgrading (for example, from `3.4.3.13` to `3.10.0.x`), the Consumer Group policy is + deprecated and policies are now applied through the Plugins tab instead of the Policy tab. Use + the Admin API endpoint `/consumer_groups/{group_name_or_id}/overrides/plugins/rate-limiting-advanced` + to configure the policy, then create a Rate Limiting plugin instance with the + `enforce_consumer_groups` option enabled and add your Consumer Group to the list. Note that if + Dynamic Ordering is configured anywhere in the workspace, consumer/consumer group scoped Rate + Limiting plugins may not work. +related_resources: + - text: Known limitations of dynamic plugin ordering + url: /gateway/entities/plugin/#known-limitations-of-dynamic-plugin-ordering + - text: Kong Admin API documentation + url: /api/gateway/admin-ee/#/operations/put-consumer_groups-group_name_or_id-overrides-plugins-rate-limiting-advanced +--- + +## Overview + +How to configure Consumer Groups Rate Limiting policy after upgrading from Kong version 3.4.3.13 to 3.10.0.x or in latest version where consumer group policy is deprecated. + +## Steps + +After upgrading Kong gateway to latest version, the method for configuring the Rate Limiting Policy / Rate limiting plugin for Consumer Groups has changed. Previously, the policy could be configured directly in the Consumer Group's policy section. However, in latest versions, the configuration process has been updated, and policies now need to be applied through the Plugins tab. + +Also, it's important to note that if Dynamic Ordering is configured anywhere in the workspace, there are limitations with executing the Rate Limiting plugin at the Consumer or Consumer Group level and the plugins may not work if they are consumer/consumer group scoped.  There is an improvement to support Dynamic Plugin Ordering for Consumer/Consumer Group scoped plugins. + +Consumer groups -> Policy tab in older versions: + +Consumer Groups --> Policy Tab is disabled + +Consumer Groups --> Configure Plugins directly + +NOTE: The consumer groups policy configuration still works in latest versions. It is not removed as we hope customers will organically move to new model over the time. + +To correctly configure the Rate Limiting Advanced plugin for a Consumer Group in latest Kong version, follow these steps: + +1. Use the Admin API to configure the policy for your newly created Consumer Groups (In above example, I have created "free-150tps" consumer group and we can see that there is no option to configure policy in latest version. + + The relevant API endpoint is `/consumer_groups/{group_name_or_id}/overrides/plugins/rate-limiting-advanced`. + + For detailed API documentation, refer to the Kong Admin API documentation. + + Example: + + ```bash + curl -X PUT https://adminapi:8001/consumer_groups//overrides/plugins/rate-limiting-advanced \ + -H 'content-type: application/json' \ + -d '{"config":{"limit": [10], "window_size":[10], "window_type":"sliding"}}' + ``` + +2. Create a Rate Limiting plugin instance and enable the `enforce_consumer_groups` option. Add the name of your Consumer Group "free-150tps" to the list. + +3. Confirm that the Rate Limiting works as per the policy configured. This setup should now correctly enforce the specified TPS (Transactions Per Second) limit for any consumer added to this Consumer Group "free-150tps". diff --git a/app/_support/how-to-define-custom-rbac-endpoint-permissions-across-all-workspaces.md b/app/_support/how-to-define-custom-rbac-endpoint-permissions-across-all-workspaces.md new file mode 100644 index 0000000000..bf4a665116 --- /dev/null +++ b/app/_support/how-to-define-custom-rbac-endpoint-permissions-across-all-workspaces.md @@ -0,0 +1,56 @@ +--- +title: How to define RBAC endpoint permissions across all workspaces +content_type: support +description: Because Kong Manager omits the workspace from RBAC endpoint permission requests, call the Admin API directly with the workspace key set to * to apply a custom endpoint permission across all workspaces. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I apply custom endpoint permissions for roles to all workspaces akin to the bootstrapped default workspace permissions? + a: | + Kong Manager does not send the workspace details in the JSON body when creating or updating + RBAC endpoint permissions, so permissions apply only to the currently selected workspace. + To apply a permission to all workspaces, call the `admin-api` directly and add a `workspace` + key with a value of `*` to the JSON payload (for example, + `{"endpoint":"*","actions":"create,read,update,delete","negative":false,"workspace":"*"}`). + Regardless of the workspace in the request URL, the permission then applies to all workspaces. +related_resources: [] +--- + +## Overview + +This article describes how to apply custom RBAC endpoint permissions for roles across all workspaces. + +## Steps + +Endpoint permissions can be assigned to a single workspace or all workspaces, however Kong Manager does not send the workspace details in the JSON body of the request to create / update the endpoint permissions. + +The effect of this is that permissions that are created using Kong Manager will use the currently selected workspace in Kong Manager, as the workspace to be applied to. + +To circumvent the Kong Manager behavior, the `admin-api` can be called directly and the permissions can be set to apply to all workspaces. + +Example request to create role permissions using Kong Manager: + +```bash +curl -X POST 'https://admin.local.docker:8444/default/rbac/roles/SuperAdmin/endpoints' \ + -H 'Content-Type: application/json' \ + -H 'Kong-Admin-Token: admin' \ + --data-raw '{"endpoint":"*","actions":"create,read,update,delete","negative":false}' +``` + +To apply to all workspaces, add the `workspace` key with a value of `*` to the JSON payload: + +```bash +curl -X POST 'https://admin.local.docker:8444/default/rbac/roles/SuperAdmin/endpoints' \ + -H 'Kong-Admin-Token: admin' \ + -H 'Content-Type: application/json' \ + --data-raw '{"endpoint":"*","actions":"create,read,update,delete","negative":false,"workspace":"*"}' +``` + +Regardless of the workspace used in the request URL, the applicable workspace for the permission will be all workspaces `*`. + +Security Note: This works regardless of which workspace the role was assigned to the user in and therefore could propose a security risk if used incorrectly. + +For example, the `workspace-read-only` role that is auto-created upon new workspace instantiation, could have its permissions modified in this way so that all admins that previously only had read only access to the single workspace, can now read from all endpoints in all workspaces. diff --git a/app/_support/how-to-enable-ocsp-stapling-in-kong-and-handle-ocsp-server-downtime.md b/app/_support/how-to-enable-ocsp-stapling-in-kong-and-handle-ocsp-server-downtime.md new file mode 100644 index 0000000000..a0391d27e3 --- /dev/null +++ b/app/_support/how-to-enable-ocsp-stapling-in-kong-and-handle-ocsp-server-downtime.md @@ -0,0 +1,43 @@ +--- +title: Enable OCSP stapling in Kong and handle OCSP server downtime +content_type: support +description: Configure OCSP Stapling in Kong with the relevant NGINX environment variables, and understand how Kong behaves when the OCSP responder is unavailable. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How to Enable OCSP Stapling in Kong and Handle OCSP Server Downtime? + a: | + Enable OCSP Stapling by setting `KONG_NGINX_PROXY_SSL_STAPLING=on` and + `KONG_NGINX_PROXY_SSL_STAPLING_VERIFY=on` in your Kong configuration, and set + `KONG_NGINX_PROXY_SSL_TRUSTED_CERTIFICATE` to include the complete certificate chain of the + issuer so the OCSP response is validated correctly. If Kong cannot retrieve the OCSP response + from the responder, it will not staple the response, but the TLS handshake will still succeed + and the connection will be established, so your service remains accessible. +related_resources: [] +--- + +## Overview + +This article describes how to enable OCSP Stapling in Kong and how Kong handles OCSP server downtime. + +## Steps + +Enabling OCSP (Online Certificate Status Protocol) Stapling in Kong is crucial for enhancing the security of your server-client communications. OCSP Stapling allows the server to provide a timestamped OCSP response from the Certificate Authority to the client during the TLS handshake, proving the certificate's validity. This process reduces the client's need to contact the CA, improving privacy and performance. + +To configure OCSP Stapling in Kong, you need to set specific environment variables in your Kong configuration. These variables are: + +```shell +KONG_NGINX_PROXY_SSL_STAPLING=on +KONG_NGINX_PROXY_SSL_STAPLING_VERIFY=on +``` + +Additionally, to ensure the OCSP response is validated correctly, you should include the complete certificate chain of the issuer for the server certificate whose OCSP response we are validating. This can be achieved by setting the following variables: + +```shell +KONG_NGINX_PROXY_SSL_TRUSTED_CERTIFICATE= +``` + +It's important to note that if Kong is unable to retrieve the OCSP response from the responder, it will not staple the response in the TLS handshake. However, the handshake will still succeed, and the connection will be established as expected. This behavior ensures that your service remains accessible even if the OCSP server is temporarily unavailable. diff --git a/app/_support/how-to-enforce-strict-json-body-schemas-when-using-the-oas-validation-plugin.md b/app/_support/how-to-enforce-strict-json-body-schemas-when-using-the-oas-validation-plugin.md new file mode 100644 index 0000000000..a26f411068 --- /dev/null +++ b/app/_support/how-to-enforce-strict-json-body-schemas-when-using-the-oas-validation-plugin.md @@ -0,0 +1,79 @@ +--- +title: How to enforce strict JSON body schemas when using the OAS validation plugin +content_type: support +description: Set the `additionalProperties` field to `false` in your OAS body schema so the OAS validation plugin rejects request bodies containing keys not defined in the schema. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I enforce strict JSON body schemas when using the OAS validation plugin? + a: | + Set the `additionalProperties` field to `false` in your OAS body schema definition. + When `additionalProperties` is `true` or omitted, the plugin allows any extra JSON keys as + long as the required keys are present (a fail-open scenario). When set to `false`, a request + body containing any key not explicitly defined in the schema fails validation (the fail-safe + scenario). +related_resources: [] +--- + +## Overview + +This article describes how to enforce strict JSON body schemas when using the OAS validation plugin. + +## Steps + +When using OAS-Validation plugin to validate a request against a particular OAS specification, a boolean field called `additionalProperties` can be set to either `true` or `false` in the body schema definition. + +Given the following JSON body in the request: + +```json +{ + "value": "test", + "blah": "invalid" +} +``` + +And this OAS schema for a JSON body: + +```yaml + schemas: + Echo: + type: object + required: + - value + properties: + value: + type: string + optionalValue: + type: + - "null" + - string +``` + +When `additionalProperties` is set to `true` or omitted entirely from the schema, the plugin will allow any number of JSON keys in the request body, as long as there is a `value` key present. + +The request body above will pass validation as this is a fail-open scenario. + +When `additionalProperties` is set to `false`: + +```yaml + schemas: + Echo: + type: object + required: + - value + properties: + value: + type: string + optionalValue: + type: + - "null" + - string + additionalProperties: false +``` + +The additional `blah` key in the request body is not explicitly defined in the schema above, and `additionalProperties` is not disallowed, therefore the request will fail validation. + +This is the fail-safe scenario. diff --git a/app/_support/how-to-get-the-exit-transformer-to-return-a-null-in-the-response-body.md b/app/_support/how-to-get-the-exit-transformer-to-return-a-null-in-the-response-body.md new file mode 100644 index 0000000000..b170806aba --- /dev/null +++ b/app/_support/how-to-get-the-exit-transformer-to-return-a-null-in-the-response-body.md @@ -0,0 +1,90 @@ +--- +title: How to get the Exit Transformer to return a null in the response body +content_type: support +description: Configure the Exit Transformer plugin to return a field with a null value in the response body by updating the untrusted Lua environment variables and using cjson.null. +products: + - gateway +works_on: + - on-prem + - konnect +related_resources: + - text: "Disclaimer - review the `untrusted_lua` configuration reference before updating this on your environment" + url: /gateway/configuration/#untrusted-lua +tldr: + q: How do I configure the Exit Transformer plugin to return a field with a null value in the response body? + a: | + Update the untrusted Lua environment variables to allow `cjson`: + set `KONG_UNTRUSTED_LUA: "sandbox"` and `KONG_UNTRUSTED_LUA_SANDBOX_REQUIRES: "cjson"` + (review the `untrusted_lua` configuration reference first). Then, in your `exit-transformer` + function, assign the field using `(require "cjson").null` instead of a bare `null`. When the + plugin runs, the field is returned with a `null` value in the response body. +--- + +## Overview + +When configuring the `exit-transformer` to respond with an updated response body, we notice that any field returned with `null` as its value is not being returned in the response body. + +For example: + +```lua +return function(status, body, headers) + if status > 399 then + body = { hello = "test", goodbye = null} + end + + return status, body, headers +end +``` + +How can we resolve this and have a `null` returned as its value? + +## Steps + +To accomplish this we need to make a few updates to the environment variables and then change the Lua slightly to require `cjson`. + +First, let's update the environment variables below: + +```yaml +KONG_UNTRUSTED_LUA: "sandbox" +KONG_UNTRUSTED_LUA_SANDBOX_REQUIRES: "cjson" +``` + +Please review the untrusted Lua configuration reference before updating this on your environment. + +Next, let's add the following to our exit transformer (to do this I saved this to a file called `bodyNull.lua`): + +```lua +return function(status, body, headers) + if status > 399 then + body = { hello = (require "cjson").null } + end + + return status, body, headers +end +``` + +Then we need to create the exit transformer plugin: + +```bash +curl --location --request POST 'http://localhost:8001/plugins' \ +--header 'Kong-Admin-Token: ' \ +--form 'config.functions=@"//bodyNull.lua"' \ +--form 'name="exit-transformer"' +``` + +For this specific test I created a `request-termination` plugin set to a status of 400 to trigger the Lua from the exit transformer. + +```bash +curl -X POST http://localhost:8001/plugins \ + --header 'Kong-Admin-Token: ' \ + --data "name=request-termination" \ + --data "config.status_code=400" \ +``` + +Now if we run a request we can see that the field `hello` is returned with a null value. + +```json +{ + "hello": null +} +``` diff --git a/app/_support/how-to-get-the-time-when-kong-send-request-to-upstream-and-the-time-kong-get-whole-response-from-upstream.md b/app/_support/how-to-get-the-time-when-kong-send-request-to-upstream-and-the-time-kong-get-whole-response-from-upstream.md new file mode 100644 index 0000000000..e5e1b86d8a --- /dev/null +++ b/app/_support/how-to-get-the-time-when-kong-send-request-to-upstream-and-the-time-kong-get-whole-response-from-upstream.md @@ -0,0 +1,93 @@ +--- +title: Getting the time when Kong sends a request to upstream and receives the whole response +content_type: support +description: Use the pre-function and post-function plugins to log the time Kong sends a request to upstream and the time Kong receives the whole response. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I get the time when Kong sends a request to upstream and the time Kong gets the whole response from upstream? + a: | + Use the `post-function` and `pre-function` plugins. The time Kong sends a request to upstream can be + obtained at the end of the access phase with `post-function` (which runs after all other plugins), and + the time Kong gets the whole response can be obtained at the beginning of the log phase with `pre-function` + (which runs before all other plugins). First set `untrusted_lua_sandbox_requires = socket` (or the env var + `KONG_UNTRUSTED_LUA_SANDBOX_REQUIRES = socket` in a container) so the plugins can use the socket package, + then enable the plugins on the target route/service to call `socket.gettime()*1000` and log the times, + which appear in the Kong error log. +related_resources: + - text: plugin execution order + url: /gateway/entities/plugin/#dynamic-plugin-ordering + - text: pre-function plugin configuration + url: /plugins/pre-function/ + - text: post-function plugin configuration + url: /plugins/post-function/ + - text: "`untrusted_lua_sandbox_requires`" + url: /gateway/configuration/#untrusted-lua-sandbox-requires +--- + +## Overview + +When I send a request to Kong, how can I get the following times? + +(1) The time when Kong sends a request to upstream + +(2) The time Kong gets the whole response from upstream + +## Steps + +The above is the flow of how Kong handles requests. (1) could be obtained at the end of the access phase, (2) could be obtained at the beginning of the log phase. + +The `post-function` plugin runs after all the other plugins, so we could use it to get (1) in the access phase. + +The `pre-function` plugin runs before all the other plugins, so we could use it to get (2) in the log phase. + +We could implement it by following the steps below: + +1. Set the parameter below for Kong to use the socket package in the `pre-function`/`post-function` plugins: + + ```bash + untrusted_lua_sandbox_requires = socket + ``` + + If you run Kong with a container, set the env var below instead: + + ```bash + KONG_UNTRUSTED_LUA_SANDBOX_REQUIRES = socket + ``` + +2. Enable the `post-function` plugin below on the target route/service object to log (1): + + ```yaml + plugins: + - name: post-function + config: + access: + - |- + local socket = require "socket" + local s_time = socket.gettime()*1000 + kong.log('sending time(ms): ', s_time) + enabled: true + ``` + +3. Enable the `pre-function` plugin below on the target route/service object to log (2): + + ```yaml + plugins: + - name: pre-function + config: + log: + - |- + local socket = require "socket" + local r_time = socket.gettime()*1000 + kong.log('receiving time(ms): ', r_time) + enabled: true + ``` + +4. Send a request to Kong again, then you will be able to get the times below (ms) from the Kong error log: + + (1) The time when Kong sends a request to upstream + + (2) The time Kong gets the whole response from upstream diff --git a/app/_support/how-to-log-the-pod-name-of-the-kong-node-that-processed-a-request-using-a-log-plugin.md b/app/_support/how-to-log-the-pod-name-of-the-kong-node-that-processed-a-request-using-a-log-plugin.md new file mode 100644 index 0000000000..4e0c1c054a --- /dev/null +++ b/app/_support/how-to-log-the-pod-name-of-the-kong-node-that-processed-a-request-using-a-log-plugin.md @@ -0,0 +1,67 @@ +--- +title: How to log the pod name of the Kong node that processed a request when using a Kong log plugin +content_type: support +description: Add the data plane pod name to Kong log plugin output by exposing the container HOSTNAME variable through the config.custom_fields_by_lua property. +products: + - kic +works_on: + - on-prem + - konnect +tldr: + q: How do I add the data plane pod name to the log produced by a Kong log plugin to identify which Kong pod processed the request? + a: | + In a Kubernetes environment, the container exposes a `HOSTNAME` variable set to the pod the + container runs on. Using the Environment Variables Vault Secret Management Backend, add this + variable to the log plugin output by adding a new field to the plugin's + `config.custom_fields_by_lua` property, for example + `x-pod-name: return kong.vault.get("{vault://env/hostname}")`. This can be set via Kong + Manager, an Admin API call, or declarative config. +related_resources: [] +--- + +## Overview + +This article describes how to add the data plane pod name to the log produced by a Kong log plugin to make it easier to identify a specific Kong pod that processed the request. + +## Steps + +In a kubernetes environment a container should have a `HOSTNAME` variable which is set to the pod the container is deployed on. + +With the Environment Variables Vault Secret Management Backend it is easy to add this `HOSTNAME` variable to the log plugin output by adding a new field to the `config.custom_fields_by_lua` property of your log plugin as in these examples: + +A). via Kong Manager: + +B). Via an admin API call using the file-log plugin as an example: + +```bash +curl -H "kong-admin-token: " -X POST //plugins \ +-H 'content-type: application/json' \ +-d @payload.json +``` + +where `payload.json` has the following content: + +```json +{ + "name":"file-log", + "config":{ + "custom_fields_by_lua":{ + "x-pod-name":"return kong.vault.get(\"{vault://env/hostname}\")" + }, + "path":"/tmp/file.log" + } +} +``` + +C). Via declarative config: + +```yaml +plugins: +- config: + custom_fields_by_lua: + x-pod-name: return kong.vault.get("{vault://env/hostname}") + path: /tmp/filelog + reopen: false + enabled: true + name: file-log +``` diff --git a/app/_support/how-to-setup-a-maintenance-page-inside-kong-gateway.md b/app/_support/how-to-setup-a-maintenance-page-inside-kong-gateway.md new file mode 100644 index 0000000000..06398ce42b --- /dev/null +++ b/app/_support/how-to-setup-a-maintenance-page-inside-kong-gateway.md @@ -0,0 +1,41 @@ +--- +title: How to set up a maintenance page inside {{site.base_gateway}} +content_type: support +description: "How to display a maintenance page in {{site.base_gateway}} using the Request Termination plugin to return a custom message and content type." +products: + - gateway +works_on: + - on-prem + - konnect +related_resources: + - text: Request Termination plugin + url: /plugins/request-termination/ +tldr: + q: How do I set up a maintenance page in {{site.base_gateway}} to display a message to users when an endpoint is being worked on? + a: | + Use the Request Termination plugin, enabled on the specific route or service (or globally for + all services). Set `config.body` to your maintenance messaging and `config.content_type` to + the desired content type, for example `text/html` to return an HTML page. +--- + +## Overview + +This article describes how to set up a maintenance page in {{site.base_gateway}}. If an endpoint is being worked on, you can enable this page to display a message to users. + +## Steps + +One of the easier ways to accomplish this is by using the Request Termination plugin. + +To begin, create the Request Termination plugin on the specific route/service in question (if all services then you can set the plugin globally). + +`config.body` can be used to add your messaging. + +`config.content_type` can be used to set your desired content type. For example, if you wish to return an HTML page, `text/html`. + +As a sample to demonstrate this we can add the following. + +`config.body`: + +```html +

This page is under maintenance. It will be completed by 3pm PST. Here is a turtle to look at to pass the time.

+``` diff --git a/app/_support/how-to-use-topologyspreadconstraints-to-distribute-pods-evenly.md b/app/_support/how-to-use-topologyspreadconstraints-to-distribute-pods-evenly.md new file mode 100644 index 0000000000..58938d95b8 --- /dev/null +++ b/app/_support/how-to-use-topologyspreadconstraints-to-distribute-pods-evenly.md @@ -0,0 +1,131 @@ +--- +title: How to use topologySpreadConstraints to distribute pods evenly +content_type: support +description: Use topologySpreadConstraints, supported in the Kong Helm charts from version 2.0.0, to evenly distribute Kong pods across your Kubernetes cluster nodes. +products: + - gateway + - kic +works_on: + - on-prem + - konnect +tldr: + q: How do I use topologySpreadConstraints to distribute Kong pods evenly across my Kubernetes cluster nodes? + a: | + `topologySpreadConstraints` is supported in the Kong Helm charts from version `2.0.0`. Add a + `topologySpreadConstraints` block to your Helm values, setting `topologyKey` to a node label shared by + all target nodes (for example `kubernetes.io/hostname` or a custom `zone` label) and `labelSelector.matchLabels` + to match the Kong pods (for example `app.kubernetes.io/instance: kong-enterprise`). The `maxSkew` and + `whenUnsatisfiable` parameters control how rigidly the pods must be spread. +related_resources: + - text: "topologySpreadConstraints support in the Kong Helm charts (CHANGELOG 2.0.0)" + url: https://github.com/Kong/charts/blob/7fb11b7658f48de14d04a6d8155f1b2e1c8dbba8/charts/kong/CHANGELOG.md#200 + - text: "Kubernetes Pod Topology Spread Constraints documentation" + url: https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/ +--- + +## Overview + +How do I use `topologySpreadConstraints` to load balance my pods over my Kubernetes cluster nodes? + +## Steps + +We added support for `topologySpreadConstraints` to our Helm charts from version 2.0.0. + +The following commands are useful when working with `topologySpreadConstraints`: + +```bash +# See which node pods are residing on: +kubectl get pods -n kong -o wide +# Show Kubernetes node information with labels: +kubectl get nodes --show-labels +# Label a node with a particular label: +kubectl label nodes k8s-worker-2 zone=2 +``` + +Cluster node details: + +The labels are used by `topologySpreadConstraints` to determine where various pods should go. + +```text +NAME STATUS ROLES AGE VERSION LABELS +k8s-master-node Ready control-plane 116d v1.24.3 kubernetes.io/hostname=k8s-master-node +k8s-worker-1 Ready 116d v1.24.3 kubernetes.io/hostname=k8s-worker-1,zone=1 +k8s-worker-2 Ready 116d v1.24.3 kubernetes.io/hostname=k8s-worker-2,zone=2 +``` + +Example 1: + +Distribute 2 pods over 3 nodes by applying the following in your Helm values. + +```yaml +replicaCount: 2 +topologySpreadConstraints: + - maxSkew: 1 + topologyKey: kubernetes.io/hostname + whenUnsatisfiable: DoNotSchedule + labelSelector: + matchLabels: + app.kubernetes.io/instance: kong-enterprise +``` + +For a more detailed explanation of what each config option does, review the Kubernetes documentation. + +The important parameters are: + +`topologyKey`: Denotes the pod label used to select underlying hosts for distribution. To use all nodes, set this to a label that all nodes contain, such as `kubernetes.io/hostname`. + +`labelSelector.matchLabels`: Finds matching pods. To balance Kong pods, all the pods have the `app.kubernetes.io/instance: kong-enterprise` label. + +The `maxSkew` and `whenUnsatisfiable` parameters control how rigidly the rules are applied and how evenly the pods must be spread. + +Results: + +Pod and resident node: + +```text +kong-enterprise-kong-785c5d5db8-bqqfx | k8s-master-node | +kong-enterprise-kong-785c5d5db8-pn9z5 | k8s-worker-1 | +``` + +Example 2: + +Distribute 3 pods over 3 nodes by changing `replicaCount` to 3 and upgrading the Helm deployment. + +Results: + +Pod and resident node: + +```text +kong-enterprise-kong-785c5d5db8-bqqfx | k8s-master-node | +kong-enterprise-kong-785c5d5db8-pn9z5 | k8s-worker-1 | +kong-enterprise-kong-785c5d5db8-8gq6s | k8s-worker-2 | +``` + +Example 3: + +Distribute 3 pods over 2 nodes using the zone label. + +```yaml +replicaCount: 3 +topologySpreadConstraints: + - maxSkew: 1 + topologyKey: zone + whenUnsatisfiable: DoNotSchedule + labelSelector: + matchLabels: + app.kubernetes.io/instance: kong-enterprise +``` + +Results: + +Pod and resident node: + +```text +kong-enterprise-kong-579f9678bd-mm5mj | k8s-worker-2 | +kong-enterprise-kong-579f9678bd-p788t | k8s-worker-1 | +kong-enterprise-kong-579f9678bd-whqrg | k8s-worker-2 | +``` + +Conclusion: + +This is a fairly simple load balancing scenario, however `topologySpreadConstraints` have greater flexibility than the previous affinity rules allowed. diff --git a/app/_support/how-to-view-the-contents-of-shared-dictionaries.md b/app/_support/how-to-view-the-contents-of-shared-dictionaries.md new file mode 100644 index 0000000000..75bac656e0 --- /dev/null +++ b/app/_support/how-to-view-the-contents-of-shared-dictionaries.md @@ -0,0 +1,79 @@ +--- +title: How to view the contents of shared dictionaries +content_type: support +description: Use a pre-function plugin to report the contents of the SHM LRU dictionaries in Kong. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I view the contents of the shared dictionaries in Kong? + a: | + Use a `pre-function` plugin to report the contents of the SHM LRU dictionaries. The Lua code calls + `ngx.shared["kong_rate_limiting_counters"]:get_keys(2000)` to fetch the first 2000 keys from the dictionary + and logs each key with `kong.log.err`. The retrieved keys then appear in the Kong error log. +related_resources: [] +--- + +## Overview + +How do you see the contents of the shared dictionaries? + +## Steps + +Use a `pre-function` plugin to report the contents of the SHM LRU dictionaries in Kong. + +The following examples show `pre-function` plugin code that inspects the `kong_rate_limiting_counters` dictionary for the first 2000 keys. + +The Lua code: + +```lua +kong.log.err("PRE FUNCTION EXECUTED") +local keys, err = ngx.shared["kong_rate_limiting_counters"]:get_keys(2000) +if not keys then + kong.log.err("cannot fetch keys in shared dict! ", err) +else + kong.log.err("keys retrieved: ", #keys) + for _, key in ipairs(keys) do + kong.log.err(key) + end +end +kong.log.err("PRE FUNCTION ENDED") +``` + +K8s plugin code: + +```yaml +apiVersion: configuration.konghq.com/v1 +kind: KongPlugin +metadata: + name: capture-kong-locks-pre-function +config: + access: + - | + kong.log.err("PRE FUNCTION EXECUTED") + local keys, err = ngx.shared["kong_rate_limiting_counters"]:get_keys(2000) + if not keys then + kong.log.err("cannot fetch keys in shared dict! ", err) + else + kong.log.err("keys retrieved: ", #keys) + for _, key in ipairs(keys) do + kong.log.err(key) + end + end + kong.log.err("PRE FUNCTION ENDED") +plugin: pre-function +``` + +The result in the Kong log: + +```text +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:6 [pre-function] keys retrieved: 4, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:8 [pre-function] jLcEZU3Fr004xsxam1jELW07UfLf8D7p|1675669469|1|172.18.0.1|diff, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2209#0: *991 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:1 [pre-function] PRE FUNCTION EXECUTED, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:8 [pre-function] jLcEZU3Fr004xsxam1jELW07UfLf8D7p|1675669469|1|172.18.0.1|sync, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:8 [pre-function] jLcEZU3Fr004xsxam1jELW07UfLf8D7p|1675669470|1|172.18.0.1|diff, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:8 [pre-function] jLcEZU3Fr004xsxam1jELW07UfLf8D7p|1675669470|1|172.18.0.1|sync, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +2023/02/06 07:44:30 [error] 2207#0: *990 [kong] [string "kong.log.err("PRE FUNCTION EXECUTED")..."]:11 [pre-function] PRE FUNCTION ENDED, client: 172.18.0.1, server: kong, request: "GET /bin HTTP/1.1", host: "localhost:8000" +``` diff --git a/app/_support/kic-faqs.md b/app/_support/kic-faqs.md new file mode 100644 index 0000000000..02aedb38de --- /dev/null +++ b/app/_support/kic-faqs.md @@ -0,0 +1,67 @@ +--- +title: "{{site.kic_product_name}} FAQs" +content_type: support +description: "Answers to common questions about {{site.kic_product_name}}, including gathering metrics and creating additional Kubernetes resources with the Helm chart." +products: + - kic +works_on: + - on-prem + - konnect +tldr: + q: Where can I find answers to common {{site.kic_product_name}} questions? + a: | + This page collects frequently asked questions about {{site.kic_product_name}}: how to gather + metrics from the controller, and how to create additional Kubernetes resources when deploying + with the Kong Helm chart. +related_resources: [] +--- + +## Observability + +### How do I gather {{site.kic_product_name}} metrics? + +By default, the {{site.kic_product_name}} exposes a `/metrics` endpoint over port `10255`. + +1. Port forward the controller pod on port `10255`: + + ```bash + kubectl port-forward -n kong 10255:10255 + ``` + +2. Query the metrics endpoint: + + ```bash + curl http://localhost:10255/metrics + ``` + +This lists all available metrics along with an explanation of their meanings. To export these metrics to Prometheus, see the [Prometheus and Grafana documentation](/kubernetes-ingress-controller/observability/prometheus/). + +## Helm chart + +### How do I create additional Kubernetes resources with the Helm chart? + +The Kong Helm charts support an array parameter, `extraObjects`, that you can use to create additional Kubernetes resources on deployment. Each new resource is an entry in the array. For example, the following array contains two manifests that create Kubernetes `Secret` objects: + +```yaml +image: + repository: kong/kong-gateway + +extraObjects: + - apiVersion: v1 + data: + kongCredType: YmFzaWMtYXV0aA== + password: a29uZw== + username: Z3J1YmVy + kind: Secret + metadata: + name: basic-auth + - apiVersion: v1 + data: + password: a29uZw== + username: bWNjbGFuZQo= + kind: Secret + metadata: + name: uid-pw +``` + +For the full list of chart options, see the [Kong Helm chart parameters](https://github.com/Kong/charts/tree/main/charts/kong#general-parameters). diff --git a/app/_support/kong-gateway-error-connecting-kong-to-postgresql-failed-to-retrieve-postgresql-server-version-num-fatal-no-pg-hba-conf-entry-ssl-off.md b/app/_support/kong-gateway-error-connecting-kong-to-postgresql-failed-to-retrieve-postgresql-server-version-num-fatal-no-pg-hba-conf-entry-ssl-off.md new file mode 100644 index 0000000000..ecd0dbee28 --- /dev/null +++ b/app/_support/kong-gateway-error-connecting-kong-to-postgresql-failed-to-retrieve-postgresql-server-version-num-fatal-no-pg-hba-conf-entry-ssl-off.md @@ -0,0 +1,50 @@ +--- +title: "{{site.base_gateway}}: Error connecting Kong to PostgreSQL: `Failed to retrieve PostgreSQL server_version_num: FATAL: No pg_hba.conf entry` & \"ssl off\" or \"no encryption\"" +content_type: support +description: This error occurs when Kong connects to the PostgreSQL DB server without SSL, but the server requires SSL connections. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why do I get `Failed to retrieve PostgreSQL server_version_num: FATAL: no pg_hba.conf entry` with \"SSL off\" or \"no encryption\" when connecting Kong to PostgreSQL?" + a: | + This error occurs when Kong connects to the PostgreSQL server without SSL, but the server requires SSL + connections. The recommended fix is to configure Kong to connect using SSL by setting `pg_ssl` to `on`, + and if needed setting `pg_ssl_version` to `tlsv1_2` or the version your server requires. Alternatively, + you can disable the SSL-only requirement on the PostgreSQL server to allow non-SSL connections, though + this reduces security and is discouraged. +related_resources: [] +--- + +## Problem + +You see this error when trying to connect Kong to your PostgreSQL database (DB) server. + +```text +Error: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: FATAL: no pg_hba.conf entry for host "", user "", database "kong", SSL off +``` + +Another variation of the error looks like this: + +```text +Error: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: FATAL: no pg_hba.conf entry for host "", user "", database "kong", no encryption" +``` + +## Cause + +This error occurs when Kong connects to the PostgreSQL DB server without SSL, but the server requires SSL connections. + +## Solution + +To resolve this error, consider the following options: + +1. We recommend that you configure Kong to connect to the PostgreSQL DB server using SSL. The PostgreSQL configuration properties are documented in the documentation on PostgreSQL settings. + + Set the `pg_ssl` property to `on`, and if needed, set `pg_ssl_version` to `tlsv1_2` or the version required by your PostgreSQL server. Consult the documentation for detailed setup instructions. + +2. Disable the SSL-only requirement on the PostgreSQL server to allow non-SSL connections. + + {:.warning} + > Disabling the SSL-only requirement reduces security. We do not recommend this option. diff --git a/app/_support/kong-gateway-fixing-invalid-nginx-configuration-error-when-enabling-nginx-proxy-brotli-in-kong-conf.md b/app/_support/kong-gateway-fixing-invalid-nginx-configuration-error-when-enabling-nginx-proxy-brotli-in-kong-conf.md new file mode 100644 index 0000000000..d7a313df98 --- /dev/null +++ b/app/_support/kong-gateway-fixing-invalid-nginx-configuration-error-when-enabling-nginx-proxy-brotli-in-kong-conf.md @@ -0,0 +1,40 @@ +--- +title: "{{site.base_gateway}}: Fixing the 'Invalid Nginx Configuration' error when enabling `nginx_proxy_brotli` in kong.conf" +content_type: support +description: "When configuring `nginx_proxy_brotli` in your `kong.conf`, you might encounter an error that prevents successful deployment, stating that the nginx configuration is invalid." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: Why do I get an "Invalid Nginx Configuration" error when enabling `nginx_proxy_brotli` in kong.conf? + a: | + This error is typically caused by incorrectly quoting the MIME type values in the + `nginx_proxy_brotli_types` directive. To resolve it, remove the quotes around the MIME type values so + the types are listed unquoted, for example + `nginx_proxy_brotli_types = text/plain text/css application/json ...`. +related_resources: [] +--- + +## Problem + +When configuring `nginx_proxy_brotli` in `kong.conf`, you might encounter an error that prevents successful deployment, stating that the nginx configuration is invalid. + +## Cause + +This issue is typically related to how the `nginx_proxy_brotli_types` directive is formatted in the configuration file. Incorrectly quoting the MIME type values can lead to nginx configuration errors. + +## Solution + +To resolve this issue, adjust the `nginx_proxy_brotli_types` directive in your `kong.conf` file by removing the quotes around the MIME type values. + +Here is an example of how to correctly format the `nginx_proxy_brotli_types` directive in your `kong.conf`: + +```bash +nginx_proxy_brotli = on +nginx_proxy_brotli_comp_level = 5 +nginx_proxy_brotli_types = text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js +``` + +Notice that the MIME types are not enclosed in quotes. This format should prevent the nginx configuration error. diff --git a/app/_support/kong-gateway-how-can-i-ensure-that-a-request-forwarded-through-a-kong-post-function-plugin-retains-the-cookie-passed-from-the-browser.md b/app/_support/kong-gateway-how-can-i-ensure-that-a-request-forwarded-through-a-kong-post-function-plugin-retains-the-cookie-passed-from-the-browser.md new file mode 100644 index 0000000000..b402ba9e57 --- /dev/null +++ b/app/_support/kong-gateway-how-can-i-ensure-that-a-request-forwarded-through-a-kong-post-function-plugin-retains-the-cookie-passed-from-the-browser.md @@ -0,0 +1,60 @@ +--- +title: "{{site.base_gateway}}: Retaining the browser cookie when forwarding a request through a post-function plugin" +content_type: support +description: When a request is forwarded through a Kong post-function plugin, the browser Cookie may not be retained; use pre-function and post-function plugins with kong.ctx.shared to capture and forward the Cookie header. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "{{site.base_gateway}}: How can I ensure that a request forwarded through a Kong post-function plugin retains the Cookie passed from the browser?" + a: | + The `Cookie` header is not retained because the request handling does not propagate it to the upstream + service. Use a `pre-function` plugin to capture the header and store it in shared context with + `kong.ctx.shared.cookie = kong.request.get_header("Cookie")`, then use a `post-function` plugin to set it + on the upstream request with `kong.service.request.set_header("Cookie", kong.ctx.shared.cookie)`. Because + `kong.ctx.shared` is specific to the current request, Cookies from concurrent requests are handled + without interference. +related_resources: [] +--- + +## Problem + +When a request is forwarded through a Kong `post-function` plugin, the `Cookie` passed from the browser is not retained, leading to unauthorized errors due to the missing cookie information. + +## Cause + +This problem can arise when the request handling does not correctly propagate the `Cookie` header to the upstream service. + +## Solution + +You can use a combination of `pre-function` and `post-function` plugins to capture and forward the `Cookie` header correctly. To implement this solution, follow these steps: + +1. Use the `pre-function` plugin to store the `Cookie` in a shared context (`kong.ctx.shared`): + + In the `pre-function` plugin, capture the `Cookie` header from the incoming request and store it in the shared context. This makes the `Cookie` available across different phases of the request processing. + + ```lua + kong.ctx.shared.cookie = kong.request.get_header("Cookie") + ``` + + Or, use a specific cookie name rather than the entire cookie header: + + ```lua + kong.ctx.shared.session_cookie = ngx.var.cookie_session + ``` + +2. Retrieve the `Cookie` from the shared context in the `post-function` plugin and set it in the request to the upstream service: + + In the `post-function` plugin, retrieve the `Cookie` from the shared context and set it as a header in the request being forwarded to the upstream service. + + ```lua + kong.service.request.set_header("Cookie", kong.ctx.shared.cookie) + ``` + +This approach ensures that the `Cookie` passed from the browser is retained and forwarded correctly through the `post-function` plugin, allowing the request to be authenticated and authorized by the upstream service. + +The shared context (`kong.ctx.shared`) is specific to the current request, which ensures that Cookies from concurrent requests are handled correctly without interference. + +This code snippet is a basic example and requires review before you implement it in any environment. Do not use this code without thorough testing. Depending on your specific requirements and setup, you might need to add additional error handling or logic to suit your needs. diff --git a/app/_support/kong-gateway-install-custom-lua-library-manually-not-using-luarocks.md b/app/_support/kong-gateway-install-custom-lua-library-manually-not-using-luarocks.md new file mode 100644 index 0000000000..72bc0005a4 --- /dev/null +++ b/app/_support/kong-gateway-install-custom-lua-library-manually-not-using-luarocks.md @@ -0,0 +1,51 @@ +--- +title: "{{site.base_gateway}}: Install custom Lua library manually (not using `luarocks`)" +content_type: support +description: The custom library would need to be installed on Kong for the plugin to be able to reference the library. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: How do I install a custom Lua library for a custom plugin manually, without using LuaRocks or a custom image? + a: | + The custom library must be installed on Kong so the plugin can reference it. Map the library into + `/usr/local/share/lua/5.1/` using a Docker volume, for example: + + ```yaml + volumes: + - ./sampleLibrary/:/usr/local/share/lua/5.1/sampleLibrary + ``` + + The referenced `custom.lua` file must sit directly inside `/usr/local/share/lua/5.1/sampleLibrary`; + if it is in a subdirectory it will fail to load. +related_resources: [] +--- + +## Problem + +We would like to deploy a custom plugin that uses a custom library without installing through a custom image or through `luarocks`. + +We are receiving the error: + +``` +[error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/tools/utils.lua:706: error loading module ‘kong.plugins.samplePlugin.handler’: ...l/share/lua/5.1/kong/plugins/samplePlugin/handler.lua:2: module ‘custom.lua’ not found:No LuaRocks module found for custom.lua +``` + +## Solution + +The custom library would need to be installed on Kong for the plugin to be able to reference the library. + +We can create a volume on our docker compose file to map the library over to the following location: + +`/usr/local/share/lua/5.1/` + +Docker compose example: + +```yaml +volumes: + - ./sampleLibrary/:/usr/local/share/lua/5.1/sampleLibrary +``` + +The `custom.lua` file that is being referenced needs to be inside the `/usr/local/share/lua/5.1/sampleLibrary` folder. If it is in a subdirectory it will continue to fail to load. diff --git a/app/_support/kong-gateway-overridingduplication-of-correlation-id-header.md b/app/_support/kong-gateway-overridingduplication-of-correlation-id-header.md new file mode 100644 index 0000000000..fc6b1524ae --- /dev/null +++ b/app/_support/kong-gateway-overridingduplication-of-correlation-id-header.md @@ -0,0 +1,61 @@ +--- +title: "{{site.base_gateway}}: Overriding and duplication of the Correlation ID header" +content_type: support +description: "Explains why the Correlation ID plugin is bypassed when a request already supplies the Kong-Request-Id header, and the resulting override and duplication concerns." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: Why does the Correlation ID plugin not generate a new UUID when a request already includes the Kong-Request-Id header? + a: | + When a request already supplies the `Kong-Request-Id` header, the Correlation ID plugin is ignored + entirely and the provided value is passed through unchanged rather than being overwritten with a new + UUID. This means a client can override the generated UUID, and there is potential for clients to + mistake a passed header value for a duplicate UUID. +related_resources: [] +--- + +## Overriding and duplication of the Correlation ID header + +How could we manipulate or override the UUID header value generated by the Correlation ID plugin? + +Suppose you have a service and route with the Correlation ID plugin enabled on either and you pass a request, you will be provided with a header and value: + +``` +Kong-Request-ID: +``` + +But now you send another request to the same entities but instead you provide the header shown above with the same UUID value, like the below example: + +```bash +curl http://localhost:18000/test123 -H "Kong-Request-Id: ab58c8d9-04d2-4b6e-9f31-431cf922ab8d" +``` + +The response you will receive will not contain a new unique UUID provided by the header, instead the plugin will be ignored entirely and your request will exhibit the `Kong-Request-Id` header with the provided value: + +```json +{ + "args": {}, + "data": "", + "files": {}, + "form": {}, + "headers": { + "Accept": "*/*", + "Host": "httpbin.org", + "Kong-Request-Id": "ab58c8d9-04d2-4b6e-9f31-431cf922ab8d", + "User-Agent": "curl/7.81.0", + "X-Amzn-Trace-Id": "Root=1-63f5227e-44d396634930aeed239e4407", + "X-Forwarded-Host": "localhost", + "X-Forwarded-Path": "/test123", + "X-Forwarded-Prefix": "/test123" + }, + "json": null, + "method": "GET", + "origin": "172.20.0.1, 64.57.201.84", + "url": "http://localhost/anything" +} +``` + +The main concern is here not just overriding UUIDs when they may be used for checking uniqueness by a service, but also the potential for clients to mistake a passed header and value for a duplicate UUID. diff --git a/app/_support/kong-gateway-why-is-the-proxy-caching-advanced-plugin-always-bypassing-the-cache-and-hitting-the-api.md b/app/_support/kong-gateway-why-is-the-proxy-caching-advanced-plugin-always-bypassing-the-cache-and-hitting-the-api.md new file mode 100644 index 0000000000..3e352dc0b4 --- /dev/null +++ b/app/_support/kong-gateway-why-is-the-proxy-caching-advanced-plugin-always-bypassing-the-cache-and-hitting-the-api.md @@ -0,0 +1,41 @@ +--- +title: "{{site.base_gateway}}: Proxy Caching Advanced plugin always bypasses the cache and hits the API" +content_type: support +description: "The Proxy Caching Advanced plugin may always bypass the cache and hit the API if the upstream response's content type is not included in the plugin's configuration." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "{{site.base_gateway}}: Why is the Proxy Caching Advanced plugin always bypassing the cache and hitting the API?" + a: | + The Proxy Caching Advanced plugin bypasses the cache (shown by the `x-cache-status: bypass` response + header) when the upstream response's content type is not included in the plugin's configuration. Check + the upstream content type with `curl -v`, then update the plugin configuration to include that content + type. For large responses you may also need to adjust NGINX buffer sizes (for example, + `KONG_NGINX_HTTP_PROXY_BUFFERS="8 16k"`). Verify the fix by confirming `x-cache-status` shows `HIT` on + repeated requests. +related_resources: [] +--- + +## Problem + +The Proxy Caching Advanced plugin always hits the API instead of caching, as indicated by the `x-cache-status: bypass` header in the response. + +## Cause + +The Proxy Caching Advanced plugin bypasses the cache and hits the API when the upstream response's content type is not included in the plugin's configuration. + +## Solution + +1. Check the upstream content type using a tool like `curl -v`. +2. Update the plugin configuration to include that content type. +3. (Optional) Adjust NGINX buffer sizes if you're dealing with large responses: + + ```bash + KONG_NGINX_HTTP_PROXY_BUFFERS="8 16k"; + ``` +4. Test the fix by checking if `x-cache-status` shows `HIT` on repeated requests. + +Ensuring the plugin is configured with the correct content types is the key to enabling proper caching behavior. diff --git a/app/_support/no-access-control-allow-origin-header-is-present-on-the-requested-resource.md b/app/_support/no-access-control-allow-origin-header-is-present-on-the-requested-resource.md new file mode 100644 index 0000000000..53f03302bf --- /dev/null +++ b/app/_support/no-access-control-allow-origin-header-is-present-on-the-requested-resource.md @@ -0,0 +1,41 @@ +--- +title: "No 'Access-Control-Allow-Origin' header on the response when using the CORS plugin" +content_type: support +description: "The CORS plugin omits the Access-Control-Allow-Origin header when config.origins contains an invalid character or when the request origin does not match a configured origin." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: Why is no Access-Control-Allow-Origin header present on the response even though I configured the CORS plugin? + a: | + The CORS plugin omits the `Access-Control-Allow-Origin` header for two main reasons. First, + `config.origins` may contain an invalid character such as a leading or trailing space, quotes, or + brackets; the field takes a simple comma-separated list and does not require brackets. Second, when + multiple origins are configured, the plugin only returns the ACAO header if the request's Origin + matches one of them. Review `config.origins`, remove invalid characters, and confirm the request's + Origin matches a configured value. +related_resources: [] +--- + +## Problem + +When using the CORS plugin we see proxy requests being denied. When viewing the error in the browser developer tools an error similar to the below is found in the console: + +``` +Access to XMLHttpRequest at 'https://proxy/echo' from origin 'https://konghq.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. +``` + +A value has been added to `config.origins` in the CORS plugin, but the header is not present on the response. + +## Cause + +This can occur for a couple of reasons: + +1. The `config.origins` contains an invalid character. These can be as simple as a leading or trailing space character or adding quotes or brackets. As the field accepts a string array it is common to assume you need to use brackets, however the field does not require them and instead a simple comma separated list can be used. +2. You have configured several origins. When you specify more than one origin the plugin will only return the ACAO header if a match is found. For example, if you configured it as `config.origins=https://konghq.com,https://kuma.io`, generating a cross site request from an Origin of `https://mockbin.org` would return this error. If the same request was issued from `https://kuma.io` the header would be added as it matches one of the defined origins. + +## Solution + +Review the value of `config.origins` and remove any invalid characters such as leading or trailing spaces, quotes, or brackets, using a simple comma separated list. When configuring several origins, confirm that the request's Origin matches one of the configured values, since the plugin only returns the ACAO header on a match. diff --git a/app/_support/what-resources-should-be-adjusted-for-kong-deployed-in-a-kubernetes-environment-to-increase-maximum-throughput.md b/app/_support/what-resources-should-be-adjusted-for-kong-deployed-in-a-kubernetes-environment-to-increase-maximum-throughput.md new file mode 100644 index 0000000000..49f6c167b1 --- /dev/null +++ b/app/_support/what-resources-should-be-adjusted-for-kong-deployed-in-a-kubernetes-environment-to-increase-maximum-throughput.md @@ -0,0 +1,43 @@ +--- +title: Adjusting resources to increase maximum throughput for Kong in Kubernetes +content_type: support +description: "When scaling Kong in Kubernetes to increase maximum throughput, ensure adequate CPU and memory resources and use performance benchmarking to determine the precise requirements." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: What resources should be adjusted for Kong deployed in a Kubernetes environment to increase maximum throughput? + a: | + Kong's maximum throughput is highly dependent on CPU and memory, so increasing the CPU allocation + is usually what's needed to handle additional requests. The exact amount varies with request type, + network latency, and other use case-specific factors, so conduct performance benchmarking to determine + the precise requirements. Review current resource utilization and pod counts, consult the Kong sizing + guidelines, run a performance test simulating the additional load, then scale vertically (more CPU per pod) + or horizontally (more pods) based on the results, and monitor closely afterward. +related_resources: + - text: Kong Sizing Guidelines + url: /gateway/resource-sizing-guidelines/#scaling-dimensions +--- + +## Problem + +To increase the maximum throughput of Kong deployed in a Kubernetes environment, you need to determine which resources to adjust. + +## Solution + +When preparing to scale Kong to increase maximum throughput, it's essential to ensure that you have adequate CPU and memory resources. The maximum throughput Kong can handle is highly dependent on these resources. To accommodate the increased load, it's better to add more compute power, which typically involves increasing the CPU allocation to handle the additional requests. + +The exact amount of CPU to increase can vary based on several factors, including the type of requests being processed, network latency, and other use case-specific details. Therefore, we recommend conducting performance benchmarking and optimization exercises to determine the precise resource requirements. + +Here are the steps you should consider: + +1. Review the current resource usage (CPU and memory usage, resource limits) and the minimum and maximum number of pods per cluster. +2. Consult the Kong sizing guidelines to understand the scaling dimensions. +3. Plan and run a performance test simulating the additional transactions per second (`tps`) to assess if the current resources can handle the increased load. +4. Based on the performance test results, adjust the CPU resources accordingly. This may involve scaling vertically (increasing the CPU resources for existing pods) or horizontally (adding more pods to the cluster). + +It's important to note that while you may have enough capacity both vertically and horizontally to support your required throughput increase, the performance test will provide the best indication of whether additional adjustments are needed. + +Remember to monitor the performance and resource usage closely after implementing the changes to ensure that Kong is operating optimally with the increased traffic. diff --git a/app/_support/when-trying-to-log-into-kong-manager-getting-error-config-secret-length-must-be-at-least-1.md b/app/_support/when-trying-to-log-into-kong-manager-getting-error-config-secret-length-must-be-at-least-1.md new file mode 100644 index 0000000000..1bce2b71f7 --- /dev/null +++ b/app/_support/when-trying-to-log-into-kong-manager-getting-error-config-secret-length-must-be-at-least-1.md @@ -0,0 +1,47 @@ +--- +title: "Kong Manager login error: config.secret length must be at least 1" +content_type: support +description: "Logging into Kong Manager fails with a `config.client_secret` schema violation when the OIDC `client_secret` value is empty." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "When trying to Log into kong manager getting error \"config.secret: length must be at least 1\"" + a: | + This error occurs when the value inside `client_secret` is empty, for example if the variable + was wiped out or contains no data, resulting in a schema violation + (`config.client_secret: {"length must be at least 1"}`) when using OIDC as the auth method. + To resolve this, add the correct value to `client_secret` and restart Kong. On the next login, + you will proceed without issue. +related_resources: [] +--- + +## Problem + +When trying to log into Kong Manager we are receiving the following error: + +``` +[error] 2142#0: *3867 [lua] kong.lua:429: fn(): kong[auth][postgres] schema violation (config.client_secret: {"length must be at least 1"}), client: 123.123.123.1, server: kong_admin, request: "GET /auth?Kong-Admin-User=sample@email.com HTTP/1.1", host:"localhost:8001", referrer: "http://localhost:8002/" +``` + +We are using OIDC as our auth method. + +Here is our config: + +```yaml +KONG_ADMIN_GUI_AUTH_CONF: '{"issuer":"http://keycloak:9977/auth/realms/master/.well-known/openid-configuration","admin_claim":"email","client_id": ["kong"],"client_secret": ["${sample}"],"ssl_verify": false,"leeway": 60,"redirect_uri": ["http://localhost:8002"],"scopes": ["openid","profile","offline_access"],"auth_methods": ["authorization_code"], "admin_auto_create_rbac_token_disabled": false }' +``` + +## Cause + +This error will occur when the value inside `client_secret` is empty. If the variable was wiped out or does not contain any data, you will receive the error: + +``` +schema violation (config.client_secret: {"length must be at least 1"}) +``` + +## Solution + +To resolve this, add the correct value to `client_secret` and restart Kong. On the next login, you can proceed without issue. diff --git a/app/_support/when-trying-to-run-luarocks-pack-on-versions-3-x-and-above-getting-executor-failed-running.md b/app/_support/when-trying-to-run-luarocks-pack-on-versions-3-x-and-above-getting-executor-failed-running.md new file mode 100644 index 0000000000..e607f269a0 --- /dev/null +++ b/app/_support/when-trying-to-run-luarocks-pack-on-versions-3-x-and-above-getting-executor-failed-running.md @@ -0,0 +1,51 @@ +--- +title: "`luarocks pack` fails with \"executor failed running\" on Kong 3.x and above" +content_type: support +description: "Running `luarocks pack` on Kong 3.x and above fails with \"executor failed running\"; install `zip` in the Dockerfile to resolve it." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "When trying to run `luarocks pack` on versions 3.x and above getting \"executor failed running\"" + a: | + When installing custom plugins after upgrading to 3.x, `luarocks pack` fails because the `zip` + tool is not installed. Run the command with `--verbose` to confirm: the logs will show + `fs.is_tool_available("zip", "zip")` and `fs.search_in_path("zip")`, indicating `zip` is needed + to handle the files. Install it in the Dockerfile with `RUN yum install zip -y`, then rerun and + the `luarocks pack` command will succeed. +related_resources: [] +--- + +## Problem + +When installing custom plugins using `luarocks` after upgrading to 3.x, the `luarocks pack` command fails with the following error. + +``` +executor failed running [/bin/sh -c luarocks pack kong-plugin-myplugin 0.1.0-1 --verbose]: exit code: 1 +``` + +## Solution + +First, verify the error in verbose mode. For example, inside a Dockerfile add the following: + +```dockerfile +RUN luarocks pack kong-plugin-myplugin 0.1.0-1 --verbose +``` + +In the logs the last message will show: + +``` +#11 0.721 fs.is_tool_available("zip", "zip") +#11 0.721 fs.search_in_path("zip") +#11 0.721 fs.change_dir_to_root() +``` + +This indicates `zip` needs to be installed to properly handle the files. Inside the Dockerfile add the following line: + +```dockerfile +RUN yum install zip -y +``` + +Now rerun the Dockerfile and the `luarocks pack` command will succeed. diff --git a/app/_support/why-are-some-plugin-configuration-values-in-kong-s-deck-dump-file-enclosed-in-single-quotes.md b/app/_support/why-are-some-plugin-configuration-values-in-kong-s-deck-dump-file-enclosed-in-single-quotes.md new file mode 100644 index 0000000000..6408e1e2d9 --- /dev/null +++ b/app/_support/why-are-some-plugin-configuration-values-in-kong-s-deck-dump-file-enclosed-in-single-quotes.md @@ -0,0 +1,41 @@ +--- +title: "Single quotes around plugin configuration values in Kong's deck dump file" +content_type: support +description: "When working with Kong's deck dump files, especially for plugins like the OpenID Connect plugin, you might notice that some configuration values are enclosed in single quotes." +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: "Why are some plugin configuration values in Kong's deck dump file enclosed in single quotes?" + a: | + deck dump files use YAML, and per the YAML 1.2 specification certain characters can cause the parser + to misinterpret an unquoted string as a different data type or structure. To prevent this, YAML encloses + such strings in quotes to ensure they are parsed correctly. This happens for values containing characters + like `#`, `:`, `{`, `}`, `[`, `]`, `!`, `%`, `@`, and others, as well as leading zeros in numeric strings + or strings matching `true`, `false`, `on`, `off`, or `null`. For example, the value `}E]0Y1a$-P` appears + as `'}E]0Y1a$-P'` in the dump. +related_resources: + - text: "YAML 1.2 specification" + url: "https://yaml.org/spec/1.2.2/" +--- + +## Single quotes around plugin configuration values in a deck dump file + +When working with Kong's deck dump files, especially for plugins like the OpenID Connect plugin, you might notice that some configuration values are enclosed in single quotes. This behavior is particularly observed when values start with characters such as `!` or `}`. Understanding why this happens requires a bit of insight into how YAML, the format used by deck dump files, handles data. YAML is designed to be easily readable by humans and is used by Kong's underlying libraries to parse the configuration data output by the `deck` command. According to the YAML 1.2 specification, certain characters can cause the YAML parser to misinterpret unquoted strings as different data types or structures. To prevent this, YAML encloses strings that start with or contain these characters in quotes. This ensures the correct parsing and handling of these values. + +Here are some common scenarios where you'll see values enclosed in quotes in a YAML file: + +- Characters such as `#`, `:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `?`, `|`, `-`, ``, `=`, `!`, `%`, `@`, `` ` ``, along with any quote character (`'` or `"`), can trigger this behavior. +- Leading zeros in purely numeric strings. +- Strings that match boolean values (`true`, `false`, `on`, `off`) or `null` values exactly. + +For example, if you have a configuration value for `TokenPostArgsValues` set as `}E]0Y1a$-P`, in the deck dump file, it will appear as: + +```yaml +token_post_args_values: + - '}E]0Y1a$-P' +``` + +This quoting is necessary to ensure that the YAML parser correctly interprets the value as a string rather than mistakenly treating it as a different data type or structure. For a comprehensive understanding of all characters and scenarios that might cause a value to be quoted, refer to the YAML 1.2 specification. This explanation clarifies why certain values in Kong's deck dump files are enclosed in single quotes, ensuring accurate parsing and handling of configuration data. diff --git a/app/_support/why-does-the-output-of-the-file-log-plugin-get-mixed-with-kong-log-entries.md b/app/_support/why-does-the-output-of-the-file-log-plugin-get-mixed-with-kong-log-entries.md new file mode 100644 index 0000000000..0d4b47fe17 --- /dev/null +++ b/app/_support/why-does-the-output-of-the-file-log-plugin-get-mixed-with-kong-log-entries.md @@ -0,0 +1,47 @@ +--- +title: File-log plugin output mixed with Kong log entries +content_type: support +description: In containerized environments, the Docker or Kubernetes host node collects logs via a PIPE that every container outputs to /dev/stdout. +products: + - gateway +works_on: + - on-prem + - konnect +tldr: + q: Why does the output of the file-log plugin get mixed with Kong log entries? + a: | + In containerized environments, the host node collects logs via a PIPE that every container outputs to + `/dev/stdout`. When writing data larger than the PIPE buffer (usually 4096 bytes), the Linux kernel can't + ensure the atomicity of the `write()` syscall, which explains the interleaving for logs bigger than 4kB. + The `PIPE_BUF` limit is hard-coded in the kernel and can't be increased via config. As a workaround, + remove any unneeded response headers or data from the file-log output using the `custom_fields_by_lua` field. +related_resources: + - text: file-log + url: /plugins/file-log/ + - text: "`custom_fields_by_lua`" + url: /plugins/file-log/#custom-fields-by-lua + - text: Kong charts values.yaml + url: https://github.com/Kong/charts/blob/kong-2.47.0/charts/kong/values.yaml#L101-L108 + - text: Pipe Atomicity + url: https://www.gnu.org/software/libc/manual/html_node/Pipe-Atomicity.html + - text: Limits for Files + url: https://www.gnu.org/software/libc/manual/html_node/Limits-for-Files.html + - text: Linux PIPE_BUF limit + url: https://github.com/torvalds/linux/blob/v5.15/include/uapi/linux/limits.h#L14 +--- + +## Why does the output of the file-log plugin get mixed with Kong log entries + +When running in Kubernetes, some of Kong's logs are directed to `/dev/stdout`. + +If we also use a file-log plugin to write to `/dev/stdout`, why does the output of the file-log plugin get mixed with Kong log entries? + +In containerized environments, the Docker or Kubernetes host node collects logs via a PIPE that every container outputs to `/dev/stdout`. + +When writing data through a PIPE, the size of the data has to fit into a PIPE buffer, which is usually 4096 Bytes. In other words, when writing data larger than 4kB through a PIPE, the Linux kernel can’t ensure the atomicity of the syscall `write()`. + +This could explain why the interleaving occurs for logs whose size is bigger than 4kB. Unfortunately, there’s no config setting to increase the `PIPE_BUF` directly as it is hard-coded in the kernel. + +The file-log plugin uses `write()` directly to output to a file. This is already a blocking I/O operation, which could affect performance, and there is no locking mechanism in `/dev/stdout`. + +As a workaround, we recommend removing any unneeded response headers or any data that isn't required from the file-log by using the `custom_fields_by_lua` field.