A Terraform module to provision a minimal k3s Kubernetes cluster tailored to LEAP's VPN stack on Hetzner Cloud.
- Deploys a minimal k3s cluster
- Uses Hetzner Cloud resources (servers, networks)
- Automated provisioning via Terraform
- Private networking between nodes
- Easily extensible for more nodes or features
We propose the following setup of services across worker nodes:
k3s controller node (backend, reverse-proxy):
- Ingress : Traefik
- cert-manager (https://cert-manager.io/) and other kube-master components
k3s worker node 1 (menshen):
- menshen
- invitectl to add invite codes to db menshen depends on
k3s worker node 2: (monitoring):
- kube-prometheus-stack including
- Prometheus
- Grafana
Follow these steps to set up your project:
- Create a Hetzner Account on https://www.hetzner.com/ and log in.
- Create a new project: on the dashboard, click
New Project, enter a name and clickAdd project.
It's easiest to start with the template files for backend and gateway cluster located under hetzner/examples. These files contain:
- The necessary code to import this repository as a Terraform module.
- All required variables for a basic setup.
Just copy the whole hetzner/exmaples directory to a directory you wish (your working directory) and adapt the Terraform examples.
Make sure you have access to the git repo and can git clone it, otherwise Terraform initialization will fail.
Alternatively you can use the ssh-method to clone the repo during the init-process by replacing the source = ... line by source = "git::ssh://git@0xacab.org/leap/container-platform/terraform-k3s.git"
In your template file you can choose how many and which servers you want to provision in which datacenter. It is important to first check which server types are available in which region. It is easiest to achieve this by navigating to your cloud project in the Hetzner console and pretending to want to create a server by clicking through the interface. Go to Servers on the top of the left navigation bar and click "Add Server". There you can see all current datacenter locations with their codes and server types available in them. Choose only servers with 'x86' architecture. You need to fill in the server names in the template file.
Below is a list of the variables you should provide in your config:
| Variable | Type | Description |
|---|---|---|
| k3s_cluster_name | string | The name for your k3s cluster. |
| k3s_leader_count | number | The number of leader nodes. Must be odd. Except for corner cases where you want to provision a multi-leader cluster, 1 is fine. |
| k3s_controller_server_type | string | The hetzner server code for controller nodes. Choose one from https://www.hetzner.com/cloud/#pricing, only choose Intel/AMD machines." |
| k3s_base_os | string | The name of the operating system to install on all controller nodes and the default for worker nodes. We only tested on "Debian 13". |
| gateway_mode_enabled | boolean | Set to true if you want to provision a gateway cluster, false for backend. Defaults to false. |
| datacenter | string | Hetzner datacenter name (e.g., hel1-dc2 for Helsinki). Choose one from https://docs.hetzner.com/cloud/general/locations but make sure the chosen server type is available at the chosen location. |
| k3s_network_name | string | Name for the network. |
| network_zone | string | Name of network zone. Must match the datacenter location. See https://docs.hetzner.com/cloud/general/locations/ |
| admins | list(object({ name = string, public_key = string })) | List of admin SSH key objects containing a freely selectable name and a corresponding public ssh key. |
| k3s_worker_nodes | list(object({ name = string, count = number, server_type = optional string, image = optional string, labels = optional map(string) })) | A list of groups of worker nodes, each sharing a common operating system and server flavor. The variable count determines how many nodes of this kind you want to spin up. The image defaults to the value of k3s_base_os. In a single-node cluster like a gateway this variable should be [] as there is only one controller node and no worker nodes. See vars.tf for more configuration options. |
| other_labels | map(string) | Labels to tag the virtual machines in your cloud project for easy filtering in the Hetzner console. Both key and value must be 63 characters or less, beginning and ending with an alphanumeric character and alphanumerics can be used inbetween. |
First you need to generate a Read-Write token for your project in order to create and delete resources with Terraform. Enter your new project in the Hetzner console and navigate to Security settings (left sidebar, bottom).
Go to the API tokens tab and click Generate API token. Make sure to activate Read & Write permissions. Store the generated token securily. It is only shown once in the web interface.
In order to provide this API token to Terraform you can use an environment variable. Open a shell, fill in your newly created token and run:
export TF_VAR_hcloud_token=<your-API-token>In the same shell and in the folder with your Terraform project file, run
terraform init
This initializes a working directory containing Terraform configuration files.
When everything works out run
terraform plan
This creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure.Read the plan and make sure things are getting created as expected.
Last run
terraform apply
This executes the actions proposed in the Terraform plan to create, update, or destroy infrastructure. Your k3s cluster is now being provisioned. 🎊
You can check on the Hetzner Cloud Console Dashboard if all of your resources are created as expected.
The podlily repository contains a script access_cluster.sh that can be used to port-forward into the cluster. This method also allows provisioning from your local machine to remotes. Copy the script into this directory to use it. You can find more information on the script in its documentation that is also part of podlily.