Clarification on Concurrency: Workers vs Threads, scaling math, and max_threads

[anas-chtourou]

Hello FrankenPHP team!

We are currently tuning our production environment on Kubernetes and finalizing our internal documentation.
We have a few internal discussions regarding how FrankenPHP handles concurrency, specifically the math behind workers, threads, and auto-scaling (max_threads).

We would love to get official clarification on our assumptions to ensure we are configuring things optimally. This will likely help other users tuning their production environments too!

1. Workers vs Threads & Sizing Math
Our assumption:

• a "Worker" in FrankenPHP is essentially a lightweight Goroutine running the PHP script in a loop
• a "Thread" is a real OS/POSIX thread initialized via CGo.

If we have the following configuration:

• name: FRANKENPHP_CONFIG

  value: num_threads 33 max_threads 65

• name: FRANKENPHP_WORKER_CONFIG

  value: num 8

Our mathematical deduction:
Since 8 workers are permanently loaded in RAM, we calculated that the maximum number of threads available to be dynamically spawned or used for other tasks is 40, based on this ratio from our internal docs:
Max Available Threads = (max_threads + num) - num_threads (e.g., (65 + 8) - 33 = 40 as supported by the screenshot above of a load test, busy threads doesn't go over 40).

Question 1: Is this mathematical ratio correct? Do workers permanently occupy a thread from the num_threads pool, or do they dynamically grab them?

Question 2: Are there any other hidden mathematical rules, constraints, or ratios (similar to num_threads * memory_limit + GOMEMLIMIT for pod memory) that we should be aware of when sizing these parameters?

2. The utility of max_threads vs. Kubernetes Pod Scaling
We are debating the best approach to handle traffic spikes.

Approach A: max_threads isn't particularly useful in a Kubernetes context. It is better to strictly limit the number of threads per pod (to heavily control memory/CPU usage) and rely entirely on Kubernetes HPA (Horizontal Pod Autoscaler) to add more pods when the load increases.

Approach B: max_threads is extremely useful because it acts as a fast buffer. Spawning an additional thread at runtime happens in milliseconds, which makes the server more resilient to sudden latency spikes. Relying solely on K8s HPA means waiting seconds or minutes for a new pod to become ready, risking queued or dropped requests during a sudden spike.

Question 3: Could you confirm if max_threads is indeed designed to act as this fast buffer for traffic spikes, and how do you recommend balancing max_threads (vertical scaling within the pod) versus horizontal pod scaling?

3. Understanding Metrics for HPA
We are using your Prometheus metrics to build our dashboards
We noticed frankenphp_busy_threads accurately tracks our active threads.

We also track frankenphp_worker_busy_workers and frankenphp_queue_depth.

Question 4: For Kubernetes HPA, would you recommend scaling based primarily on frankenphp_queue_depth to ensure pods scale up before requests start piling up and timing out?

Thanks for this project and for taking the time to clarify these points!

[AlliBalliBaba]

1:
Yes, the threads are the underlying PHP threads and your calculation is correct. The 8 threads are occupied permanently, while it will scale up to 40 threads with enough pressure from queued requests.

If you're not using regular Cgi mode, the "regular threads" will end up doing nothing. So it might be better to just omit num_threads entirely. In that case only 1 regular thread will be assigned and (max_threads - 1 =64) threads will be available to the worker.

If you have a Caddyfile you can also omit the global max_threads setting and define it inside of the worker directly, makes the math less confusing:

# inside php_server or php
worker /path/to/worker.php {
    num 8 # always at least 8 workers ready for requests
    max_threads 40 # max 40 concurrent requests for this worker
}

The absolute max memory is then ideed composed of the max go memory and the max php memory * threads. Realistically, that limit will be too generous in most cases.

2:
Yes max_threads is a fast buffer, specifically for IO bound work. Threads will scale in milliseconds. It's most useful to handle spikes of very slow requests or to mitigate situations where the server is oberburdened.

Having a high max_threads also just helps fully utilizing your CPU if you are not sure about your traffic patterns. In a Kubernetes context you're probably monitoring this much closer, so you can also get away with a good fixed thread setting.

Also worth mentioning that there might be cases in which you do not want to scale, like when you are communicating with slow external endpoints. In that case max_threads can also help when splitting a thread pool to throttle concurrency to a specific endpoint or to minimize unused held connections.

3:
Yes scaling based on queue depth is probably a good approach since it considers both CPU- and IO-bound overburdening. Additionally, scaling on CPU alone also helps if single requests tend to sometimes consume a lot of CPU or if you want to minimize latency.

I'm currently not using Kubernetes in production so take my advice with a grain of salt. I know @withinboredom uses Kubernetes, so maybe he can give some additional advice.