Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions docs/cloud/connectivity/aws-connectivity.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,18 @@ After creating the PrivateLink endpoint, configure your clients to use it throug

## Requirements

Your AWS PrivateLink endpoint must be in the same region as your Temporal Cloud namespace. If using [replication for High Availability](/cloud/high-availability), the PL connection must be in the same region as one of the replicas.
* Your AWS PrivateLink endpoint must be in the same region as your Temporal Cloud namespace. If using [replication for High Availability](/cloud/high-availability), the PL connection must be in the same region as one of the replicas.
See [cross-region PrivateLink connectivity](#cross-region-privatelink) to access the Namespace from a different region.
* Your Private DNS must be configured to direct Worker / Client traffic to your VPC Endpoint, as described below.
* If the Worker / Client is not using the Namespace Endpoint, it may need to set the `server_name` config to the Namespace Endpoint string, as described below.

AWS Cross Region endpoints are not supported.
### Cross-region PrivateLink Connectivity {#cross-region-privatelink}

Temporal Cloud does **not** support [cross-region connectivity for AWS PrivateLink](https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-cross-region-connectivity-for-aws-privatelink/) out of the box.

You can access your VPC Endpoint in a different region using the [native cross-region routing in AWS](https://docs.aws.amazon.com/whitepapers/latest/building-scalable-secure-multi-vpc-network-infrastructure/centralized-access-to-vpc-private-endpoints.html#cross-region-endpoint-access).

When using High Availability on Temporal Cloud, it's best practice to have two VPC Endpoints, one in each of the Namespace's regions, to ensure at least one VPC Endpoint is accessible during a regional outage.

## Creating an AWS PrivateLink connection

Expand All @@ -58,7 +67,7 @@ Individual Namespaces do not use separate services.

:::

<JsonTable filename="/json/privatelink_aws.json" />
<JsonTable filename="/json/privatelink_services_aws.json" />

7. Confirm your service by clicking on the _Verify service_ button. AWS should respond "Service name verified."

Expand Down Expand Up @@ -111,7 +120,7 @@ This approach is **optional**; Temporal Cloud works without it. It simply stream
| Endpoint type | PHZ domain format | Example |
| ------------------ | ---------------------------------- | -------------------------------------- |
| Namespace endpoint | `<namespace-id>.tmprl.cloud` | `payments.abcde.tmprl.cloud` |
| Regional endpoint | `<cloud>-<region>.region.tmprl.cloud` | `aws-ap-northeast-2.region.tmprl.cloud` |
| Regional endpoint | `<region>.<cloud>.api.temporal.io` | `ap-northeast-2.aws.api.temporal.io` |

### Step-by-step instructions

Expand Down Expand Up @@ -188,7 +197,7 @@ The DNS resolver inside your VPC returns the private endpoint, while TLS still v

## Configure private DNS for Namespaces with High Availability

For Namespaces with [High Availability features](/cloud/high-availability), you need to override DNS for `region.tmprl.cloud` so each region resolves to the local VPC Endpoint, and you need to ensure Workers can reach whichever region is active. Failover is transparent to clients only when this is set up correctly.
For Namespaces with [High Availability features](/cloud/high-availability), you need to override DNS for `region.tmprl.io` so each region resolves to the local VPC Endpoint, and you need to ensure Workers can reach whichever region is active. Failover is transparent to clients only when this is set up correctly.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still be region.tmprl.cloud.


The complete guidance — including single-cloud (AWS-only) HA, multi-cloud HA (AWS PrivateLink + GCP Private Service Connect), and a recommended failover-testing plan — lives on a single page: [Connectivity for High Availability](/cloud/high-availability/ha-connectivity).

Expand All @@ -203,12 +212,13 @@ For single-region Namespaces, you can avoid creating DNS records for each Namesp

With this approach, new Namespaces do not require new DNS records.

:::warning Not compatible with High Availability Namespaces
:::warning Extra care needed for Namespaces with High Availability

Direct VPCE targeting bypasses the Temporal-managed DNS CNAME that normally routes a Namespace to its active region, so each Worker is pinned to whichever VPC Endpoint you configure. To keep both regions reachable, you must run Workers in each region and configure them with **different endpoints**:

This approach does not work for Namespaces with High Availability features.
HA Namespaces rely on Temporal's public DNS CNAME records to route traffic to the active region during failover.
If you bypass DNS, your Workers cannot follow the CNAME to the new region.
For HA Namespaces, use [private DNS](#configuring-private-dns-for-aws-privatelink) instead.
- Workers in the primary region: the local VPC Endpoint for that region (e.g., `vpce-...-us-east-1.vpce.amazonaws.com:7233`).
- Workers in the replica region: the local VPC Endpoint for that region (e.g., `vpce-...-us-west-2.vpce.amazonaws.com:7233`).
- All Workers: the same SNI override — the Namespace Endpoint value (e.g., `my-namespace.my-account.tmprl.cloud`).

:::

Expand Down
8 changes: 5 additions & 3 deletions docs/cloud/connectivity/gcp-connectivity.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ If you use GCP Private Service Connect, you must manually update your workers to

## Requirements

Your GCP Private Service Connect connection must be in the same region as your Temporal Cloud namespace. If using [replication for High Availability](/cloud/high-availability), the PSC connection must be in the same region as one of the replicas.
* Your GCP Private Service Connect endpoint must be in the same region as your Temporal Cloud namespace. If using [replication for High Availability](/cloud/high-availability), the PSC connection must be in the same region as one of the replicas.
* Your Private DNS must be configured to direct Worker / Client traffic to your PSC endpoint, as described below.
* If the Worker / Client is not using the Namespace Endpoint, it may need to set the `server_name` config to the Namespace Endpoint string, as described below.

## Creating a Private Service Connect connection

Expand Down Expand Up @@ -131,7 +133,7 @@ This approach is **optional**; Temporal Cloud works without it. It simply stream
| ------------------------------------------ | ---------------------------------- | ---------------------------------------------- |
| Single-region namespace with mTLS auth | `<account>.tmprl.cloud` | `payments.abcde.tmprl.cloud` ↔ `X.X.X.X` |
| Single-region namespace with API-key auth | `<cloud_provider>.api.temporal.io` | `us-central1.gcp.api.temporal.io` ↔ `X.X.X.X` |
| Multi-region namespace | `region.tmprl.cloud` | `gcp-us-central1.region.tmprl.cloud` ↔ `X.X.X.X` |
| Multi-region namespace | `<cloud_provider>.api.temporal.io` | `us-central1.gcp.api.temporal.io` ↔ `X.X.X.X` |

### Step-by-step instructions

Expand All @@ -152,7 +154,7 @@ Save the internal IP -- you will point the A record at it.
1. Open _Network Services → Cloud DNS → Create zone_.
2. Select zone type **Private**.
3. Enter a **Zone name** (e.g., `temporal-cloud`).
4. Enter a **DNS name** based on the table above (e.g., `payments.abcde.tmprl.cloud` or `aws-us-east-1.region.tmprl.cloud`).
4. Enter a **DNS name** based on the table above (e.g., `payments.abcde.tmprl.cloud` or `us-east-1.aws.api.temporal.io`).
5. Select **Add networks** and choose the Project and Network that contains your PSC endpoint.
6. Click **Create**.

Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/connectivity/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ The TLS server name override depends on your authentication method:
| -------------- | ---------------------- |
| mTLS (single-region Namespace) | The Namespace Endpoint, e.g. `my-namespace.my-account.tmprl.cloud` |
| API key (single-region Namespace) | The regional API endpoint, e.g. `us-east-1.aws.api.temporal.io` or `us-central1.gcp.api.temporal.io` |
| Multi-region Namespace (mTLS or API key) | The active region endpoint, e.g. `aws-us-east-1.region.tmprl.cloud` |
| Multi-region Namespace (mTLS or API key) | The active region endpoint, e.g. `us-east-1.aws.api.temporal.io` |

If you authenticate with an API key over PrivateLink/PSC and use the wrong server name, the TLS handshake will fail with errors such as `connection reset by peer` even though `nc` reports the port as open.

Expand Down
6 changes: 6 additions & 0 deletions docs/cloud/get-started/namespaces.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,12 @@ There are two types of gRPC endpoints for accessing a Namespace in Temporal Clou
- A Temporal Client can use a regional endpoint to ensure connection to a Namespace always happens within that region. This can be useful in advanced [High Availability](/cloud/high-availability) setups where you want explicit control over which region handles requests.
- When using mTLS to authenticate, the Temporal Client must set the `server_name` property to `<namespace endpoint value>` in its request to the value of the Namespace endpoint. This tells the client to expect a different SNI header during the TLS handshake, since the request to the regional endpoint is redirected to the specific Namespace.

:::note Update outdated regional endpoints

The older regional endpoint format ending in `region.tmprl.cloud` (for example, `aws-us-east-1.region.tmprl.cloud`) is outdated. Update any Clients, Workers, and private DNS records to the current format ending in `api.temporal.io` (for example, `us-east-1.aws.api.temporal.io:7233`) to ensure uninterrupted access to your Temporal Cloud Namespaces.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I this it's unlikely people are using region.tmprl.cloud in their clients directly, as we don't support certs for those domains. Maybe someone could use it in combination with a server_name override, but that feels unlikely.


:::

### Configuring a Temporal Client with API keys or mTLS

To use API keys to connect with the [Temporal CLI](/cli), [Client SDK](/develop), [tcld](/cloud/tcld),
Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/high-availability/failovers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ In most scenarios, we recommend you let Temporal handle failovers for you.
After failover, be aware of the following points:

- When working with Multi-region Namespaces, your CNAME may change.
For example, it may switch from aws-us-west-1.region.tmprl.cloud to aws-us-east-1.region.tmprl.cloud.
For example, it may switch from us-west-1.aws.api.temporal.io to us-east-1.aws.api.temporal.io.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like this is not the current behavior though with HA namespaces?

This change doesn't affect same-region Namespaces.

- Your Namespace endpoint _will not change_.
Expand Down
26 changes: 13 additions & 13 deletions docs/cloud/high-availability/ha-connectivity.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ A Namespace with High Availability features has two replicas — a primary and a
Temporal Cloud expresses the active replica through DNS:

- The Namespace DNS record (`<ns>.<account>.tmprl.cloud`) is a CNAME.
- It points to the active region's regional record (`<provider>-<region>.region.tmprl.cloud`).
- It points to the active region's regional record (`<region>.<provider>.api.temporal.io`).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment for this whole page, I don't think we're CNAME'ing HA Namespace Endpoints to api.temporal.io domains

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

daniel@Daniels-MacBook-Pro saas-components % dig daniel.a2dd6.tmprl.cloud

; <<>> DiG 9.10.6 <<>> daniel.a2dd6.tmprl.cloud
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13742
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;daniel.a2dd6.tmprl.cloud.      IN      A

;; ANSWER SECTION:
daniel.a2dd6.tmprl.cloud. 15    IN      CNAME   aws-us-east-1.region.tmprl.cloud.
aws-us-east-1.region.tmprl.cloud. 60 IN A       54.86.97.16
aws-us-east-1.region.tmprl.cloud. 60 IN A       35.170.125.250
aws-us-east-1.region.tmprl.cloud. 60 IN A       34.195.80.228
aws-us-east-1.region.tmprl.cloud. 60 IN A       100.28.203.93
aws-us-east-1.region.tmprl.cloud. 60 IN A       52.86.32.76
aws-us-east-1.region.tmprl.cloud. 60 IN A       34.199.152.251

- On failover, Temporal Cloud rewrites the CNAME target.

Namespace DNS records have a 15-second TTL. Clients should converge to the new region within roughly 30 seconds (about twice the TTL) once their resolver cache expires.
Expand All @@ -40,19 +40,19 @@ For private connectivity, your job is to make sure that:
This is the most common setup: both replicas live in AWS regions, and Workers connect via AWS PrivateLink.

When using PrivateLink, you connect to Temporal Cloud through a VPC Endpoint, which uses addresses local to your network.
Temporal treats each `region.tmprl.cloud` zone as a separate zone, so you override resolution per region.
Temporal treats each `aws.api.temporal.io` zone as a separate zone, so you override resolution per region.

Before failover, with the active region being `aws-us-west-2`:

| Record name | Record type | Value |
| ----------------------------------- | ----------- | -------------------------------- |
| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-west-2.region.tmprl.cloud |
| ha-namespace.account-id.tmprl.cloud | CNAME | us-west-2.aws.api.temporal.io |

After a failover to `aws-us-east-1`, Temporal Cloud rewrites the CNAME:

| Record name | Record type | Value |
| ----------------------------------- | ----------- | -------------------------------- |
| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-east-1.region.tmprl.cloud |
| ha-namespace.account-id.tmprl.cloud | CNAME | us-east-1.aws.api.temporal.io |

The Temporal-managed CNAME changed from us-west-2 to us-east-1 — your private DNS does not need to change.

Expand All @@ -64,16 +64,16 @@ The Temporal-managed CNAME changed from us-west-2 to us-east-1 — your private

### Setting up the DNS override (AWS)

In AWS, use a Route 53 private hosted zone for `region.tmprl.cloud` to override resolution per region:
In AWS, use a Route 53 private hosted zone for `aws.api.temporal.io` to override resolution per region:

| Record name | Record type | Value (your VPC Endpoint DNS) |
| ------------------------------------ | ----------- | ------------------------------------------------------------ |
| `aws-us-west-2.region.tmprl.cloud` | CNAME | `vpce-...-us-west-2.vpce.amazonaws.com` |
| `aws-us-east-1.region.tmprl.cloud` | CNAME | `vpce-...-us-east-1.vpce.amazonaws.com` |
| `us-west-2.aws.api.temporal.io` | CNAME | `vpce-...-us-west-2.vpce.amazonaws.com` |
| `us-east-1.aws.api.temporal.io` | CNAME | `vpce-...-us-east-1.vpce.amazonaws.com` |

Link the private zone to every VPC where Workers run.

When your Workers connect to the Namespace, they first resolve `<ns>.<account>.tmprl.cloud`, which CNAMEs to `<aws-active-region>.region.tmprl.cloud`, which then resolves to your local VPC Endpoint.
When your Workers connect to the Namespace, they first resolve `<ns>.<account>.tmprl.cloud`, which CNAMEs to `<aws-active-region>.aws.api.temporal.io`, which then resolves to your local VPC Endpoint.

You also need to decide how Workers reach whichever region becomes active. Either:

Expand All @@ -82,12 +82,12 @@ You also need to decide how Workers reach whichever region becomes active. Eithe

## Single-cloud HA on GCP Private Service Connect

For GCP-only HA, the same model applies, but use a Cloud DNS private zone for `region.tmprl.cloud` and point each `gcp-<region>.region.tmprl.cloud` record at the local PSC endpoint IP address.
For GCP-only HA, the same model applies, but use a Cloud DNS private zone for `gcp.api.temporal.io` and point each `<region>.gcp.api.temporal.io` record at the local PSC endpoint IP address.

| Record name | Record type | Value (your PSC endpoint IP) |
| ---------------------------------------- | ----------- | ----------------------------------- |
| `gcp-us-central1.region.tmprl.cloud` | A | `10.x.x.x` (PSC endpoint IP) |
| `gcp-us-east1.region.tmprl.cloud` | A | `10.x.x.x` (PSC endpoint IP) |
| `us-central1.gcp.api.temporal.io` | A | `10.x.x.x` (PSC endpoint IP) |
| `us-east1.gcp.api.temporal.io` | A | `10.x.x.x` (PSC endpoint IP) |

A Connectivity Rule is required for each PSC connection — see [GCP PSC setup](/cloud/connectivity/gcp-connectivity) and [Connectivity Rules](/cloud/connectivity#connectivity-rules).

Expand All @@ -97,7 +97,7 @@ If your replicas span clouds — for example, AWS `us-east-1` (active) and GCP `

Plan for these three things:

1. **DNS overrides for both clouds.** Your private DNS for `region.tmprl.cloud` needs entries for both the AWS region (CNAME → AWS VPCE) and the GCP region (A → PSC IP). This typically means a Route 53 private hosted zone in your AWS Worker VPCs *and* a Cloud DNS private zone in your GCP Worker network — both for the same `region.tmprl.cloud` parent — each with the records relevant to the cloud the Workers run in.
1. **DNS overrides for both clouds.** Your private DNS needs entries for both the AWS region (CNAME → AWS VPCE under `aws.api.temporal.io`) and the GCP region (A → PSC IP under `gcp.api.temporal.io`). This typically means a Route 53 private hosted zone for `aws.api.temporal.io` in your AWS Worker VPCs *and* a Cloud DNS private zone for `gcp.api.temporal.io` in your GCP Worker network — each with the records relevant to the cloud the Workers run in.
2. **Worker reachability across clouds.** Your AWS-resident Workers must be able to reach the GCP PSC endpoint when GCP is active, and vice versa. Options include:
- Run Workers in both clouds (preferred — simplest, lowest latency, matches the failover model).
- Establish cross-cloud connectivity (e.g., AWS Transit Gateway + GCP Cloud Interconnect, or a third-party transit) so Workers in one cloud can resolve and reach the other cloud's private endpoint.
Expand Down Expand Up @@ -149,6 +149,6 @@ The following tables list the available Temporal regions and the DNS record over

<JsonTable filename="/json/privatelink_gcp.json" />

When using a Namespace with High Availability features, the Namespace's DNS record `<ns>.<account>.tmprl.cloud` points to a regional DNS record in the format `<provider>-<region>.region.tmprl.cloud`, where `<provider>-<region>` is the currently active region for your Namespace.
When using a Namespace with High Availability features, the Namespace's DNS record `<ns>.<account>.tmprl.cloud` points to a regional DNS record in the format `<region>.<provider>.api.temporal.io`, where `<provider>` and `<region>` correspond to the currently active region for your Namespace.

During failover, Temporal Cloud changes the target of the Namespace DNS record from one region to another. Namespace DNS records are configured with a 15-second <a href="https://en.wikipedia.org/wiki/Time_to_live">TTL</a>. Any DNS cache should re-resolve the record within this time. As a rule of thumb, receiving an updated DNS record takes about twice (2x) the TTL — clients should converge to the newly targeted region within, at most, a 30-second delay, assuming their resolver and language runtime honor the TTL.
Loading
Loading