-
Notifications
You must be signed in to change notification settings - Fork 14
Add mass environment preview for per-PR preview environments
#236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
739a6cc
Add `mass environment preview` for per-PR preview environments
coryodaniel e3edb7c
Rename EnvironmentDefaultEntry to DefaultEntry
coryodaniel 2255e26
Add --follow to `environment preview`
coryodaniel 651550d
Switch --follow from polling to SDK event streams
coryodaniel bab43e3
Fix golangci-lint findings in follow
coryodaniel c0e0c82
Pad instance-id prefixes in --follow output
coryodaniel 43b744d
Fix nilnil lint: return zero values from errorAPI stubs
coryodaniel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| --- | ||
| id: mass_environment_preview.md | ||
| slug: /cli/commands/mass_environment_preview | ||
| title: Mass Environment Preview | ||
| sidebar_label: Mass Environment Preview | ||
| --- | ||
| ## mass environment preview | ||
|
|
||
| Converge a preview environment from a YAML config | ||
|
|
||
| ### Synopsis | ||
|
|
||
| # Preview Environment | ||
|
|
||
| Converges a preview environment from a YAML config: forks a base environment, | ||
| pins environment defaults, applies per-instance overrides, and triggers a deploy. | ||
|
|
||
| Re-running the command against the same config is safe — every step is | ||
| idempotent. Use it to ramp up a per-PR environment on every git push, and again | ||
| to reset the env back to the declared state. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| mass environment preview <ID> [flags] | ||
| ``` | ||
|
|
||
| ## Arguments | ||
|
|
||
| - `ID`: the local segment of the preview environment's identifier (e.g. | ||
| `pr123`). Must match `^[a-z0-9]{1,20}$` — lowercase alphanumeric only, no | ||
| dashes. The full stored identifier becomes `<project>-<ID>`, where | ||
| `project` comes from the config file. | ||
|
|
||
| ## Flags | ||
|
|
||
| - `--file, -f`: path to the preview YAML config (default `preview.yaml`). | ||
| - `--name, -n`: human-readable environment name (defaults to `ID`). | ||
| - `--description, -d`: optional environment description. | ||
| - `--attributes, -a`: custom attributes for ABAC, e.g. `-a environment=preview,region=uswest`. Overrides `attributes:` in the config file. | ||
| - `--follow`: stream every deployment's logs to stdout until the rollout completes. Each line is prefixed with the instance id so the interleaved output stays grep-friendly when multiple deployments run in parallel. | ||
|
|
||
| ## Environment-variable expansion | ||
|
|
||
| `${VAR}` / `$VAR` references in the config file are expanded from the | ||
| process environment before parsing. Use this for CI-injected values like PR | ||
| numbers: | ||
|
|
||
| ```yaml | ||
| instances: | ||
| chatsvc: | ||
| params: | ||
| host: chatty-pr-${GITHUB_PR}.example.com | ||
| attributes: | ||
| pr: "${GITHUB_PR}" | ||
| ``` | ||
|
|
||
| Undefined variables expand to empty strings. | ||
|
|
||
| ## Config schema | ||
|
|
||
| ```yaml | ||
| # Required: the project the preview env lives in. | ||
| project: demo | ||
|
|
||
| # Required: the local segment of the env to fork from. The full parent | ||
| # identifier is `<project>-<baseEnvironment>`. | ||
| baseEnvironment: production | ||
|
|
||
| # Optional fork-level macros. Defaults to false. | ||
| copyEnvironmentDefaults: true # carry the parent's default resources over | ||
| copySecrets: false # fan copyInstance(copySecrets: true) to every instance | ||
| copyRemoteReferences: false # fan copyInstance(copyRemoteReferences: true) to every instance | ||
|
|
||
| # Optional. Required when the organization declares attributes at the | ||
| # environment scope. Both keys and values must be strings. | ||
| attributes: | ||
| region: us-east-1 | ||
| pr: "${GITHUB_PR}" | ||
|
|
||
| # Optional: pin specific resources as defaults for this env. `resourceType` is | ||
| # documentation for readers; the CLI only needs `resourceId`. | ||
| environmentDefaults: | ||
| - resourceType: aws-iam-role | ||
| resourceId: 161aeb95-e1c5-4f8d-803e-ef82087d7ad4 | ||
|
|
||
| # Optional: per-instance overrides. Listed instances with no fields just | ||
| # inherit from the fork's seed. | ||
| instances: | ||
| chatdb: | ||
| version: "~2.0" # stable channel | ||
|
|
||
| chatsvc: | ||
| version: "latest+dev" # `+dev` pulls from the development channel | ||
| params: | ||
| ingress: | ||
| enabled: true | ||
| host: chatty-pr-${GITHUB_PR}.mdawssbx.com | ||
| path: / | ||
| secrets: | ||
| - name: STRIPE_KEY | ||
| value: FOO | ||
|
|
||
| # listed without overrides — inherit from the fork | ||
| imported: | ||
| sessions: | ||
| sessionsapi: | ||
| sessionsfn: | ||
| sharedvpc: | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| ```bash | ||
| # Converge a preview env for PR 123 from the default `preview.yaml` | ||
| mass environment preview pr123 | ||
|
|
||
| # Same, with a friendly name | ||
| mass environment preview pr123 -n "Chat PR #123" | ||
|
|
||
| # Point at a config in another path | ||
| mass environment preview pr123 -f .github/preview.yml | ||
| ``` | ||
|
|
||
|
|
||
| ``` | ||
| mass environment preview [ID] [flags] | ||
| ``` | ||
|
|
||
| ### Options | ||
|
|
||
| ``` | ||
| -a, --attributes attributes: Custom attributes for ABAC (e.g. -a environment=preview,region=uswest). Overrides attributes: in the config file. (default []) | ||
| -d, --description string Optional environment description | ||
| -f, --file string Path to the preview config YAML (default "preview.yaml") | ||
| --follow Stream every deployment's logs to stdout until the rollout completes. Each line is prefixed with the instance id. | ||
| -h, --help help for preview | ||
| -n, --name string Environment name (defaults to ID if not provided) | ||
| ``` | ||
|
|
||
| ### SEE ALSO | ||
|
|
||
| * [mass environment](/cli/commands/mass_environment) - Environment management |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Example preview environment config. Drive a converge with: | ||
| # | ||
| # mass environment preview pr123 -f preview.yaml | ||
| # | ||
| # Every step is idempotent — re-running the command resets the preview env | ||
| # back to the declared state. | ||
|
|
||
| # The project the preview env lives in. | ||
| project: demo | ||
|
|
||
| # The base env to fork from. The full parent identifier is | ||
| # `<project>-<baseEnvironment>` — `demo-production` here. | ||
| baseEnvironment: production | ||
|
|
||
| # Carry the parent env's default resource connections into the fork. | ||
| copyEnvironmentDefaults: true | ||
|
|
||
| # Optional. Set environment-scope attributes for ABAC. Required when the org | ||
| # declares attributes at the environment scope. `${VAR}` references are | ||
| # expanded from the process environment before parsing — handy for piping | ||
| # CI metadata in without rewriting the file. | ||
| attributes: | ||
| region: us-east-1 | ||
| pr: "${GITHUB_PR}" | ||
|
|
||
| # Override env-level defaults for specific resource types. `resourceType` is | ||
| # documentation for the human reader; the CLI only needs `resourceId`. | ||
| environmentDefaults: | ||
| - resourceType: aws-iam-role | ||
| resourceId: 161aeb95-e1c5-4f8d-803e-ef82087d7ad4 | ||
| - resourceType: aws-vpc | ||
| resourceId: 1e9fc8a3-f011-433f-b937-b5e525fd753c | ||
|
|
||
| # Per-instance overrides. Listed instances with no fields just inherit from | ||
| # the fork's seed. | ||
| instances: | ||
| chatsvc: | ||
| version: "latest+dev" # `+dev` pulls from the development channel | ||
| params: | ||
| ingress: | ||
| enabled: true | ||
| host: chatty-pr-${GITHUB_PR}.mdawssbx.com | ||
| path: / | ||
| secrets: | ||
| - name: STRIPE_KEY | ||
| value: FOO | ||
|
|
||
| chatdb: | ||
| version: "~2.0" # stable channel | ||
|
|
||
| # listed without overrides — inherit from the fork | ||
| imported: | ||
| sessions: | ||
| sessionsapi: | ||
| sessionsfn: | ||
| sharedvpc: |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should include an option to watch for deployments and tail prefixing each line with the instance.id for grepability since some may run in parallel
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in 0c78eb8 — added
--followto bothenvironment preview(here) andenvironment deploy(in #238, since that's where the deploy command lives). Output looks like:Each tail goroutine line-buffers and serializes through a mutex'd sink, so parallel deployments don't tear lines.
One implementation note worth flagging: I went with polling rather than subscribing to
environmentEvents/instanceEventsas you suggested. The SDK's Absinthe/Phoenix-channels machinery is ininternal/absinthe, so doing real subscriptions from the CLI would mean re-porting that layer just for this flag. Polling per-instanceDeployments.Listevery ~2s has a few seconds of discovery latency per new deployment, which seemed acceptable next to per-instance deploy times measured in minutes. Happy to switch to the subscription path the moment the SDK exposes event subscriptions publicly — at that point it's a one-file swap insideinternal/commands/environment/follow.go.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filed the SDK ask: massdriver-cloud/massdriver-sdk-go#21 — request to expose
environmentEvents/instanceEventsas typedSubscribemethods on the existing services. Once that ships, the swap insidefollow.gois one file.