Skip to content

Commit f6bb91f

Browse files
authored
feat: Add experimental "enable Pushpin" mode (#1509)
This PR adds code to the CLI to enable `fastly compute serve` with "enable Pushpin", a new mode that locates and runs Pushpin before starting Viceroy to run guest code. This is a companion PR to fastly/Viceroy#497, and will only function after that feature has landed. ## How to enable it: It can be invoked in one of two ways: 1. set a new flag `--experimental-enable-pushpin` 2. or, set a new flag in `fastly.toml`: ```toml [local_server.pushpin] enable = true ``` ## What it does: * `fastly compute serve` starts Pushpin before starting Viceroy, and detects its startup. It also sets up to shut it down when the CLI process ends. * `pushpin` must be started at this time, because routes are set based on the backends defined in the `fastly.toml` manifest file. * "kill process" is done differently between Windows and Unix, so build flags are used to include the right implementation of `killProcess`. * Pushpin routes are set up based on the `fastly.toml` manifest file. * A route is set up per backend. * The name of each backend is set as an `id=` condition. For example, a backend named `origin` will set `id=origin`. Viceroy handles Fanout handoff by forwarding a request for this backend to Pushpin and set the header `Pushpin-Route: origin`, allowing Pushpin to match the route. * If the backend definition's URL has a path segment, then it is set as a `replace_beg` configuration on the route. For example, if a backend is configured for `http://localhost:3000/realtime`, then this is set as `replace_beg=realtime`. Pushpin will forward a request for `/api/user` to this backend as `http://localhost:3000/realtime/api/user`. * `over_http` is set for every route target, as to always enable WebSocket-over-HTTP. * If the backend URL is HTTPS, then `ssl` is added. * If the backend definition sets `override_host`, then it will be set as the `host` value of the route target. * When starting Viceroy, `--local-pushpin-proxy-port=<port>` is appended as a command-line argument, which starts Viceroy in "enable Pushpin" mode. * Pushpin proxy runs by default on port `7677`, but this can be overridden: * using `proxy_port=<port>` under `local_server.pushpin` section of the `fastly.toml` file, or * using `--pushpin-proxy-port=<port>` * Pushpin's publishing endpoint runs by default on port `5561`, but this can be overridden: * using `publish_port=<port>` under `local_server.pushpin` section of the `fastly.toml` file, or * using `--pushpin-publish-port=<port>` * The `pushpin` binary is searched on the system path, but can be specified: * using `pushpin_path=/path/to/pushpin` under the `local_server.pushpin` section of the `fastly.toml` file, or * using `--pushpin-path=/path/to/pushpin`. * The CLI writes the config and routes to temporary files in the OS temporary directory. This is cleaned up as part of shutdown. * Pushpin runner places some runtime files into a temporary directory specified in pushpin.conf. We set this to an OS temporary directory, so that multiple instances can run if necessary. This is cleaned up as part of shutdown. * Pushpin runner places logs into a directory specified in pushpin.conf. This is placed in the ./pushpin-logs directory under the current project All Submissions: * [x] Have you followed the guidelines in our Contributing document? * [x] Have you checked to ensure there aren't other open [Pull Requests](https://github.com/fastly/cli/pulls) for the same update/change? ### New Feature Submissions: * [x] Does your submission pass tests? ### Changes to Core Features: * [x] Have you added an explanation of what your changes do and why you'd like us to include them? * [ ] Have you written new tests for your core changes, as applicable? * [x] Have you successfully run tests with your changes locally? ### User Impact * [x] What is the user impact of this change? ### Are there any considerations that need to be addressed for release? The new feature is opt-in using the `--experimental-enable-pushpin` flag, and does nothing if the new flag is not set.
1 parent 9371714 commit f6bb91f

9 files changed

Lines changed: 758 additions & 38 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
### Breaking:
66

77
### Enhancements:
8+
- Add experimental "enable Pushpin" mode ([#1509](https://github.com/fastly/cli/pull/1509))
89

910
### Bug fixes:
1011

go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ require (
4343
github.com/dnaeon/go-vcr v1.2.0 // indirect
4444
github.com/go-jose/go-jose/v4 v4.0.5 // indirect
4545
github.com/kr/pretty v0.3.1 // indirect
46+
github.com/mitchellh/go-ps v1.0.0 // indirect
4647
github.com/otiai10/mint v1.6.3 // indirect
4748
github.com/rogpeppe/go-internal v1.14.1 // indirect
4849
github.com/stretchr/testify v1.10.0 // indirect

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,8 @@ github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6T
8787
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
8888
github.com/mholt/archiver/v3 v3.5.1 h1:rDjOBX9JSF5BvoJGvjqK479aL70qh9DIpZCl+k7Clwo=
8989
github.com/mholt/archiver/v3 v3.5.1/go.mod h1:e3dqJ7H78uzsRSEACH1joayhuSyhnonssnDhppzS1L4=
90+
github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc=
91+
github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg=
9092
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
9193
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
9294
github.com/mitchellh/mapstructure v1.5.1-0.20231216201459-8508981c8b6c h1:cqn374mizHuIWj+OSJCajGr/phAmuMug9qIX3l9CflE=

pkg/commands/compute/compute_test.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,8 +101,12 @@ func TestFlagDivergenceServe(t *testing.T) {
101101
ignoreServeFlags := []string{
102102
"addr",
103103
"debug",
104+
"experimental-enable-pushpin",
104105
"file",
105106
"profile-guest",
107+
"pushpin-path",
108+
"pushpin-proxy-port",
109+
"pushpin-publish-port",
106110
"profile-guest-dir",
107111
"skip-build",
108112
"viceroy-args",
Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
[global]
2+
include={libdir}/internal.conf
3+
4+
# directory to save runtime files
5+
rundir=%[1]s
6+
7+
# prefix for zmq ipc specs
8+
ipc_prefix=
9+
10+
# port offset for zmq tcp specs and http control server
11+
port_offset=0
12+
13+
# TTL (seconds) for connection stats
14+
stats_connection_ttl=120
15+
16+
# whether to send individual connection stats
17+
stats_connection_send=true
18+
19+
20+
[runner]
21+
# services to start
22+
services=connmgr,proxy,handler
23+
24+
# plain HTTP port to listen on for client connections
25+
http_port=%[4]d
26+
27+
# list of HTTPS ports to listen on for client connections (you must have certs set)
28+
#https_ports=443
29+
30+
# list of unix socket paths to listen on for client connections
31+
#local_ports={rundir}/{ipc_prefix}server
32+
33+
# directory to save log files
34+
logdir=%[2]s
35+
36+
# logging level. 2 = info, >2 = verbose
37+
log_level=2
38+
39+
# client full request header must fit in this buffer
40+
client_buffer_size=8192
41+
42+
# maximum number of client connections
43+
client_maxconn=50000
44+
45+
# whether connections can use compression
46+
allow_compression=false
47+
48+
# paths
49+
# mongrel2_bin=mongrel2
50+
# m2sh_bin=m2sh
51+
# zurl_bin=zurl
52+
53+
54+
[proxy]
55+
# routes config file (path relative to location of this file)
56+
routesfile=%[3]s
57+
58+
# enable debug mode to get informative error responses
59+
debug=false
60+
61+
# whether to use automatic CORS and JSON-P wrapping
62+
auto_cross_origin=false
63+
64+
# whether to accept x-forwarded-proto
65+
accept_x_forwarded_protocol=false
66+
67+
# whether to assert x-forwarded-proto
68+
set_x_forwarded_protocol=proto-only
69+
70+
# how to treat x-forwarded-for. example: "truncate:0,append"
71+
x_forwarded_for=
72+
73+
# how to treat x-forwarded-for if grip-signed
74+
x_forwarded_for_trusted=
75+
76+
# the following headers must be marked in order to qualify as orig
77+
orig_headers_need_mark=
78+
79+
# whether to accept Pushpin-Route header
80+
accept_pushpin_route=true
81+
82+
# value to append to the CDN-Loop header
83+
cdn_loop=
84+
85+
# include client IP address in logs
86+
log_from=false
87+
88+
# include client user agent in logs
89+
log_user_agent=false
90+
91+
# for signing proxied requests
92+
sig_iss=pushpin
93+
94+
# for signing proxied requests. use "base64:" prefix for binary key
95+
sig_key=changeme
96+
97+
# use this to allow grip to be forwarded upstream (e.g. to fanout.io)
98+
upstream_key=
99+
100+
# for the sockjs iframe transport
101+
sockjs_url=http://cdn.jsdelivr.net/sockjs/0.3.4/sockjs.min.js
102+
103+
# updates check has three modes:
104+
# report: check for new pushpin version and report anonymous usage info to
105+
# the pushpin developers
106+
# check: check for new pushpin version only, don't report anything
107+
# off: don't do any reporting or checking
108+
# pushpin will output a log message when a new version is available. report
109+
# mode helps the pushpin project build credibility, so please enable it if you
110+
# enjoy this software :)
111+
updates_check=report
112+
113+
# use this field to identify your organization in updates requests. if left
114+
# blank, updates requests will be anonymous
115+
organization_name=
116+
117+
118+
[handler]
119+
# ipc permissions (octal)
120+
#ipc_file_mode=777
121+
122+
# bind PULL for receiving publish commands
123+
push_in_spec=tcp://127.0.0.1:%[6]d
124+
125+
# list of bind SUB for receiving published messages
126+
push_in_sub_specs=tcp://127.0.0.1:%[7]d
127+
128+
# whether the above SUB socket should connect instead of bind
129+
push_in_sub_connect=false
130+
131+
# addr/port to listen on for receiving publish commands via HTTP
132+
push_in_http_addr=0.0.0.0
133+
push_in_http_port=%[5]d
134+
135+
# maximum headers and body size in bytes when receiving publish commands via HTTP
136+
push_in_http_max_headers_size=8192
137+
push_in_http_max_body_size=65536
138+
139+
# bind PUB for sending stats (metrics, subscription info, etc)
140+
stats_spec=ipc://{rundir}/{ipc_prefix}stats
141+
142+
# bind REP for responding to commands
143+
command_spec=tcp://127.0.0.1:%[8]d
144+
145+
# max messages per second
146+
message_rate=2500
147+
148+
# max rate-limited messages
149+
message_hwm=25000
150+
151+
# set to report blocks counts in stats (content size / block size)
152+
#message_block_size=
153+
154+
# max time (milliseconds) for out-of-order messages to wait
155+
message_wait=5000
156+
157+
# time (seconds) to cache message ids
158+
id_cache_ttl=60
159+
160+
# retry/recover sessions soon after the first subscription to a channel
161+
update_on_first_subscription=true
162+
163+
# max subscriptions per connection
164+
connection_subscription_max=20
165+
166+
# time (seconds) to linger response mode subscriptions
167+
subscription_linger=60
168+
169+
# TTL (seconds) for subscription stats
170+
stats_subscription_ttl=60
171+
172+
# interval (seconds) to send report stats
173+
stats_report_interval=10
174+
175+
# stats output format
176+
stats_format=tnetstring

0 commit comments

Comments
 (0)