You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/guide/8-edge.md
+40Lines changed: 40 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,6 +38,46 @@ python your_app.py
38
38
In the above example, the `edge-endpoint` is running on the same machine as the application, so the endpoint URL is `http://localhost:30101`. If the `edge-endpoint` is running on a different machine, you should replace `localhost` with the IP address or hostname of the machine running the `edge-endpoint`.
39
39
:::
40
40
41
+
## Configuring the Edge Endpoint at Runtime
42
+
43
+
:::note
44
+
Runtime edge configuration is currently in beta and available through the `ExperimentalApi`.
45
+
:::
46
+
47
+
You can programmatically configure which detectors run on the edge and how they behave, without redeploying.
48
+
49
+
This allows applications to define the desired state of the Edge Endpoint, thereby eliminating the need to manually configure the Edge Endpoint separately.
50
+
51
+
```python notest
52
+
from groundlight import ExperimentalApi
53
+
from groundlight.edge import EdgeEndpointConfig
54
+
from groundlight.edge.config importNO_CLOUD, EDGE_ANSWERS_WITH_ESCALATION
# Apply the configuration and wait for detectors to be ready
65
+
print("Applying configuration...")
66
+
config = gl.edge.set_config(config)
67
+
print(f"Applied config with {len(config.detectors)} detector(s)")
68
+
```
69
+
70
+
`set_config` replaces the current configuration and blocks until all detectors have inference pods ready to serve requests (or until the timeout expires).
We have benchmarked the `edge-endpoint` handling 500 requests/sec at a latency of less than 50ms on an off-the-shelf [Katana 15 B13VGK-1007US](https://us.msi.com/Laptop/Katana-15-B13VX/Specification) laptop (Intel® Core™ i9-13900H CPU, NVIDIA® GeForce RTX™ 4070 Laptop GPU, 32GB DDR5 5200MHz RAM) running Ubuntu 20.04.
0 commit comments