diff --git a/architecture/06-cli.md b/architecture/06-cli.md index b47a67ee61..80b6f64d52 100644 --- a/architecture/06-cli.md +++ b/architecture/06-cli.md @@ -6,15 +6,16 @@ The Cog CLI is a Go binary that provides commands for the full model lifecycle: ## Commands Overview -| Command | Job To Be Done | -| ----------- | ------------------------------------- | -| `cog init` | Bootstrap a new model project | -| `cog build` | Create a container image | -| `cog run` | Run a prediction in a container | -| `cog exec` | Run arbitrary commands in a container | -| `cog serve` | Start HTTP server in a container | -| `cog push` | Deploy to Replicate | -| `cog login` | Authenticate with Replicate | +| Command | Job To Be Done | +| ---------------- | ------------------------------------- | +| `cog init` | Bootstrap a new model project | +| `cog build` | Create a container image | +| `cog run` | Run a prediction in a container | +| `cog exec` | Run arbitrary commands in a container | +| `cog serve` | Start HTTP server in a container | +| `cog playground` | Browser UI to talk to a running model | +| `cog push` | Deploy to Replicate | +| `cog login` | Authenticate with Replicate | ## Development Commands @@ -98,6 +99,44 @@ Builds the image (if needed) and starts a container running the [Container Runti **Code**: `pkg/cli/serve.go` +--- + +### cog playground + +**Job**: Open a browser UI for talking to a running model. + +```bash +cog serve -p 8393 # terminal 1: start the model API +cog playground # terminal 2: opens the UI in your browser +``` + +Unlike the other commands, `cog playground` doesn't build an image or run model code. It starts a small Go web server that serves a schema-driven browser UI -- a Postman-like tool for Cog models -- and reverse-proxies requests to a _separate_ running model API, typically one started with `cog serve`. The UI reflects the model's [Schema](./02-schema.md) from `/openapi.json` and lets you run sync, streaming (SSE), and async predictions with either a generated form or raw JSON input. + +The browser only ever talks to the playground's own origin, which forwards to the target API chosen at runtime (via an `X-Cog-Target` header). Proxying sidesteps CORS -- the model API sets none -- and keeps SSE streaming intact. Async predictions have no GET-by-id endpoint, so the server also hosts a webhook sink and relays delivered events back to the browser over its own SSE stream. + +```mermaid +flowchart LR + Browser["Browser UI"] + + subgraph Playground["cog playground (host)"] + Static["Static UI assets
(go:embed)"] + Proxy["Reverse proxy
/proxy/*"] + Sink["Webhook sink + relay
/webhook/{token} → /events"] + end + + Model["Target model API
(e.g. cog serve)"] + + Browser -->|"load UI"| Static + Browser -->|"schema, predictions, SSE"| Proxy + Proxy -->|"X-Cog-Target"| Model + Model -->|"webhook (async)"| Sink + Sink -->|"SSE events"| Browser +``` + +The UI is plain HTML/JS (no build step); JSON is edited and displayed with a vendored Ace editor. Assets are compiled into the binary with `go:embed`. + +**Code**: `pkg/cli/playground.go` (server, reverse proxy, webhook sink); `pkg/cli/playground/` (embedded UI assets) + ## Build Commands ### cog build @@ -219,9 +258,11 @@ pkg/cli/ ├── predict.go # prediction execution and legacy cog predict ├── exec.go # cog exec ├── serve.go # cog serve +├── playground.go # cog playground (UI server, reverse proxy, webhook sink) ├── push.go # cog push ├── login.go # cog login -└── init.go # cog init +├── init.go # cog init +└── playground/ # embedded playground UI assets (go:embed) ``` Commands delegate to packages under `pkg/`: diff --git a/docs/cli.md b/docs/cli.md index 9f472fae32..cddcb4caf2 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -175,6 +175,49 @@ cog login [flags] --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token ``` +## `cog playground` + +Open a browser playground for talking to a running model. + +Starts a local web server that serves a schema-driven UI (a Postman-like tool +for Cog models). Point it at any running Cog HTTP API -- for example one started +with 'cog serve' -- and the playground reflects that model's inputs and outputs +from its OpenAPI schema in real time. + +Requests are reverse-proxied through this server, so the target API does not +need to set CORS headers. The server also hosts a webhook sink so async +predictions can be observed in the browser. + +Async/webhook testing against a containerized model requires the webhook URL to +be reachable from inside the container. On Docker Desktop the default +'host.docker.internal' works once the server listens on a reachable interface +(e.g. --host 0.0.0.0). + +``` +cog playground [flags] +``` + +**Examples** + +``` + # Start a model API in one terminal + cog serve -p 8393 + + # Open the playground pointing at it + cog playground --target http://localhost:8393 +``` + +**Options** + +``` + -h, --help help for playground + --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default "127.0.0.1") + --no-open Do not open the browser automatically + -p, --port int Port to listen on (0 picks a free port) + --target string Default target model API URL (default "http://localhost:8393") + --webhook-host string Hostname the model uses to reach this server for webhooks (default "host.docker.internal") +``` + ## `cog push` Build a Docker image from cog.yaml and push it to a container registry. diff --git a/docs/llms.txt b/docs/llms.txt index 11cb30b033..3fb099e102 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -421,6 +421,49 @@ cog login [flags] --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token ``` +## `cog playground` + +Open a browser playground for talking to a running model. + +Starts a local web server that serves a schema-driven UI (a Postman-like tool +for Cog models). Point it at any running Cog HTTP API -- for example one started +with 'cog serve' -- and the playground reflects that model's inputs and outputs +from its OpenAPI schema in real time. + +Requests are reverse-proxied through this server, so the target API does not +need to set CORS headers. The server also hosts a webhook sink so async +predictions can be observed in the browser. + +Async/webhook testing against a containerized model requires the webhook URL to +be reachable from inside the container. On Docker Desktop the default +'host.docker.internal' works once the server listens on a reachable interface +(e.g. --host 0.0.0.0). + +``` +cog playground [flags] +``` + +**Examples** + +``` + # Start a model API in one terminal + cog serve -p 8393 + + # Open the playground pointing at it + cog playground --target http://localhost:8393 +``` + +**Options** + +``` + -h, --help help for playground + --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default "127.0.0.1") + --no-open Do not open the browser automatically + -p, --port int Port to listen on (0 picks a free port) + --target string Default target model API URL (default "http://localhost:8393") + --webhook-host string Hostname the model uses to reach this server for webhooks (default "host.docker.internal") +``` + ## `cog push` Build a Docker image from cog.yaml and push it to a container registry. @@ -2171,6 +2214,75 @@ class Runner(BaseRunner): ``` +--- + +# Playground + +`cog playground` opens a browser UI for talking to a running Cog model — a Postman-like tool that reflects your model's inputs and outputs from its OpenAPI schema and lets you run predictions interactively. + +It doesn't build an image or run your model. Point it at a model API you're already running — typically [`cog serve`](cli.md#cog-serve) — and it proxies requests to that API. + +## Quick start + +Start your model's HTTP server in one terminal: + +```sh +cog serve -p 8393 +``` + +Open the playground in another: + +```sh +cog playground --target http://localhost:8393 +``` + +This serves the UI on a local port and opens it in your browser. You can change the target API from the UI at any time. + +## What you can do + +- **Schema-driven form.** Inputs render from the model's `/openapi.json` as the appropriate widgets (text, number, boolean, enum, list, file, secret). Optional fields without a default start unchecked so they're omitted. +- **Form or JSON.** Toggle between the generated form and a JSON editor; the two stay in sync. +- **Files by upload or URL.** File inputs accept an uploaded file (sent as a data URI) or a URL, with an inline preview for images, audio, and video. +- **Sync, streaming, or async.** Run modes appear based on what the model supports — streaming (SSE) when the predictor uses `@cog.streaming`, and async via webhooks. +- **Rendered or raw output.** View the rendered result (media, text, JSON) or switch to **Raw** to see exactly what arrived over the wire. A Copy button grabs the whole payload. + +## Options + +| Flag | Description | +| ---------------- | ---------------------------------------------------------------------------------------------- | +| `--target` | Default model API URL (also changeable in the UI). Defaults to `http://localhost:8393`. | +| `-p, --port` | Port to listen on. `0` (default) picks a free port. | +| `--host` | Address to bind. Use `0.0.0.0` to receive webhooks from a containerized model. | +| `--webhook-host` | Hostname the model uses to reach the playground for webhooks (default `host.docker.internal`). | +| `--no-open` | Don't open the browser automatically. | + +## CORS and webhooks + +Requests are reverse-proxied through the playground, so the model API doesn't need to send any CORS headers. + +[Async predictions](http.md#webhooks) are observed via webhooks (there's no status-polling endpoint), so the playground hosts a webhook sink and relays events to the browser. For this to work against a model running in a container, the playground must be reachable from inside the container: + +```sh +cog playground --host 0.0.0.0 --webhook-host host.docker.internal +``` + +> [!NOTE] +> Sync and streaming predictions work without any of this — the webhook setup is only needed for async runs. + +## Remote models + +If your model runs on another machine, forward its port over SSH and point the playground at it: + +```sh +ssh -L 8393:localhost:5000 user@remote +cog playground --target http://localhost:8393 +``` + +Sync and streaming work over the tunnel. For async/webhooks, run the playground next to the model on the remote and forward only the UI port instead. + +See the [CLI reference](cli.md#cog-playground) for the full list of flags. + + --- # Private package registry diff --git a/docs/playground.md b/docs/playground.md new file mode 100644 index 0000000000..58794f321f --- /dev/null +++ b/docs/playground.md @@ -0,0 +1,65 @@ +# Playground + +`cog playground` opens a browser UI for talking to a running Cog model — a Postman-like tool that reflects your model's inputs and outputs from its OpenAPI schema and lets you run predictions interactively. + +It doesn't build an image or run your model. Point it at a model API you're already running — typically [`cog serve`](cli.md#cog-serve) — and it proxies requests to that API. + +## Quick start + +Start your model's HTTP server in one terminal: + +```sh +cog serve -p 8393 +``` + +Open the playground in another: + +```sh +cog playground --target http://localhost:8393 +``` + +This serves the UI on a local port and opens it in your browser. You can change the target API from the UI at any time. + +## What you can do + +- **Schema-driven form.** Inputs render from the model's `/openapi.json` as the appropriate widgets (text, number, boolean, enum, list, file, secret). Optional fields without a default start unchecked so they're omitted. +- **Form or JSON.** Toggle between the generated form and a JSON editor; the two stay in sync. +- **Files by upload or URL.** File inputs accept an uploaded file (sent as a data URI) or a URL, with an inline preview for images, audio, and video. +- **Sync, streaming, or async.** Run modes appear based on what the model supports — streaming (SSE) when the predictor uses `@cog.streaming`, and async via webhooks. +- **Rendered or raw output.** View the rendered result (media, text, JSON) or switch to **Raw** to see exactly what arrived over the wire. A Copy button grabs the whole payload. + +## Options + +| Flag | Description | +| ---------------- | ---------------------------------------------------------------------------------------------- | +| `--target` | Default model API URL (also changeable in the UI). Defaults to `http://localhost:8393`. | +| `-p, --port` | Port to listen on. `0` (default) picks a free port. | +| `--host` | Address to bind. Use `0.0.0.0` to receive webhooks from a containerized model. | +| `--webhook-host` | Hostname the model uses to reach the playground for webhooks (default `host.docker.internal`). | +| `--no-open` | Don't open the browser automatically. | + +## CORS and webhooks + +Requests are reverse-proxied through the playground, so the model API doesn't need to send any CORS headers. + +[Async predictions](http.md#webhooks) are observed via webhooks (there's no status-polling endpoint), so the playground hosts a webhook sink and relays events to the browser. For this to work against a model running in a container, the playground must be reachable from inside the container: + +```sh +cog playground --host 0.0.0.0 --webhook-host host.docker.internal +``` + +> [!NOTE] +> Sync and streaming predictions work without any of this — the webhook setup is only needed for async runs. + +## Remote models + +If your model runs on another machine, forward its port over SSH and point the playground at it: + +```sh +ssh -L 8393:localhost:5000 user@remote +cog playground --target http://localhost:8393 +``` + +Sync and streaming work over the tunnel. For async/webhooks, run the playground next to the model on the remote and forward only the UI port instead. + +See the [CLI reference](cli.md#cog-playground) for the full list of flags. diff --git a/examples/streaming-text/README.md b/examples/streaming-text/README.md index 77265e6f1e..9d7953c4b4 100644 --- a/examples/streaming-text/README.md +++ b/examples/streaming-text/README.md @@ -9,7 +9,7 @@ This example shows how a Cog runner can yield text chunks as a model generates t From this directory: ```sh -cog predict -i prompt="Write a short haiku about databases" +cog run -i prompt="Write a short haiku about databases" ``` This returns the final accumulated output after the prediction completes. @@ -46,6 +46,6 @@ data: {"id":"streaming-demo","status":"succeeded",...} ## How it works -`predict.py` defines `run() -> Iterator[str]`. Each `yield` becomes one streamed output chunk. The example uses Hugging Face `TextIteratorStreamer` to receive generated text from `model.generate()` while generation is still running. +`run.py` defines `run() -> Iterator[str]`. Each `yield` becomes one streamed output chunk. The example uses Hugging Face `TextIteratorStreamer` to receive generated text from `model.generate()` while generation is still running. The normal prediction response still contains the accumulated output for compatibility. Requesting `Accept: text/event-stream` is useful when clients want to display tokens as they arrive. diff --git a/examples/streaming-text/requirements.txt b/examples/streaming-text/requirements.txt index a3ba48567c..334d2c30b9 100644 --- a/examples/streaming-text/requirements.txt +++ b/examples/streaming-text/requirements.txt @@ -1,3 +1,3 @@ -torch==2.12.0 +torch==2.8.0 transformers==5.0.0rc3 accelerate==1.6.0 diff --git a/examples/streaming-text/run.py b/examples/streaming-text/run.py index b51bee3332..cebde0be00 100644 --- a/examples/streaming-text/run.py +++ b/examples/streaming-text/run.py @@ -12,12 +12,12 @@ class Runner(BaseRunner): def setup(self) -> None: self.device = "cuda" if torch.cuda.is_available() else "cpu" - dtype = torch.float16 if self.device == "cuda" else torch.float32 + dtype = torch.bfloat16 if self.device == "cuda" else torch.float32 self.tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) self.model = AutoModelForCausalLM.from_pretrained( MODEL_NAME, - torch_dtype=dtype, + dtype=dtype, ).to(self.device) self.model.eval() diff --git a/mkdocs.yml b/mkdocs.yml index 6c394d3c31..7fbb47fbac 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -11,6 +11,7 @@ nav: - Prediction API: python.md - Training API: training.md - HTTP API: http.md + - Playground: playground.md - CLI: cli.md - Environment variables: environment.md - Private registry: private-package-registry.md diff --git a/pkg/cli/playground.go b/pkg/cli/playground.go new file mode 100644 index 0000000000..bd468da624 --- /dev/null +++ b/pkg/cli/playground.go @@ -0,0 +1,333 @@ +package cli + +import ( + "context" + "embed" + "encoding/json" + "errors" + "fmt" + "io" + "io/fs" + "net" + "net/http" + "net/http/httputil" + "net/url" + "os/exec" + "runtime" + "strings" + "sync" + "time" + + "github.com/spf13/cobra" + + "github.com/replicate/cog/pkg/util/console" +) + +//go:embed playground +var playgroundUI embed.FS + +// maxWebhookBody caps a single webhook payload relayed to the browser. +const maxWebhookBody = 10 * 1024 * 1024 + +var ( + playgroundPort = 0 + playgroundTarget = "http://localhost:8393" + playgroundHost = "127.0.0.1" + playgroundWebhookHost = "host.docker.internal" + playgroundNoOpen = false +) + +func newPlaygroundCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "playground", + Short: "Open a browser playground for talking to a running model", + Long: `Open a browser playground for talking to a running model. + +Starts a local web server that serves a schema-driven UI (a Postman-like tool +for Cog models). Point it at any running Cog HTTP API -- for example one started +with 'cog serve' -- and the playground reflects that model's inputs and outputs +from its OpenAPI schema in real time. + +Requests are reverse-proxied through this server, so the target API does not +need to set CORS headers. The server also hosts a webhook sink so async +predictions can be observed in the browser. + +Async/webhook testing against a containerized model requires the webhook URL to +be reachable from inside the container. On Docker Desktop the default +'host.docker.internal' works once the server listens on a reachable interface +(e.g. --host 0.0.0.0).`, + Example: ` # Start a model API in one terminal + cog serve -p 8393 + + # Open the playground pointing at it + cog playground --target http://localhost:8393`, + RunE: cmdPlayground, + Args: cobra.MaximumNArgs(0), + SuggestFor: []string{"ui", "gui"}, + } + + cmd.Flags().IntVarP(&playgroundPort, "port", "p", playgroundPort, "Port to listen on (0 picks a free port)") + cmd.Flags().StringVar(&playgroundTarget, "target", playgroundTarget, "Default target model API URL") + cmd.Flags().StringVar(&playgroundHost, "host", playgroundHost, "Address to bind (use 0.0.0.0 to receive webhooks from containers)") + cmd.Flags().StringVar(&playgroundWebhookHost, "webhook-host", playgroundWebhookHost, "Hostname the model uses to reach this server for webhooks") + cmd.Flags().BoolVar(&playgroundNoOpen, "no-open", playgroundNoOpen, "Do not open the browser automatically") + + return cmd +} + +// playgroundServer holds the runtime state for a playground instance. +type playgroundServer struct { + hub *eventHub + webhookBase string +} + +func cmdPlayground(cmd *cobra.Command, _ []string) error { + uiFS, err := fs.Sub(playgroundUI, "playground") + if err != nil { + return fmt.Errorf("loading playground assets: %w", err) + } + + ln, err := net.Listen("tcp", fmt.Sprintf("%s:%d", playgroundHost, playgroundPort)) + if err != nil { + return fmt.Errorf("starting playground server: %w", err) + } + port := ln.Addr().(*net.TCPAddr).Port + + srvState := &playgroundServer{ + hub: newEventHub(), + webhookBase: fmt.Sprintf("http://%s:%d", playgroundWebhookHost, port), + } + + mux := srvState.routes(uiFS) + + browserHost := playgroundHost + if browserHost == "0.0.0.0" || browserHost == "" { + browserHost = "127.0.0.1" + } + uiURL := fmt.Sprintf("http://%s:%d/?target=%s", browserHost, port, url.QueryEscape(playgroundTarget)) + console.Infof("Cog playground running at %s", uiURL) + console.Info("Press Ctrl+C to stop.") + if !playgroundNoOpen { + maybeOpenBrowser(uiURL) + } + + srv := &http.Server{ + Handler: mux, + ReadHeaderTimeout: 10 * time.Second, + } + + // Shut down gracefully when the command's context is canceled (Ctrl+C). + go func() { + <-cmd.Context().Done() + shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + _ = srv.Shutdown(shutdownCtx) + }() + + if err := srv.Serve(ln); err != nil && !errors.Is(err, http.ErrServerClosed) { + return err + } + return nil +} + +// routes builds the HTTP handler: static UI, the reverse proxy, and the webhook +// sink + event relay. +func (s *playgroundServer) routes(uiFS fs.FS) *http.ServeMux { + mux := http.NewServeMux() + mux.Handle("/", http.FileServerFS(uiFS)) + mux.HandleFunc("/proxy/", handlePlaygroundProxy) + mux.HandleFunc("/webhook/", s.handleWebhook) + mux.HandleFunc("/events", s.handleEvents) + mux.HandleFunc("/config", s.handleConfig) + return mux +} + +// handleConfig reports runtime configuration the UI needs, notably the webhook +// base URL the model should call back on. +func (s *playgroundServer) handleConfig(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]string{"webhookBase": s.webhookBase}) +} + +// handleWebhook receives a webhook delivery from a model and relays its body to +// any browser subscribed to the matching token's event stream. +func (s *playgroundServer) handleWebhook(w http.ResponseWriter, r *http.Request) { + token := strings.TrimPrefix(r.URL.Path, "/webhook/") + if token == "" { + http.Error(w, "missing token", http.StatusNotFound) + return + } + body, err := io.ReadAll(io.LimitReader(r.Body, maxWebhookBody)) + if err != nil { + http.Error(w, "cannot read webhook body", http.StatusBadRequest) + return + } + s.hub.publish(token, body) + w.Header().Set("Content-Type", "application/json") + _, _ = fmt.Fprint(w, "{}") +} + +// handleEvents streams relayed webhook payloads to the browser over SSE. +func (s *playgroundServer) handleEvents(w http.ResponseWriter, r *http.Request) { + token := r.URL.Query().Get("token") + if token == "" { + http.Error(w, "missing token", http.StatusBadRequest) + return + } + flusher, ok := w.(http.Flusher) + if !ok { + http.Error(w, "streaming unsupported", http.StatusInternalServerError) + return + } + + // Subscribe before flushing headers so the caller is guaranteed to be + // receiving by the time it observes the response (no missed events). + ch := s.hub.subscribe(token) + defer s.hub.unsubscribe(token, ch) + + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Cache-Control", "no-cache") + w.Header().Set("Connection", "keep-alive") + flusher.Flush() + + keepAlive := time.NewTicker(15 * time.Second) + defer keepAlive.Stop() + ctx := r.Context() + for { + select { + case <-ctx.Done(): + return + case msg := <-ch: + writeSSEData(w, msg) + flusher.Flush() + case <-keepAlive.C: + _, _ = fmt.Fprint(w, ": keep-alive\n\n") + flusher.Flush() + } + } +} + +// writeSSEData emits a payload as a single SSE event, prefixing every line with +// "data: ". This preserves embedded newlines without letting them terminate the +// event early or inject additional SSE fields (e.g. a spoofed "event:" line). +func writeSSEData(w io.Writer, msg []byte) { + for line := range strings.SplitSeq(string(msg), "\n") { + _, _ = fmt.Fprintf(w, "data: %s\n", line) + } + _, _ = fmt.Fprint(w, "\n") +} + +// handlePlaygroundProxy reverse-proxies /proxy/* to the target model API. The +// target origin is taken from the X-Cog-Target header (set by fetch requests) +// or the cog_target query parameter (used for plain navigations like the schema +// link). Proxying keeps the browser same-origin, sidestepping CORS, and streams +// SSE responses through unbuffered. +func handlePlaygroundProxy(w http.ResponseWriter, r *http.Request) { + rawTarget := r.Header.Get("X-Cog-Target") + if rawTarget == "" { + rawTarget = r.URL.Query().Get("cog_target") + } + if rawTarget == "" { + writeProxyError(w, http.StatusBadRequest, "no target API set") + return + } + + target, err := url.Parse(strings.TrimRight(rawTarget, "/")) + if err != nil || target.Host == "" || (target.Scheme != "http" && target.Scheme != "https") { + writeProxyError(w, http.StatusBadRequest, "invalid target API URL") + return + } + + proxy := &httputil.ReverseProxy{ + FlushInterval: -1, // flush immediately so SSE streams in real time + Rewrite: func(pr *httputil.ProxyRequest) { + // Forward the path after /proxy, dropping the cog_target hint. + path := strings.TrimPrefix(pr.In.URL.Path, "/proxy") + if path == "" { + path = "/" + } + pr.Out.URL.Path = path + query := pr.Out.URL.Query() + query.Del("cog_target") + pr.Out.URL.RawQuery = query.Encode() + pr.SetURL(target) + pr.Out.Host = target.Host + pr.Out.Header.Del("X-Cog-Target") + }, + ErrorHandler: func(w http.ResponseWriter, _ *http.Request, err error) { + writeProxyError(w, http.StatusBadGateway, "cannot reach target API: "+err.Error()) + }, + } + // The proxy target is user-specified by design (a local model API); SSRF to + // it is the intended behavior of this dev tool, not a vulnerability. + proxy.ServeHTTP(w, r) //nolint:gosec // user-directed proxy target is intentional +} + +func writeProxyError(w http.ResponseWriter, status int, message string) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + // message is server-controlled text; encode minimally as a JSON string. + _, _ = fmt.Fprintf(w, `{"error":%q}`, message) +} + +// maybeOpenBrowser best-effort opens a URL in the default browser. +func maybeOpenBrowser(target string) { + var cmd *exec.Cmd + switch runtime.GOOS { + case "darwin": + cmd = exec.Command("open", target) + case "windows": + cmd = exec.Command("rundll32", "url.dll,FileProtocolHandler", target) + default: + cmd = exec.Command("xdg-open", target) + } + _ = cmd.Start() +} + +// eventHub fans out relayed webhook payloads to browser SSE subscribers keyed +// by an opaque token. +type eventHub struct { + mu sync.Mutex + subs map[string]map[chan []byte]struct{} +} + +func newEventHub() *eventHub { + return &eventHub{subs: make(map[string]map[chan []byte]struct{})} +} + +func (h *eventHub) subscribe(token string) chan []byte { + h.mu.Lock() + defer h.mu.Unlock() + ch := make(chan []byte, 32) + if h.subs[token] == nil { + h.subs[token] = make(map[chan []byte]struct{}) + } + h.subs[token][ch] = struct{}{} + return ch +} + +func (h *eventHub) unsubscribe(token string, ch chan []byte) { + h.mu.Lock() + defer h.mu.Unlock() + subs := h.subs[token] + if subs == nil { + return + } + delete(subs, ch) + close(ch) + if len(subs) == 0 { + delete(h.subs, token) + } +} + +func (h *eventHub) publish(token string, msg []byte) { + h.mu.Lock() + defer h.mu.Unlock() + for ch := range h.subs[token] { + // Non-blocking: drop for a slow subscriber rather than stall the model. + select { + case ch <- msg: + default: + } + } +} diff --git a/pkg/cli/playground/api.js b/pkg/cli/playground/api.js new file mode 100644 index 0000000000..29df38a001 --- /dev/null +++ b/pkg/cli/playground/api.js @@ -0,0 +1,191 @@ +// CogApi talks to the target model only through the playground's own reverse +// proxy (same origin). Every request carries the chosen target base URL in the +// X-Cog-Target header; the Go server forwards it. This avoids CORS (Cog sets +// none) and keeps SSE streaming working. + +const PROXY_PREFIX = "/proxy"; + +export class CogApi { + constructor() { + this.target = ""; + } + + setTarget(url) { + this.target = (url || "").trim().replace(/\/+$/, ""); + } + + _headers(extra) { + return Object.assign({ "X-Cog-Target": this.target }, extra || {}); + } + + _url(endpoint, id) { + return id + ? `${PROXY_PREFIX}${endpoint}/${encodeURIComponent(id)}` + : PROXY_PREFIX + endpoint; + } + + _body(input, webhook, webhookFilter) { + const body = { input }; + if (webhook) { + body.webhook = webhook; + body.webhook_events_filter = webhookFilter; + } + return body; + } + + // getConfig returns playground server config (e.g. the webhook base URL). + async getConfig() { + try { + const r = await fetch("/config"); + if (r.ok) return r.json(); + } catch { + /* ignore */ + } + return {}; + } + + async health() { + const r = await fetch(PROXY_PREFIX + "/health-check", { + headers: this._headers(), + }); + if (!r.ok) throw new Error("HTTP " + r.status); + return r.json(); + } + + async schema() { + const r = await fetch(PROXY_PREFIX + "/openapi.json", { + headers: this._headers(), + }); + if (!r.ok) throw new Error("HTTP " + r.status); + return r.json(); + } + + // submit runs a prediction/training in blocking (sync) or async mode. A + // non-empty `id` makes the request idempotent (PUT). Returns the response + // envelope (the 202 acknowledgement in async mode). + async submit({ endpoint, id, input, asyncMode, webhook, webhookFilter, signal }) { + const headers = this._headers({ "Content-Type": "application/json" }); + if (asyncMode) headers["Prefer"] = "respond-async"; + const r = await fetch(this._url(endpoint, id), { + method: id ? "PUT" : "POST", + headers, + body: JSON.stringify(this._body(input, webhook, webhookFilter)), + signal, + }); + const body = await r.json().catch(() => ({})); + if (!r.ok) throw httpError(r.status, body); + return body; + } + + // stream runs a prediction in SSE mode, yielding parsed { type, data } events. + async *stream({ endpoint, id, input, webhook, webhookFilter, signal }) { + const resp = await fetch(this._url(endpoint, id), { + method: id ? "PUT" : "POST", + headers: this._headers({ + "Content-Type": "application/json", + Accept: "text/event-stream", + }), + body: JSON.stringify(this._body(input, webhook, webhookFilter)), + signal, + }); + if (!resp.ok) { + const text = await resp.text(); + let body = {}; + try { + body = JSON.parse(text); + } catch { + /* not JSON */ + } + throw httpError(resp.status, body, text); + } + + const reader = resp.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + buffer += decoder.decode(value, { stream: true }); + let sep; + while ((sep = buffer.indexOf("\n\n")) >= 0) { + const raw = buffer.slice(0, sep); + buffer = buffer.slice(sep + 2); + const event = parseSSEEvent(raw); + if (event) { + event.raw = raw; + yield event; + } + } + } + if (buffer.trim()) { + const event = parseSSEEvent(buffer); + if (event) { + event.raw = buffer; + yield event; + } + } + } + + // cancel requests cancellation of an in-flight prediction/training by id. + async cancel(endpoint, id) { + await fetch(`${PROXY_PREFIX}${endpoint}/${encodeURIComponent(id)}/cancel`, { + method: "POST", + headers: this._headers(), + }); + } +} + +// httpError builds an Error from a non-2xx response, attaching the structured +// `detail` array (422 validation errors) when present so callers can render +// field-level messages. +function httpError(status, body, fallbackText) { + const detail = body && Array.isArray(body.detail) ? body.detail : null; + const message = + (body && (body.error || (typeof body.detail === "string" ? body.detail : null))) || + fallbackText || + "HTTP " + status; + const err = new Error(message); + err.status = status; + if (detail) err.detail = detail; + return err; +} + +// parseSSEEvent parses one "event: ...\ndata: ..." block. The data payload is +// JSON-decoded when possible. +export function parseSSEEvent(block) { + let eventType = ""; + const dataLines = []; + for (const line of block.split("\n")) { + if (line.startsWith("event:")) { + eventType = line.slice(6).trim(); + } else if (line.startsWith("data:")) { + dataLines.push(line.slice(5).replace(/^ /, "")); + } + } + if (!eventType) return null; + const dataStr = dataLines.join("\n"); + let data = dataStr; + try { + data = JSON.parse(dataStr); + } catch { + /* keep raw string */ + } + return { type: eventType, data }; +} + +// fileToDataURI reads a File into a base64 data: URI suitable for a cog.Path +// input. +export function fileToDataURI(file) { + return new Promise((resolve, reject) => { + const reader = new FileReader(); + reader.onload = () => resolve(reader.result); + reader.onerror = reject; + reader.readAsDataURL(file); + }); +} + +export function formatBytes(bytes) { + if (bytes < 1024) return bytes + " B"; + if (bytes < 1048576) return (bytes / 1024).toFixed(1) + " KB"; + return (bytes / 1048576).toFixed(1) + " MB"; +} diff --git a/pkg/cli/playground/app.js b/pkg/cli/playground/app.js new file mode 100644 index 0000000000..e62fef5c0e --- /dev/null +++ b/pkg/cli/playground/app.js @@ -0,0 +1,587 @@ +import { el, clear } from "./dom.js"; +import { CogApi } from "./api.js"; +import { buildForm } from "./form.js"; +import { resolveRef, defaultInput } from "./schema.js"; +import { toggleTheme, currentTheme } from "./theme.js"; +import { + createJSONEditor, + destroyEditor, + refreshEditorThemes, + formatEditor, +} from "./editor.js"; +import { + setBadge, + showError, + clearError, + renderValidationErrors, + renderMetrics, + renderOutput, + renderText, +} from "./output.js"; + +const DEFAULT_TARGET = "http://localhost:8393"; +const STORAGE_KEY = "cog-playground-target"; +const TERMINAL = ["succeeded", "failed", "canceled"]; + +const api = new CogApi(); + +const state = { + schema: null, + inputSchema: null, + outputSchema: null, + supportsStreaming: false, + supportsAsync: false, + form: null, + mode: "form", // "form" | "json" + runMode: "sync", // "sync" | "stream" | "async" + running: false, + abort: null, + eventSource: null, + lastId: null, + metrics: {}, + loadToken: 0, + healthTimer: null, + showLive: false, // a run is driving the toggled output view + outputValue: null, // current output (array of chunks, scalar, or object) + rawEvents: [], // raw frames/payloads exactly as received, for the Raw view + outputView: "text", // "text" | "raw" + webhookBase: "", + jsonEditor: null, // Ace editor for the Form->JSON input view + outputEditor: null, // read-only Ace editor for JSON/raw output +}; + +const dom = {}; +const DOM_IDS = [ + "health-badge", "version-info", "target-url", "connect-btn", "target-status", + "schema-link", "theme-toggle", "setup-panel", "setup-status", "setup-logs", + "schema-error", "form-container", "json-container", "json-editor", + "json-error", "json-format", "mode-form", "mode-json", "run-mode", + "run-mode-sync", "run-mode-stream", "run-mode-async", "prediction-id", + "webhook-options", "webhook-base-note", "stream-hint", "run-btn", "stop-btn", + "reset-btn", "result-status", "output-view", "output-view-text", + "output-view-raw", "metrics-container", "error-container", "output-container", +]; + +async function init() { + for (const id of DOM_IDS) dom[id] = document.getElementById(id); + + state.jsonEditor = createJSONEditor(dom["json-editor"], { autosize: false }); + + const params = new URLSearchParams(location.search); + dom["target-url"].value = + params.get("target") || localStorage.getItem(STORAGE_KEY) || DEFAULT_TARGET; + refreshThemeLabel(); + + const config = await api.getConfig(); + state.webhookBase = config.webhookBase || ""; + + dom["connect-btn"].addEventListener("click", connect); + dom["target-url"].addEventListener("keydown", (e) => { + if (e.key === "Enter") connect(); + }); + dom["theme-toggle"].addEventListener("click", () => { + toggleTheme(); + refreshThemeLabel(); + refreshEditorThemes(); + }); + dom["run-btn"].addEventListener("click", run); + dom["stop-btn"].addEventListener("click", stop); + dom["reset-btn"].addEventListener("click", reset); + dom["mode-form"].addEventListener("click", () => setMode("form")); + dom["mode-json"].addEventListener("click", () => setMode("json")); + dom["json-format"].addEventListener("click", formatJSON); + dom["run-mode-sync"].addEventListener("click", () => setRunMode("sync")); + dom["run-mode-stream"].addEventListener("click", () => setRunMode("stream")); + dom["run-mode-async"].addEventListener("click", () => setRunMode("async")); + dom["output-view-text"].addEventListener("click", () => setOutputView("text")); + dom["output-view-raw"].addEventListener("click", () => setOutputView("raw")); + + connect(); +} + +function refreshThemeLabel() { + dom["theme-toggle"].textContent = currentTheme() === "dark" ? "Light" : "Dark"; +} + +function connect() { + const url = dom["target-url"].value.trim(); + if (!url) return; + api.setTarget(url); + localStorage.setItem(STORAGE_KEY, url); + dom["schema-link"].href = + "/proxy/openapi.json?cog_target=" + encodeURIComponent(url); + history.replaceState(null, "", "?target=" + encodeURIComponent(url)); + + startHealthPolling(); + loadSchema(); +} + +function startHealthPolling() { + if (state.healthTimer) clearInterval(state.healthTimer); + pollHealth(); + state.healthTimer = setInterval(pollHealth, 5000); +} + +async function pollHealth() { + try { + const data = await api.health(); + setBadge(dom["health-badge"], data.status); + dom["target-status"].textContent = data.user_healthcheck_error || ""; + updateSetup(data.setup); + updateVersion(data.version); + } catch { + setBadge(dom["health-badge"], "unreachable"); + dom["target-status"].textContent = "target unreachable"; + } +} + +function updateSetup(setup) { + if (!setup) { + dom["setup-panel"].hidden = true; + return; + } + dom["setup-panel"].hidden = false; + setBadge(dom["setup-status"], setup.status); + dom["setup-logs"].textContent = setup.logs || ""; +} + +function updateVersion(version) { + if (!version) return; + const parts = []; + if (version.coglet) parts.push("coglet " + version.coglet); + if (version.cog) parts.push("cog " + version.cog); + if (version.python) parts.push("py " + version.python); + dom["version-info"].textContent = parts.join(" · "); +} + +async function loadSchema() { + const token = ++state.loadToken; + try { + const schema = await api.schema(); + if (token !== state.loadToken) return; // superseded by a newer connect + applySchema(schema); + dom["schema-error"].classList.remove("visible"); + } catch (err) { + if (token !== state.loadToken) return; + showError(dom["schema-error"], "Waiting for schema… (" + err.message + ")"); + setTimeout(() => { + if (token === state.loadToken) loadSchema(); + }, 3000); + } +} + +function applySchema(schema) { + state.schema = schema; + const schemas = (schema.components || {}).schemas || {}; + const paths = schema.paths || {}; + + state.inputSchema = resolveRef(schema, schemas.Input || {}); + state.outputSchema = resolveRef(schema, schemas.Output || {}); + state.supportsStreaming = + ((paths["/predictions"] || {}).post || {})["x-cog-streaming"] === true; + // Async predictions are observed via webhooks and cancelled via the cancel + // endpoint; treat the presence of that endpoint as the async-capable signal. + state.supportsAsync = !!paths["/predictions/{prediction_id}/cancel"]; + + state.runMode = state.supportsStreaming ? "stream" : "sync"; + configureRunModes(); + rebuildForm(defaultInput(schema, state.inputSchema)); +} + +function currentOutputSchema() { + return state.outputSchema || {}; +} + +// configureRunModes shows only the run modes the model advertises. +function configureRunModes() { + dom["run-mode-stream"].hidden = !state.supportsStreaming; + dom["run-mode-async"].hidden = !state.supportsAsync; + dom["run-mode"].hidden = !(state.supportsStreaming || state.supportsAsync); + + const available = { sync: true, stream: state.supportsStreaming, async: state.supportsAsync }; + if (!available[state.runMode]) state.runMode = "sync"; + + const out = state.outputSchema || {}; + const isIterator = + out["x-cog-array-type"] === "iterator" || out["x-cog-array-display"] === "concatenate"; + dom["stream-hint"].textContent = + !state.supportsStreaming && isIterator + ? "Add @cog.streaming to run() for real-time output" + : ""; + + updateRunModeButtons(); +} + +function updateRunModeButtons() { + for (const m of ["sync", "stream", "async"]) { + dom["run-mode-" + m].classList.toggle("active", state.runMode === m); + } + dom["webhook-options"].hidden = state.runMode !== "async"; + dom["webhook-base-note"].textContent = state.webhookBase + ? "Webhook: " + state.webhookBase + "/webhook/…" + : "No webhook host configured (set --webhook-host)."; +} + +function setRunMode(mode) { + if (dom["run-mode-" + mode].hidden) return; + state.runMode = mode; + updateRunModeButtons(); +} + +// --- input mode toggle (Form vs JSON) --- +function setMode(mode) { + if (mode === state.mode) return; + if (mode === "json") { + syncFormToJSON(); + } else { + const parsed = parseEditor(); + if (parsed === undefined) return; // invalid JSON: stay in JSON mode + rebuildForm(parsed); + } + state.mode = mode; + dom["mode-form"].classList.toggle("active", mode === "form"); + dom["mode-json"].classList.toggle("active", mode === "json"); + dom["form-container"].hidden = mode !== "form"; + dom["json-container"].hidden = mode !== "json"; + if (mode === "json") state.jsonEditor.resize(); +} + +function rebuildForm(value) { + state.form = buildForm(dom["form-container"], state.schema, state.inputSchema, value); + if (state.mode === "json") syncFormToJSON(); +} + +function syncFormToJSON() { + const value = state.form ? state.form.collect() : {}; + state.jsonEditor.setValue(JSON.stringify(value, null, 2), -1); + dom["json-error"].textContent = ""; +} + +function parseEditor() { + const raw = state.jsonEditor.getValue().trim(); + if (raw === "") { + dom["json-error"].textContent = ""; + return {}; + } + try { + const parsed = JSON.parse(raw); + dom["json-error"].textContent = ""; + return parsed; + } catch (err) { + dom["json-error"].textContent = "Invalid JSON: " + err.message; + return undefined; + } +} + +function formatJSON() { + if (!formatEditor(state.jsonEditor)) { + dom["json-error"].textContent = "Invalid JSON"; + } else { + dom["json-error"].textContent = ""; + } +} + +// --- output view toggle (Text vs Raw) --- +function setOutputView(view) { + state.outputView = view; + dom["output-view-text"].classList.toggle("active", view === "text"); + dom["output-view-raw"].classList.toggle("active", view === "raw"); + if (state.showLive) renderLive(); +} + +function showOutputView(visible) { + dom["output-view"].hidden = !visible; +} + +// renderLive renders the current output in the selected view. Raw shows the +// exact frames/payloads in a read-only code editor; Text concatenates +// plain-string output, renders media, or shows structured JSON in a read-only +// code editor (with folding) for objects/arrays. +function renderLive() { + // In Raw view the metrics are already part of the payload, so the separate + // metrics table is redundant. + dom["metrics-container"].hidden = state.outputView === "raw"; + // Streaming/async runs append output over time, so follow the tail; sync + // (one-shot) output keeps the top in view. + const follow = state.runMode !== "sync"; + if (state.outputView === "raw") { + renderCode(state.rawEvents.join("\n\n"), follow); + return; + } + const value = state.outputValue; + if (value == null) { + resetOutputArea(); + renderText(dom["output-container"], "", state.running); + } else if (isPlainText(value)) { + resetOutputArea(); + renderText(dom["output-container"], value, state.running); + } else if (Array.isArray(value) && value.length > 0 && value.every(isPlainText)) { + resetOutputArea(); + renderText(dom["output-container"], value.join(""), state.running); + } else if (hasMedia(value)) { + resetOutputArea(); + renderOutput(dom["output-container"], value, currentOutputSchema()); + } else { + renderCode(JSON.stringify(value, null, 2), follow); + } +} + +// renderCode shows text in the read-only output editor, reusing the instance +// across updates (e.g. streaming Raw) rather than recreating it. +function renderCode(text, follow = false) { + if (!state.outputEditor) { + clear(dom["output-container"]); + const host = el("div", { class: "ace-json ace-output" }); + dom["output-container"].append(host); + state.outputEditor = createJSONEditor(host, { readOnly: true, autosize: false }); + } + // cursorPos 1 sends the cursor (and viewport) to the end so the editor + // follows newly streamed content; -1 keeps the top in view for static output. + state.outputEditor.setValue(text, follow ? 1 : -1); + state.outputEditor.resize(); + if (follow) { + state.outputEditor.renderer.scrollToLine(state.outputEditor.session.getLength(), false, false); + } +} + +function resetOutputArea() { + if (state.outputEditor) { + destroyEditor(state.outputEditor); + state.outputEditor = null; + } + clear(dom["output-container"]); +} + +function isPlainText(x) { + return typeof x === "string" && !x.startsWith("data:") && !/^https?:\/\//i.test(x); +} + +function isMediaString(s) { + return typeof s === "string" && (s.startsWith("data:") || /^https?:\/\//i.test(s)); +} + +function hasMedia(value) { + return isMediaString(value) || (Array.isArray(value) && value.some(isMediaString)); +} + +// --- running --- +function activeInput() { + return state.mode === "json" ? parseEditor() : state.form.collect(); +} + +function currentId() { + const id = dom["prediction-id"].value.trim(); + return id || undefined; +} + +function run() { + if (state.running) return; + const input = activeInput(); + if (input === undefined) return; // invalid JSON + + clearError(dom["error-container"]); + resetOutputArea(); + renderMetrics(dom["metrics-container"], {}); + dom["result-status"].textContent = ""; + state.metrics = {}; + state.outputValue = null; + state.rawEvents = []; + state.lastId = currentId() || null; + + setRunning(true); + state.abort = new AbortController(); + + if (state.runMode === "async") runAsync(input); + else if (state.runMode === "stream") runStream(input); + else runSync(input); +} + +async function runSync(input) { + state.showLive = true; + showOutputView(true); + setBadge(dom["result-status"], "processing"); + try { + const response = await api.submit({ + endpoint: "/predictions", + id: currentId(), + input, + signal: state.abort.signal, + }); + state.lastId = response.id || state.lastId; + applyEnvelope(response); + state.outputValue = response.error ? null : response.output ?? null; + state.rawEvents = [JSON.stringify(response, null, 2)]; + } catch (err) { + reportRunError(err); + } finally { + setRunning(false); + renderLive(); + } +} + +async function runStream(input) { + state.showLive = true; + state.outputValue = []; + showOutputView(true); + setBadge(dom["result-status"], "processing"); + try { + for await (const event of api.stream({ + endpoint: "/predictions", + id: currentId(), + input, + signal: state.abort.signal, + })) { + if (event.raw != null) state.rawEvents.push(event.raw); + handleStreamEvent(event); + renderLive(); + } + } catch (err) { + reportRunError(err); + } finally { + setRunning(false); + renderLive(); // final render without the streaming cursor + } +} + +// runAsync submits with Prefer: respond-async and a webhook pointing at the +// playground server's sink, then observes delivered events over /events (SSE). +async function runAsync(input) { + state.showLive = true; + showOutputView(true); + setBadge(dom["result-status"], "starting"); + + const token = crypto.randomUUID(); + const webhook = state.webhookBase ? `${state.webhookBase}/webhook/${token}` : null; + if (webhook) { + const es = new EventSource("/events?token=" + token); + state.eventSource = es; + es.onmessage = (e) => { + state.rawEvents.push(e.data); + let data; + try { + data = JSON.parse(e.data); + } catch { + renderLive(); + return; + } + applyEnvelope(data); + if (!data.error && data.output != null) { + state.outputValue = data.output; + } + renderLive(); + if (TERMINAL.includes(data.status)) finishAsync(); + }; + } + + try { + const response = await api.submit({ + endpoint: "/predictions", + id: currentId(), + input, + asyncMode: true, + webhook, + webhookFilter: collectWebhookFilter(), + signal: state.abort.signal, + }); + state.lastId = response.id || state.lastId; + setBadge(dom["result-status"], response.status || "starting"); + if (!webhook) finishAsync(); // nothing to observe; stop the spinner + } catch (err) { + reportRunError(err); + finishAsync(); + } +} + +function finishAsync() { + if (state.eventSource) { + state.eventSource.close(); + state.eventSource = null; + } + setRunning(false); + renderLive(); +} + +function collectWebhookFilter() { + return Array.from(document.querySelectorAll(".wh-filter:checked")).map((c) => c.value); +} + +function handleStreamEvent(event) { + const data = event.data; + switch (event.type) { + case "start": + setBadge(dom["result-status"], data.status || "starting"); + break; + case "output": + if (!Array.isArray(state.outputValue)) state.outputValue = []; + state.outputValue.push(data.chunk); + break; + case "metric": + state.metrics[data.name] = data.value; + renderMetrics(dom["metrics-container"], state.metrics); + break; + case "error": + // Transport-level SSE error (e.g. replay truncated, broadcast lagged). + showError(dom["error-container"], data.error || "stream error"); + break; + case "completed": + applyEnvelope(data); + break; + } +} + +// applyEnvelope updates status/metrics/error from a prediction envelope (shared +// by sync responses, the streamed "completed" event, and webhooks). +function applyEnvelope(data) { + if (!data) return; + setBadge(dom["result-status"], data.status || "unknown"); + if (data.metrics) renderMetrics(dom["metrics-container"], data.metrics); + if (data.error) showError(dom["error-container"], data.error); +} + +function reportRunError(err) { + if (err.name === "AbortError") { + setBadge(dom["result-status"], "canceled"); + return; + } + // Surface the error in the Raw view too, not just the error banner. + state.rawEvents = [ + JSON.stringify(err.detail ? { detail: err.detail } : { error: err.message }, null, 2), + ]; + if (err.detail) { + renderValidationErrors(dom["error-container"], err.detail); + } else { + showError(dom["error-container"], err.message); + } + setBadge(dom["result-status"], "failed"); +} + +function setRunning(running) { + state.running = running; + dom["run-btn"].disabled = running; + dom["stop-btn"].disabled = !running; + dom["reset-btn"].disabled = running; +} + +// stop aborts the local request/stream and, if we know the prediction id, asks +// the model to cancel it (the only way to stop an async prediction). +function stop() { + if (state.abort) state.abort.abort(); + if (state.lastId) api.cancel("/predictions", state.lastId).catch(() => {}); + finishAsync(); +} + +function reset() { + if (state.running || !state.schema) return; + clearError(dom["error-container"]); + resetOutputArea(); + renderMetrics(dom["metrics-container"], {}); + dom["metrics-container"].hidden = false; + dom["result-status"].textContent = ""; + state.showLive = false; + state.outputValue = null; + state.rawEvents = []; + showOutputView(false); + rebuildForm(defaultInput(state.schema, state.inputSchema)); +} + +document.addEventListener("DOMContentLoaded", init); diff --git a/pkg/cli/playground/dom.js b/pkg/cli/playground/dom.js new file mode 100644 index 0000000000..ea75d81620 --- /dev/null +++ b/pkg/cli/playground/dom.js @@ -0,0 +1,56 @@ +// Minimal DOM helpers — a thin wrapper over document.createElement so building +// nodes reads top-to-bottom without a framework or a build step. +// +// el("div", { class: "field" }, el("label", { text: "name" }), input) +// +// Props: `class` -> className, `text` -> textContent, `html` is intentionally +// unsupported (we never inject untrusted HTML). `onclick`/`oninput`/... attach +// listeners. Boolean true sets a bare attribute; null/false/undefined skip it. +export function el(tag, props = {}, ...children) { + const node = document.createElement(tag); + for (const [key, value] of Object.entries(props)) { + setProp(node, key, value); + } + append(node, children); + return node; +} + +// setProp applies a single prop. Known DOM properties are set directly; an +// `on*` function becomes an event listener; everything else is an attribute +// (a `true` value renders as a bare boolean attribute). +function setProp(node, key, value) { + if (value == null || value === false) return; + switch (key) { + case "class": + node.className = value; + break; + case "text": + node.textContent = value; + break; + case "value": + node.value = value; + break; + case "checked": + node.checked = Boolean(value); + break; + default: + if (key.startsWith("on") && typeof value === "function") { + node.addEventListener(key.slice(2).toLowerCase(), value); + } else { + node.setAttribute(key, value === true ? "" : value); + } + } +} + +// append flattens arrays and turns primitives into text nodes. +export function append(node, children) { + for (const child of children.flat()) { + if (child == null || child === false) continue; + node.append(child.nodeType ? child : document.createTextNode(String(child))); + } +} + +// clear removes all children of a node. +export function clear(node) { + while (node.firstChild) node.removeChild(node.firstChild); +} diff --git a/pkg/cli/playground/editor.js b/pkg/cli/playground/editor.js new file mode 100644 index 0000000000..ad6f2610c4 --- /dev/null +++ b/pkg/cli/playground/editor.js @@ -0,0 +1,140 @@ +// Thin wrapper over the vendored Ace global (window.ace) for JSON editing. +// Editors are tracked so their theme can follow the light/dark toggle. + +const EDITORS = new Set(); + +// Ace theme colors are sourced from the Kumo design tokens (vendor/kumo.css) so +// the editor matches the rest of the UI and recolors with the data-mode toggle. +// We register two themes (light/dark) that share the same token-driven CSS; they +// differ only in `isDark`, which Ace uses for cursor/selection contrast. JSON +// token classes come from vendor/ace/mode-json.js: object keys -> `variable`, +// string values -> `string`, numbers -> `constant.numeric`, booleans -> +// `constant.language`, brackets -> `paren`, commas -> `punctuation.operator`. +function kumoThemeCss(cssClass) { + const s = `.${cssClass}`; + return ` +${s} { background-color: var(--color-kumo-base); color: var(--text-color-kumo-default); } +${s} .ace_gutter { background: var(--color-kumo-elevated); color: var(--text-color-kumo-subtle); } +${s} .ace_print-margin { width: 1px; background: var(--color-kumo-hairline); } +${s} .ace_cursor { color: var(--text-color-kumo-default); } +${s} .ace_marker-layer .ace_selection { background: var(--color-kumo-info-tint); } +${s} .ace_marker-layer .ace_active-line { background: color-mix(in srgb, var(--color-kumo-fill) 45%, transparent); } +${s} .ace_gutter-active-line { background-color: color-mix(in srgb, var(--color-kumo-fill) 45%, transparent); } +${s} .ace_marker-layer .ace_selected-word { border: 1px solid var(--color-kumo-line); } +${s} .ace_fold { background-color: var(--color-kumo-brand); border-color: var(--text-color-kumo-default); } +${s} .ace_variable { color: var(--text-color-kumo-link); } +${s} .ace_string { color: var(--text-color-kumo-success); } +${s} .ace_constant.ace_numeric { color: var(--text-color-kumo-warning); } +${s} .ace_constant.ace_language { color: var(--text-color-kumo-brand); } +${s} .ace_constant.ace_language.ace_escape { color: var(--text-color-kumo-info); } +${s} .ace_paren, ${s} .ace_punctuation { color: var(--text-color-kumo-subtle); } +`; +} + +function defineKumoTheme(id, cssClass, isDark) { + const cssText = kumoThemeCss(cssClass); + ace.define("ace/theme/" + id, ["require", "exports", "module", "ace/lib/dom"], function (require, exports) { + exports.isDark = isDark; + exports.cssClass = cssClass; + exports.cssText = cssText; + require("ace/lib/dom").importCssString(cssText, cssClass, false); + }); +} + +defineKumoTheme("kumo-light", "ace-kumo-light", false); +defineKumoTheme("kumo-dark", "ace-kumo-dark", true); + +function aceTheme() { + return document.documentElement.dataset.mode === "light" + ? "ace/theme/kumo-light" + : "ace/theme/kumo-dark"; +} + +// createJSONEditor turns a host element into an Ace JSON editor. Read-only mode +// is used for outputs (no cursor, not editable) but still allows code folding. +export function createJSONEditor(host, opts = {}) { + // autosize editors grow with content (good for small inline fields). + // Fixed-height editors take their height from CSS and scroll internally, + // which is required for drag-selection autoscroll to work on large content. + const { value = "", readOnly = false, autosize = true, minLines = 4, maxLines = 30 } = opts; + const editor = ace.edit(host); + editor.session.setMode("ace/mode/json"); + editor.setTheme(aceTheme()); + const options = { + readOnly, + fontSize: "12px", + showPrintMargin: false, + useWorker: false, // we validate JSON ourselves; avoids loading a worker file + highlightActiveLine: !readOnly, + highlightGutterLine: !readOnly, + showFoldWidgets: true, + fadeFoldWidgets: false, + tabSize: 2, + useSoftTabs: true, + wrap: true, + }; + if (autosize) { + options.minLines = minLines; + options.maxLines = maxLines; + } + editor.setOptions(options); + editor.setValue(value, -1); + if (readOnly) { + editor.setHighlightActiveLine(false); + try { + editor.renderer.$cursorLayer.element.style.display = "none"; + } catch { + /* cursor layer not ready; ignore */ + } + } + addCopyButton(host, editor); + EDITORS.add(editor); + return editor; +} + +// addCopyButton overlays a Copy button that copies the whole document. This is +// far easier than drag-selecting large content, which scrolls awkwardly. +function addCopyButton(host, editor) { + const button = document.createElement("button"); + button.type = "button"; + button.className = "editor-copy"; + button.textContent = "Copy"; + button.addEventListener("click", async (event) => { + event.preventDefault(); + event.stopPropagation(); + try { + await navigator.clipboard.writeText(editor.getValue()); + button.textContent = "Copied"; + setTimeout(() => { + button.textContent = "Copy"; + }, 1200); + } catch { + // Clipboard API unavailable: fall back to selecting all so Cmd/Ctrl+C works. + editor.selectAll(); + editor.focus(); + } + }); + host.appendChild(button); +} + +export function destroyEditor(editor) { + if (!editor) return; + EDITORS.delete(editor); + editor.destroy(); +} + +export function refreshEditorThemes() { + const theme = aceTheme(); + for (const editor of EDITORS) editor.setTheme(theme); +} + +// formatEditor pretty-prints the editor's JSON in place; returns false if the +// content isn't valid JSON. +export function formatEditor(editor) { + try { + editor.setValue(JSON.stringify(JSON.parse(editor.getValue()), null, 2), -1); + return true; + } catch { + return false; + } +} diff --git a/pkg/cli/playground/form.js b/pkg/cli/playground/form.js new file mode 100644 index 0000000000..a765d7c39e --- /dev/null +++ b/pkg/cli/playground/form.js @@ -0,0 +1,327 @@ +import { el, clear } from "./dom.js"; +import { fieldKind, orderedInputs, coerceEnum } from "./schema.js"; +import { fileToDataURI, formatBytes } from "./api.js"; +import { mediaNode } from "./media.js"; +import { createJSONEditor, destroyEditor } from "./editor.js"; + +// Ace editors created for object/dict fields in the current form, destroyed and +// rebuilt whenever the form is rebuilt. +let activeEditors = []; + +// buildForm renders the Input fields into `container` and returns a handle with +// collect(), which reads the current values on demand. There is no reactive +// state: inputs are built once and queried when the user runs a prediction. +export function buildForm(container, root, inputSchema, value = {}) { + for (const editor of activeEditors) destroyEditor(editor); + activeEditors = []; + clear(container); + const inputs = orderedInputs(inputSchema); + if (inputs.length === 0) { + container.append(el("p", { class: "muted", text: "This model takes no inputs." })); + return { collect: () => ({}) }; + } + + const fields = []; + for (const { name, prop, required } of inputs) { + const field = buildField(root, name, prop, required, value[name]); + container.append(field.element); + fields.push({ name, read: field.read, included: field.included }); + } + // Editors must be resized once their hosts are attached to the document. + for (const editor of activeEditors) editor.resize(); + + return { + // collect includes required fields always and optional fields only when + // their include checkbox is ticked (ticking happens automatically when the + // field is edited). + collect() { + const out = {}; + for (const { name, included, read } of fields) { + if (included()) out[name] = read(); + } + return out; + }, + }; +} + +// buildField renders one labelled field and returns its value reader plus an +// `included` predicate. Optional fields get an include checkbox so they can be +// omitted from the request; it auto-ticks when the field is edited. +function buildField(root, name, prop, required, initial) { + const kind = fieldKind(root, prop); + const widget = buildWidget(root, kind, initial); + + const label = el("label"); + let includeBox = null; + if (!required) { + includeBox = el("input", { + type: "checkbox", + class: "include-box", + checked: initial !== undefined, + title: "Include this optional field in the request", + }); + label.append(includeBox); + const touch = () => { + includeBox.checked = true; + }; + widget.element.addEventListener("input", touch); + widget.element.addEventListener("change", touch); + if (widget.onChange) widget.onChange(touch); + } + label.append(name); + if (required) label.append(el("span", { class: "req", text: " *" })); + if (kind.prop.deprecated) { + label.append(el("span", { class: "deprecated-tag", text: " (deprecated)" })); + } + + const field = el("div", { class: "field" }, label); + if (kind.prop.description) { + field.append(el("small", { class: "desc", text: kind.prop.description })); + } + const hint = constraintText(kind.prop); + if (hint) field.append(el("small", { class: "constraint", text: hint })); + field.append(widget.element); + + return { + element: field, + read: widget.read, + included: () => required || includeBox.checked, + }; +} + +// constraintText summarizes the numeric/string constraints emitted in the +// schema (minimum/maximum, minLength/maxLength, pattern) for display. +function constraintText(prop) { + const parts = []; + if (prop.minimum !== undefined && prop.maximum !== undefined) { + parts.push(`${prop.minimum}–${prop.maximum}`); + } else if (prop.minimum !== undefined) { + parts.push(`min ${prop.minimum}`); + } else if (prop.maximum !== undefined) { + parts.push(`max ${prop.maximum}`); + } + if (prop.minLength !== undefined && prop.maxLength !== undefined) { + parts.push(`${prop.minLength}–${prop.maxLength} chars`); + } else if (prop.minLength !== undefined) { + parts.push(`min ${prop.minLength} chars`); + } else if (prop.maxLength !== undefined) { + parts.push(`max ${prop.maxLength} chars`); + } + if (prop.pattern) parts.push(`pattern: ${prop.pattern}`); + return parts.join(" · "); +} + +// buildWidget maps a field kind to a DOM widget + value reader. Reused for both +// top-level fields and array items. +function buildWidget(root, kind, initial) { + switch (kind.kind) { + case "enum": + return enumWidget(kind.choices, kind.prop, initial); + case "file": + return fileWidget(initial); + case "secret": + return textWidget("password", initial ?? kind.prop.default); + case "string": + return textareaWidget(initial ?? kind.prop.default); + case "integer": + return numberWidget(kind.prop, true, initial); + case "number": + return numberWidget(kind.prop, false, initial); + case "boolean": + return booleanWidget(initial ?? kind.prop.default); + case "array": + return arrayWidget(root, kind.items, initial); + default: + return objectWidget(kind.prop, initial); + } +} + +function textWidget(type, initial) { + const input = el("input", { type, value: initial ?? "" }); + return { element: input, read: () => input.value }; +} + +function textareaWidget(initial) { + const input = el("textarea", { rows: "2", value: initial ?? "" }); + return { element: input, read: () => input.value }; +} + +function numberWidget(prop, isInt, initial) { + const input = el("input", { + type: "number", + value: initial ?? prop.default ?? "", + min: prop.minimum, + max: prop.maximum, + step: isInt ? "1" : "any", + }); + return { + element: input, + read: () => { + if (input.value === "") return ""; + return isInt ? parseInt(input.value, 10) : parseFloat(input.value); + }, + }; +} + +// Booleans render as a true/false select rather than a checkbox so they don't +// collide visually with the optional-field include checkbox. +function booleanWidget(initial) { + const select = el("select"); + const current = initial === true; + for (const option of [true, false]) { + const opt = el("option", { value: String(option), text: String(option) }); + if (option === current) opt.selected = true; + select.append(opt); + } + return { element: select, read: () => select.value === "true" }; +} + +function enumWidget(choices, prop, initial) { + const current = initial ?? prop.default; + const select = el("select"); + if (current === undefined || current === null) { + select.append(el("option", { value: "", text: "— select —" })); + } + for (const choice of choices) { + const option = el("option", { value: String(choice), text: String(choice) }); + if (choice === current) option.selected = true; + select.append(option); + } + return { element: select, read: () => coerceEnum(choices, select.value) }; +} + +// fileWidget: upload a file (-> data: URI) OR paste a URL. Mutually exclusive; +// reads as a single string value that round-trips into the JSON editor. Shows +// an inline preview for image/audio/video so you can confirm the input. +function fileWidget(initial) { + let currentValue = typeof initial === "string" ? initial : ""; + + const fileInput = el("input", { type: "file" }); + const fileName = el("span", { class: "file-name" }); + const urlInput = el("input", { + type: "text", + class: "url-input", + placeholder: "https://...", + value: currentValue, + }); + const preview = el("div", { class: "input-preview" }); + + function updatePreview() { + clear(preview); + const node = mediaNode(currentValue); + if (node) preview.append(node); + } + + fileInput.addEventListener("change", async () => { + const file = fileInput.files[0]; + if (!file) return; + currentValue = await fileToDataURI(file); + urlInput.value = ""; + fileName.textContent = `${file.name} (${formatBytes(file.size)})`; + updatePreview(); + }); + + urlInput.addEventListener("input", () => { + currentValue = urlInput.value; + fileInput.value = ""; + fileName.textContent = ""; + updatePreview(); + }); + + const controls = el( + "div", + { class: "file-widget" }, + fileInput, + fileName, + el("span", { class: "muted", text: "or URL" }), + urlInput, + ); + const element = el("div", {}, controls, preview); + updatePreview(); + return { element, read: () => currentValue }; +} + +// Object/dict/Any inputs are edited as JSON in a code editor (folding + syntax +// highlighting) instead of a bare textarea. +function objectWidget(prop, initial) { + const text = + initial !== undefined + ? typeof initial === "string" + ? initial + : JSON.stringify(initial, null, 2) + : prop.default !== undefined && prop.default !== null + ? JSON.stringify(prop.default, null, 2) + : ""; + + const host = el("div", { class: "ace-json ace-field" }); + const error = el("small", { class: "field-error" }); + const element = el("div", {}, host, error); + const editor = createJSONEditor(host, { value: text, autosize: false }); + activeEditors.push(editor); + + return { + element, + onChange: (cb) => editor.on("change", cb), + read: () => { + const raw = editor.getValue().trim(); + if (raw === "") { + error.textContent = ""; + return ""; + } + try { + const parsed = JSON.parse(raw); + error.textContent = ""; + return parsed; + } catch (err) { + error.textContent = "Invalid JSON: " + err.message; + return ""; + } + }, + }; +} + +// arrayWidget renders a growable list of item widgets. +function arrayWidget(root, items, initial) { + const rows = el("div"); + const itemKind = fieldKind(root, items); + const readers = []; + + function addRow(value) { + const widget = buildWidget(root, itemKind, value); + const reader = widget.read; + readers.push(reader); + + const remove = el("button", { + type: "button", + class: "ghost-btn danger", + text: "−", + onclick: () => { + row.remove(); + const idx = readers.indexOf(reader); + if (idx >= 0) readers.splice(idx, 1); + }, + }); + const row = el("div", { class: "array-row" }, widget.element, remove); + rows.append(row); + } + + const addBtn = el("button", { + type: "button", + class: "ghost-btn", + text: "+ Add", + onclick: () => addRow(undefined), + }); + + const initialItems = Array.isArray(initial) ? initial : []; + for (const v of initialItems) addRow(v); + if (readers.length === 0) addRow(undefined); + + const element = el("div", { class: "array-input" }, rows, addBtn); + return { + element, + read: () => + readers + .map((r) => r()) + .filter((v) => v !== "" && v !== null && v !== undefined), + }; +} diff --git a/pkg/cli/playground/index.html b/pkg/cli/playground/index.html new file mode 100644 index 0000000000..3c25d23158 --- /dev/null +++ b/pkg/cli/playground/index.html @@ -0,0 +1,105 @@ + + + + + + Cog Playground + + + + + +
+

Cog Playground

+ unknown + + + + Schema +
+ +
+ + + + +
+ +
+
+ + + +
+ + + + + +
+ + + + + +
+
+
+

Input

+
+ + +
+
+ +
+ +
+ + +
+ +
+
+

Output

+ +
+ +
+
+
+
+
+
+ + + + + + diff --git a/pkg/cli/playground/media.js b/pkg/cli/playground/media.js new file mode 100644 index 0000000000..5ecba5ec03 --- /dev/null +++ b/pkg/cli/playground/media.js @@ -0,0 +1,24 @@ +import { el } from "./dom.js"; + +// Recognized media extensions for plain URLs (data: URIs carry their own MIME). +const IMAGE_EXT = /\.(?:png|jpe?g|gif|webp|avif|bmp|svg)(?:[?#]|$)/i; +const AUDIO_EXT = /\.(?:mp3|wav|ogg|oga|flac|m4a|aac|opus)(?:[?#]|$)/i; +const VIDEO_EXT = /\.(?:mp4|webm|ogv|mov|m4v)(?:[?#]|$)/i; + +// mediaNode returns an /