diff --git a/README.md b/README.md index 9e753f46d2..21fc5d87a3 100644 --- a/README.md +++ b/README.md @@ -127,83 +127,83 @@ Choose your platform for installation instructions.
macOS - The easiest way to install Cog on macOS is with Homebrew: +The easiest way to install Cog on macOS is with Homebrew: - ```console - brew install replicate/tap/cog - ``` +```console +brew install replicate/tap/cog +``` - You can also use the install script: +You can also use the install script: - ```sh - # bash, zsh, and other shells - sh <(curl -fsSL https://cog.run/install.sh) +```sh +# bash, zsh, and other shells +sh <(curl -fsSL https://cog.run/install.sh) - # fish shell - sh (curl -fsSL https://cog.run/install.sh | psub) +# fish shell +sh (curl -fsSL https://cog.run/install.sh | psub) - # download with wget and run in a separate command - wget -qO- https://cog.run/install.sh - sh ./install.sh - ``` +# download with wget and run in a separate command +wget -qO- https://cog.run/install.sh +sh ./install.sh +``` - Or install manually: +Or install manually: - ```console - sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" - sudo chmod +x /usr/local/bin/cog - sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true - ``` +```console +sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" +sudo chmod +x /usr/local/bin/cog +sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true +``` - If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run: +If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run: - ```console - sudo xattr -d com.apple.quarantine /usr/local/bin/cog - ``` +```console +sudo xattr -d com.apple.quarantine /usr/local/bin/cog +```
Linux - You can install Cog using the install script: +You can install Cog using the install script: - ```sh - # bash, zsh, and other shells - sh <(curl -fsSL https://cog.run/install.sh) +```sh +# bash, zsh, and other shells +sh <(curl -fsSL https://cog.run/install.sh) - # fish shell - sh (curl -fsSL https://cog.run/install.sh | psub) +# fish shell +sh (curl -fsSL https://cog.run/install.sh | psub) - # download with wget and run in a separate command - wget -qO- https://cog.run/install.sh - sh ./install.sh - ``` +# download with wget and run in a separate command +wget -qO- https://cog.run/install.sh +sh ./install.sh +``` - Or install manually: +Or install manually: - ```console - sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" - sudo chmod +x /usr/local/bin/cog - ``` +```console +sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" +sudo chmod +x /usr/local/bin/cog +```
Windows - Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above. +Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.
Docker - To install Cog inside a Docker image: +To install Cog inside a Docker image: - ```dockerfile - RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)" - ``` +```dockerfile +RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)" +```
diff --git a/docs/base-images.md b/docs/base-images.md new file mode 100644 index 0000000000..843ea44fa7 --- /dev/null +++ b/docs/base-images.md @@ -0,0 +1,72 @@ +# Base images + +Cog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push. + +## What is a Cog base image? + +A Cog base image is a Docker image maintained by Replicate that includes: + +- **Python runtime** — the Python version specified in your `cog.yaml`. +- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`). +- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration. +- **Cog runtime** — the Cog SDK and coglet prediction server. + +Base images are tagged by their configuration, for example: + +``` +r8.im/cog-base:cuda12-python3.13-torch2.6 +r8.im/cog-base:python3.13 +``` + +When you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster. + +The entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list). + +## Using the Cog base image + +The `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands: + +- [`cog build`](cli.md#cog-build) +- [`cog push`](cli.md#cog-push) +- [`cog run`](cli.md#cog-run) +- [`cog serve`](cli.md#cog-serve) +- [`cog exec`](cli.md#cog-exec) + +Since it's on by default, you don't need to pass any flags: + +```bash +cog push r8.im/your-username/my-model +``` + +This builds and pushes your model using a prebuilt Cog base image for faster cold boots. + +## Disabling the Cog base image + +If you encounter build issues or need a custom base layer, you can disable the Cog base image: + +```bash +cog build --use-cog-base-image=false +``` + +When disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag. + +## Relationship to `--use-cuda-base-image` + +The `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack. + +When you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically: + +- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image. +- **CPU models**: uses a slim Python base image. + +These flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically. + +## Troubleshooting + +If `cog build` or `cog push` fails with the Cog base image enabled, try disabling it: + +```bash +cog push --use-cog-base-image=false r8.im/your-username/my-model +``` + +This falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration. diff --git a/docs/deploy.md b/docs/deploy.md index 63a5fa83c7..f6cfcf685d 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -76,11 +76,11 @@ curl http://localhost:5001/predictions -X POST \ ```json { - "status": "succeeded", - "output": "data:image/png;base64,...", - "metrics": { - "predict_time": 4.52 - } + "status": "succeeded", + "output": "data:image/png;base64,...", + "metrics": { + "predict_time": 4.52 + } } ``` @@ -144,8 +144,8 @@ the response contains a base64-encoded data URL by default: ```json { - "status": "succeeded", - "output": "data:image/png;base64,iVBORw0KGgo..." + "status": "succeeded", + "output": "data:image/png;base64,iVBORw0KGgo..." } ``` @@ -172,8 +172,8 @@ contains the uploaded URL instead of a data URL: ```json { - "status": "succeeded", - "output": "https://example.com/upload/image.png" + "status": "succeeded", + "output": "https://example.com/upload/image.png" } ``` diff --git a/docs/getting-started-own-model.md b/docs/getting-started-own-model.md index 4bc858b58c..20a830295f 100644 --- a/docs/getting-started-own-model.md +++ b/docs/getting-started-own-model.md @@ -58,6 +58,8 @@ Type "help", "copyright", "credits" or "license" for more information. This is handy for ensuring a consistent environment for development or training. +By default, Cog builds on top of a [prebuilt base image](base-images.md) that includes Python, common system libraries, and the Cog runtime. This speeds up builds and reduces cold boot times when you deploy with `cog push`. + With `cog.yaml`, you can also install system packages and other things. [Take a look at the full reference to see what else you can do.](yaml.md) ## Define how to run your model diff --git a/docs/getting-started.md b/docs/getting-started.md index e11587fd14..ae32b53cc2 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -151,6 +151,8 @@ Note: The first time you run `cog run`, the build process will be triggered to g We can bake your model's code, the trained weights, and the Docker environment into a Docker image. This image serves an HTTP server, and can be deployed to anywhere that Docker runs to serve real-time inference. +By default, Cog builds on top of a [prebuilt base image](base-images.md) that includes Python, common system libraries, and the Cog runtime. This significantly reduces cold boot times when deploying your model. + ```bash cog build -t resnet # Building Docker image... @@ -198,6 +200,9 @@ cog push The Docker image is now accessible to anyone or any system that has access to this Docker registry. +> [!TIP] +> `cog push` uses a [prebuilt Cog base image](base-images.md) by default for faster cold boots. If you run into build issues, try disabling it with `--use-cog-base-image=false`. + ## Next steps Those are the basics! Next, you might want to take a look at: diff --git a/docs/llms.txt b/docs/llms.txt index 11cb30b033..e3d4ed462a 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -127,83 +127,83 @@ Choose your platform for installation instructions.
macOS - The easiest way to install Cog on macOS is with Homebrew: +The easiest way to install Cog on macOS is with Homebrew: - ```console - brew install replicate/tap/cog - ``` +```console +brew install replicate/tap/cog +``` - You can also use the install script: +You can also use the install script: - ```sh - # bash, zsh, and other shells - sh <(curl -fsSL https://cog.run/install.sh) +```sh +# bash, zsh, and other shells +sh <(curl -fsSL https://cog.run/install.sh) - # fish shell - sh (curl -fsSL https://cog.run/install.sh | psub) +# fish shell +sh (curl -fsSL https://cog.run/install.sh | psub) - # download with wget and run in a separate command - wget -qO- https://cog.run/install.sh - sh ./install.sh - ``` +# download with wget and run in a separate command +wget -qO- https://cog.run/install.sh +sh ./install.sh +``` - Or install manually: +Or install manually: - ```console - sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" - sudo chmod +x /usr/local/bin/cog - sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true - ``` +```console +sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" +sudo chmod +x /usr/local/bin/cog +sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true +``` - If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run: +If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run: - ```console - sudo xattr -d com.apple.quarantine /usr/local/bin/cog - ``` +```console +sudo xattr -d com.apple.quarantine /usr/local/bin/cog +```
Linux - You can install Cog using the install script: +You can install Cog using the install script: - ```sh - # bash, zsh, and other shells - sh <(curl -fsSL https://cog.run/install.sh) +```sh +# bash, zsh, and other shells +sh <(curl -fsSL https://cog.run/install.sh) - # fish shell - sh (curl -fsSL https://cog.run/install.sh | psub) +# fish shell +sh (curl -fsSL https://cog.run/install.sh | psub) - # download with wget and run in a separate command - wget -qO- https://cog.run/install.sh - sh ./install.sh - ``` +# download with wget and run in a separate command +wget -qO- https://cog.run/install.sh +sh ./install.sh +``` - Or install manually: +Or install manually: - ```console - sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" - sudo chmod +x /usr/local/bin/cog - ``` +```console +sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')" +sudo chmod +x /usr/local/bin/cog +```
Windows - Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above. +Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.
Docker - To install Cog inside a Docker image: +To install Cog inside a Docker image: - ```dockerfile - RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)" - ``` +```dockerfile +RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)" +```
@@ -242,6 +242,82 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environme +--- + +# Base images + +Cog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push. + +## What is a Cog base image? + +A Cog base image is a Docker image maintained by Replicate that includes: + +- **Python runtime** — the Python version specified in your `cog.yaml`. +- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`). +- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration. +- **Cog runtime** — the Cog SDK and coglet prediction server. + +Base images are tagged by their configuration, for example: + +``` +r8.im/cog-base:cuda12-python3.13-torch2.6 +r8.im/cog-base:python3.13 +``` + +When you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster. + +The entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list). + +## Using the Cog base image + +The `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands: + +- [`cog build`](cli.md#cog-build) +- [`cog push`](cli.md#cog-push) +- [`cog run`](cli.md#cog-run) +- [`cog serve`](cli.md#cog-serve) +- [`cog exec`](cli.md#cog-exec) + +Since it's on by default, you don't need to pass any flags: + +```bash +cog push r8.im/your-username/my-model +``` + +This builds and pushes your model using a prebuilt Cog base image for faster cold boots. + +## Disabling the Cog base image + +If you encounter build issues or need a custom base layer, you can disable the Cog base image: + +```bash +cog build --use-cog-base-image=false +``` + +When disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag. + +## Relationship to `--use-cuda-base-image` + +The `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack. + +When you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically: + +- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image. +- **CPU models**: uses a slim Python base image. + +These flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically. + +## Troubleshooting + +If `cog build` or `cog push` fails with the Cog base image enabled, try disabling it: + +```bash +cog push --use-cog-base-image=false r8.im/your-username/my-model +``` + +This falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration. + + --- # CLI reference @@ -633,11 +709,11 @@ curl http://localhost:5001/predictions -X POST \ ```json { - "status": "succeeded", - "output": "data:image/png;base64,...", - "metrics": { - "predict_time": 4.52 - } + "status": "succeeded", + "output": "data:image/png;base64,...", + "metrics": { + "predict_time": 4.52 + } } ``` @@ -701,8 +777,8 @@ the response contains a base64-encoded data URL by default: ```json { - "status": "succeeded", - "output": "data:image/png;base64,iVBORw0KGgo..." + "status": "succeeded", + "output": "data:image/png;base64,iVBORw0KGgo..." } ``` @@ -729,8 +805,8 @@ contains the uploaded URL instead of a data URL: ```json { - "status": "succeeded", - "output": "https://example.com/upload/image.png" + "status": "succeeded", + "output": "https://example.com/upload/image.png" } ``` @@ -1187,6 +1263,8 @@ Type "help", "copyright", "credits" or "license" for more information. This is handy for ensuring a consistent environment for development or training. +By default, Cog builds on top of a [prebuilt base image](base-images.md) that includes Python, common system libraries, and the Cog runtime. This speeds up builds and reduces cold boot times when you deploy with `cog push`. + With `cog.yaml`, you can also install system packages and other things. [Take a look at the full reference to see what else you can do.](yaml.md) ## Define how to run your model @@ -1443,6 +1521,8 @@ Note: The first time you run `cog run`, the build process will be triggered to g We can bake your model's code, the trained weights, and the Docker environment into a Docker image. This image serves an HTTP server, and can be deployed to anywhere that Docker runs to serve real-time inference. +By default, Cog builds on top of a [prebuilt base image](base-images.md) that includes Python, common system libraries, and the Cog runtime. This significantly reduces cold boot times when deploying your model. + ```bash cog build -t resnet # Building Docker image... @@ -1490,6 +1570,9 @@ cog push The Docker image is now accessible to anyone or any system that has access to this Docker registry. +> [!TIP] +> `cog push` uses a [prebuilt Cog base image](base-images.md) by default for faster cold boots. If you run into build issues, try disabling it with `--use-cog-base-image=false`. + ## Next steps Those are the basics! Next, you might want to take a look at: diff --git a/mkdocs.yml b/mkdocs.yml index 6c394d3c31..dfc0390ad5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -12,6 +12,7 @@ nav: - Training API: training.md - HTTP API: http.md - CLI: cli.md + - Base images: base-images.md - Environment variables: environment.md - Private registry: private-package-registry.md - Notebooks: notebooks.md