From d945e75fc695ef74d98408258442873ffa559166 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Tue, 7 Jul 2026 08:37:10 -0400 Subject: [PATCH 1/3] Create modelrepotest.mdx --- serverless/modelrepotest.mdx | 121 +++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 serverless/modelrepotest.mdx diff --git a/serverless/modelrepotest.mdx b/serverless/modelrepotest.mdx new file mode 100644 index 00000000..99777a29 --- /dev/null +++ b/serverless/modelrepotest.mdx @@ -0,0 +1,121 @@ +--- +title: "Model Repo testing" +sidebarTitle: "Model Repo testing" +description: "Upload a model to Model Repo and deploy it to a Serverless endpoint." +--- + +## Scripted testing + +A shell script is available that runs all of the steps below automatically using mock model files. Download `model_repo_testing.sh` from the internal Notion page and run it to validate end-to-end without manually executing each step. + +## Manual testing + +### Prerequisites + +- Your email is [feature-flagged](https://us.posthog.com/project/105711/feature_flags/299352) for Model Repo access. +- The following environment variables are exported: + +```bash +export RUNPOD_API_URL="https://rest.runpod.io/v1" +export RUNPOD_GRAPHQL_URL="https://api.runpod.io/graphql" +export RUNPOD_API_KEY="your-api-key" + +export MODEL_NAME="$(whoami)-test-$(date +%s)" # must be unique per test run +export MODEL_PATH="/path/to/model" +``` + +- `MODEL_NAME` should be unique for each test run. Reusing the same name uploads a new version of the existing model rather than creating a new one. +- `jq` is installed for parsing JSON output. + +### Step 1: Build runpodctl + +```bash +git clone git@github.com:runpod/runpodctl.git +cd runpodctl +make +``` + +### Step 2: Upload the model + +```bash +./bin/runpodctl model add \ + --name "$MODEL_NAME" \ + --model-path "$MODEL_PATH" \ + --create-upload +``` + +This outputs a JSON string listing all uploaded files. + +### Step 3: Wait for the model to be hashed + +After upload, the model must be hashed by an asynchronous background process. This typically completes in a few minutes but can take up to 10–15 minutes. + +Poll until the hash field is non-null: + +```bash +./bin/runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash' +``` + +While hashing is in progress, the command returns `null`: + +```bash +% ./bin/runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash' +null +``` + +Once hashing is complete, it returns the hash value: + +```bash +% ./bin/runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash' +71a311bdf0ca44119ed74dbef8cf573bc89b58cbc48a10fe508f756ebb1922dc +``` + +### Step 4: Build the model URL + +```bash +export USER_ID="$(./bin/runpodctl user | jq -r '.id')" +export MODEL_HASH="$(./bin/runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash')" +export MODEL_URL="https://local/${USER_ID}/${MODEL_NAME}:${MODEL_HASH}" +``` + +### Step 5: Deploy a Serverless endpoint with the model attached + +```bash +./bin/runpodctl serverless create \ + --name "$(whoami)_ctl_test" \ + --template-id "mockworker" \ + --gpu-id "AMPERE_24" \ + --workers-max 3 \ + --workers-min 1 \ + --model-reference "$MODEL_URL" +``` + +Notes: +- `--model-reference` is only supported with `--template-id` and GPU endpoints. +- `--gpu-id` accepts a single GPU ID — do not pass a comma-separated list. +- `--model-reference` is repeatable if multiple models need to be attached. + +### Step 6: Verify the model is attached to the worker + +1. Go to **Serverless** in the left navigation bar under **Resources**. +2. Select the endpoint you created (`ctl_test` if you used the commands above). +3. Click the **Workers** tab. +4. Select a worker showing a **Running** status. +5. Click **Connect**, then use the `ssh` command or the **Web Terminal**. +6. Run the following to confirm your model files are present: + +```bash +find /runpod-volume/huggingface-cache/hub/models--$(echo $MODEL_NAME | sed 's@/@--@g')/snapshots/${MODEL_REVISION} -type f +``` + + +There is currently no way to retrieve SSH connection details for a running Serverless worker via `runpodctl`. Use the web UI to connect. + + +### Step 7: Clean up + +Delete the endpoint after testing to stop spend. Use the web UI or: + +```bash +./bin/runpodctl serverless delete +``` From f0f0a00f04c0799c95855085d84b2a834a3f04cf Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Tue, 7 Jul 2026 09:10:47 -0400 Subject: [PATCH 2/3] Update docs.json --- docs.json | 1 + 1 file changed, 1 insertion(+) diff --git a/docs.json b/docs.json index 474ced9a..d5c205a5 100644 --- a/docs.json +++ b/docs.json @@ -120,6 +120,7 @@ "serverless/development/dual-mode-worker" ] }, + "serverless/modelrepotest", { "group": "Manage endpoints", "pages": [ From a3d92158c1c6d92235d5dff482a0ec0a69eb0c47 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Wed, 8 Jul 2026 15:29:08 +0000 Subject: [PATCH 3/3] Add Install Go as step one in Model Repo testing guide --- serverless/modelrepotest.mdx | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/serverless/modelrepotest.mdx b/serverless/modelrepotest.mdx index 99777a29..c81dc363 100644 --- a/serverless/modelrepotest.mdx +++ b/serverless/modelrepotest.mdx @@ -27,7 +27,15 @@ export MODEL_PATH="/path/to/model" - `MODEL_NAME` should be unique for each test run. Reusing the same name uploads a new version of the existing model rather than creating a new one. - `jq` is installed for parsing JSON output. -### Step 1: Build runpodctl +### Step 1: Install Go + +Building runpodctl requires Go: + +```bash +brew install go +``` + +### Step 2: Build runpodctl ```bash git clone git@github.com:runpod/runpodctl.git @@ -35,7 +43,7 @@ cd runpodctl make ``` -### Step 2: Upload the model +### Step 3: Upload the model ```bash ./bin/runpodctl model add \ @@ -46,7 +54,7 @@ make This outputs a JSON string listing all uploaded files. -### Step 3: Wait for the model to be hashed +### Step 4: Wait for the model to be hashed After upload, the model must be hashed by an asynchronous background process. This typically completes in a few minutes but can take up to 10–15 minutes. @@ -70,7 +78,7 @@ Once hashing is complete, it returns the hash value: 71a311bdf0ca44119ed74dbef8cf573bc89b58cbc48a10fe508f756ebb1922dc ``` -### Step 4: Build the model URL +### Step 5: Build the model URL ```bash export USER_ID="$(./bin/runpodctl user | jq -r '.id')" @@ -78,7 +86,7 @@ export MODEL_HASH="$(./bin/runpodctl model list --name "$MODEL_NAME" | jq -r '.[ export MODEL_URL="https://local/${USER_ID}/${MODEL_NAME}:${MODEL_HASH}" ``` -### Step 5: Deploy a Serverless endpoint with the model attached +### Step 6: Deploy a Serverless endpoint with the model attached ```bash ./bin/runpodctl serverless create \ @@ -95,7 +103,7 @@ Notes: - `--gpu-id` accepts a single GPU ID — do not pass a comma-separated list. - `--model-reference` is repeatable if multiple models need to be attached. -### Step 6: Verify the model is attached to the worker +### Step 7: Verify the model is attached to the worker 1. Go to **Serverless** in the left navigation bar under **Resources**. 2. Select the endpoint you created (`ctl_test` if you used the commands above). @@ -112,7 +120,7 @@ find /runpod-volume/huggingface-cache/hub/models--$(echo $MODEL_NAME | sed 's@/@ There is currently no way to retrieve SSH connection details for a running Serverless worker via `runpodctl`. Use the web UI to connect. -### Step 7: Clean up +### Step 8: Clean up Delete the endpoint after testing to stop spend. Use the web UI or: