Skip to content

Commit 34bfcfa

Browse files
author
Harika
committed
Adding model-deployment guide
Signed-off-by: Harika <codewith3@gmail.com>
1 parent 0af42cb commit 34bfcfa

1 file changed

Lines changed: 10 additions & 18 deletions

File tree

third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,10 @@
88
- [3. DNS and SSL/TLS Setup](#3-dns-and-ssltls-setup)
99
- [4. Hugging Face Token Setup](#4-hugging-face-token-setup)
1010
- [Single Node Deployment Guide](#single-node-deployment-guide)
11-
- [1. Clone the Repository](#1-clone-the-repository)
12-
- [2. Configure the Setup Files and Environment](#2-configure-the-setup-files-and-environment)
13-
- [3. Run the Deployment](#3-run-the-deployment)
14-
- [4. Verify the Deployment](#4-verify-the-deployment)
15-
- [5. Test the Inference](#5-test-the-inference)
11+
- [1. Configure the Setup Files and Environment](#1-configure-the-setup-files-and-environment)
12+
- [2. Run the Deployment](#2-run-the-deployment)
13+
- [3. Verify the Deployment](#3-verify-the-deployment)
14+
- [4. Test the Inference](#4-test-the-inference)
1615
- [Summary](#summary)
1716

1817
---
@@ -135,21 +134,14 @@ SSH keys are required to allow **Ansible** or automation scripts to connect secu
135134
## Single Node Deployment Guide
136135
This section explains how to deploy Intel® AI for Enterprise Inference on a single Ubuntu 22.04 server.
137136

138-
### 1. Clone the Repository
137+
### 1. Configure the Setup Files and Environment
139138

140-
```bash
141-
git clone https://github.com/opea-project/Enterprise-Inference.git
142-
cd Enterprise-Inference
143-
git checkout ${RELEASE}
144-
```
145-
> **Note:** Update the RELEASE environment variable to point to the desired Enterprise Inference version(for example: release-1.4.0)
146-
147-
### 2. Configure the Setup Files and Environment
139+
Clone the repository, If repo is not downloaded on target machine.
148140

149141
**Update inference-config.cfg**
150142

151143
```bash
152-
vi core/inventory/inference-config.cfg
144+
vi Enterprise-Inference/core/inventory/inference-config.cfg
153145
```
154146
> **Note:** Update configuration files for single node genai deployment, Below are the changes needed.
155147
> * Replace cluster_url with your DNS , it must match with DNS used in certs generation.
@@ -189,7 +181,7 @@ cp -f docs/examples/single-node/hosts.yaml core/inventory/hosts.yaml
189181
> Note: The ansible_user field is set to ubuntu by default. Change it to the actual username used.
190182

191183

192-
### 3. Run the Deployment
184+
### 2. Run the Deployment
193185

194186
> **Note:**
195187
> The `--models` argument selects a model using its **numeric ID**
@@ -214,7 +206,7 @@ When prompted, choose option **1) Provision Enterprise Inference Cluster** and c
214206
If using Intel® Gaudi® hardware, make sure firmware and drivers are updated before running this script.
215207

216208

217-
### 4. Verify the Deployment
209+
### 3. Verify the Deployment
218210
Verify Pods Status
219211
```bash
220212
kubectl get pods -A
@@ -224,7 +216,7 @@ Expected States:
224216
- No CrashLoopBackOff
225217
- No Pending pods
226218

227-
### 5. Test the Inference
219+
### 4. Test the Inference
228220

229221
```bash
230222
export BASE_URL=https://api.example.com

0 commit comments

Comments
 (0)