Skip to content

Commit 233099d

Browse files
zschafft-intelzschafft-intel
committed
call pqos setup, add .csv results parsing script, initial documentation for running demo and analysis of end to end plc benchmark
1 parent c6b57a4 commit 233099d

5 files changed

Lines changed: 663 additions & 1 deletion

File tree

‎DEMO_AND_ANALYSIS.md‎

Lines changed: 285 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,285 @@
1+
# HDE2E Demo and Analysis Guide
2+
3+
This document describes how to run the High Density End-2-End (HDE2E) demo benchmark and analyze the results.
4+
5+
## Prerequisites
6+
7+
- `uv` package manager installed
8+
- Network proxy configured (if behind corporate firewall)
9+
- Docker installed and running
10+
- `sudo` access for hardware-level operations
11+
12+
## Quick Start
13+
14+
### 1. Run the HDE2E Demo
15+
16+
```bash
17+
# If NOT behind a proxy:
18+
sudo $(which uv) run main.py
19+
20+
# If behind a proxy, set environment variables:
21+
sudo http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT https_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT $(which uv) run main.py
22+
```
23+
24+
Replace `YOUR_PROXY_HOST` and `YOUR_PROXY_PORT` with your proxy settings (e.g., `proxy.company.com:8080`).
25+
26+
**What this does:**
27+
- Collects system information
28+
- Applies PQOS cache and memory bandwidth optimizations (if enabled)
29+
- Launches CODESYS control and IO PLC instances
30+
- Sets up Docker networking and port forwarding
31+
- Runs the HDE2E latency and jitter benchmark
32+
33+
**Expected output:**
34+
- System info saved to `outputs/YYYY-MM-DD/HH-MM-SS/sysinfo.json`
35+
- Docker containers running:
36+
- `Control_PLC_01` on cores 3,5
37+
- `Control_PLC_02` on cores 7,9
38+
- `IO_PLC_01` and `IO_PLC_02` on remote system (10.34.106.119)
39+
- Port forwarding configured:
40+
- Control web UI: ports 30080, 30081
41+
- IO web UI: ports 30090, 30091
42+
- CODESYS gateway: ports 11741, 11742
43+
44+
### 2. Retrieve Results from IO System
45+
46+
The benchmark generates result files on the remote IO system. Copy them to your local machine:
47+
48+
```bash
49+
# Create results directory
50+
mkdir -p ~/Results_testing
51+
52+
# Copy result files from IO system (10.34.106.119)
53+
# Results are stored in: ~/dockerMount/IO_PLC_01/data/codesyscontrol/PlcLogic/Results/
54+
scp intel@10.34.106.119:~/dockerMount/IO_PLC_01/data/codesyscontrol/PlcLogic/Results/*.csv ~/Results_testing/
55+
```
56+
57+
**Note:** You may need to use the same SSH password from `conf/config.yaml` or set up key-based authentication.
58+
59+
### 3. Analyze Results
60+
61+
```bash
62+
# If NOT behind a proxy:
63+
sudo $(which uv) run hde2e-analyze /path/to/results [options]
64+
65+
# If behind a proxy:
66+
sudo http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT https_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT $(which uv) run hde2e-analyze /path/to/results [options]
67+
```
68+
69+
**Options:**
70+
- `-v, --verbose` - Print detailed statistics
71+
- `-d, --debug` - Print debug information
72+
- `--save` - Save plots as PNG files
73+
- `--show` - Display plots interactively (use only with single instance for testing)
74+
- `--rows N` - Read only first N rows of input data
75+
- `--version` - Show version
76+
77+
**Example with output:**
78+
```bash
79+
sudo http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT https_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT $(which uv) run hde2e-analyze /home/intel/Results_testing -v --save
80+
```
81+
82+
**Output files generated:**
83+
- `stat_Latency.csv` - Latency statistics
84+
- `stat_Jitter.csv` - Jitter statistics
85+
- `box_Latency.png` - Box plot for latency
86+
- `box_Jitter.png` - Box plot for jitter
87+
- `bar_Latency.png` - Bar chart for latency statistics
88+
- `bar_Jitter.png` - Bar chart for jitter statistics
89+
- `scatter_*.png` - Scatter plots for each latency component and jitter
90+
91+
## Configuration
92+
93+
### Demo Mode Settings
94+
95+
Edit `conf/config.yaml` to customize the demo:
96+
97+
```yaml
98+
demo:
99+
demo_mode: true # Enable demo mode
100+
t_core: [9,11] # Target cores
101+
io_system:
102+
ip: 10.34.106.119 # Remote IO system IP
103+
nic: enp3s0 # Network interface
104+
ssh_user: intel # SSH username
105+
ssh_password: "..." # SSH password
106+
ssh_port: 22 # SSH port
107+
t_cpus: "1,3" # IO system target CPUs
108+
control_system:
109+
ip: localhost # Control system IP
110+
nic: eno12399np0 # Network interface
111+
t_cpus: "3,5,7,9" # Control CPUs
112+
```
113+
114+
### PQOS Optimizations
115+
116+
PQOS settings are automatically applied when running the demo. Configure them in `conf/config.yaml`:
117+
118+
```yaml
119+
pqos:
120+
interface: "os" # Use Linux resctrl interface
121+
reset_before_apply: true # Reset before applying
122+
123+
classes:
124+
- id: 1
125+
description: "real-time workload"
126+
l3_mask: "0x00ff" # L3 cache mask (8 ways)
127+
l2_mask: "0x00ff" # L2 cache mask (8 ways)
128+
mba: 100 # Memory bandwidth 100%
129+
pids: [] # PIDs (empty for demo)
130+
cores: [] # Cores (empty for demo)
131+
132+
- id: 0
133+
description: "background worker"
134+
l3_mask: "0x7f00" # L3 cache mask (7 ways)
135+
l2_mask: "0xff00" # L2 cache mask (8 ways)
136+
mba: 10 # Memory bandwidth 10%
137+
cores: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
138+
pids: [115, 118]
139+
```
140+
141+
## Complete Workflow Example
142+
143+
```bash
144+
# Set proxy variables (if behind a proxy)
145+
export http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT
146+
export https_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT
147+
148+
# 1. Run the demo (takes ~5-10 minutes)
149+
sudo $(which uv) run main.py
150+
151+
# 2. The benchmark completes - result files are generated on IO system
152+
153+
# 3. Copy results from remote IO system to local machine
154+
mkdir -p ~/Results_testing
155+
scp intel@10.34.106.119:~/dockerMount/IO_PLC_01/data/codesyscontrol/PlcLogic/Results/*.csv ~/Results_testing/
156+
157+
# 4. Verify results were copied
158+
ls -lh ~/Results_testing/Codesys-*.csv
159+
160+
# 5. Analyze results with statistics and plots
161+
sudo $(which uv) run hde2e-analyze ~/Results_testing -v --save
162+
163+
# 6. View generated files
164+
ls -lh ~/Results_testing/*.{csv,png}
165+
```
166+
167+
## Cleaning Up
168+
169+
### Stop Running Containers
170+
171+
```bash
172+
sudo docker stop Control_PLC_01 Control_PLC_02
173+
sudo docker rm Control_PLC_01 Control_PLC_02
174+
```
175+
176+
### Kill Port Forwarding
177+
178+
```bash
179+
sudo pkill -f socat
180+
```
181+
182+
### Full Cleanup (if containers are stuck)
183+
184+
```bash
185+
sudo systemctl restart docker
186+
```
187+
188+
## Troubleshooting
189+
190+
### SSH / SCP Issues
191+
192+
**Error: `Permission denied (publickey,password)`**
193+
- Ensure you have the correct SSH password from `conf/config.yaml`
194+
- Try with explicit password prompt:
195+
```bash
196+
scp -P 22 intel@10.34.106.119:/home/intel/dockerMount/IO_PLC_01/data/codesyscontrol/PlcLogic/Results*.csv ~/Results_testing/
197+
```
198+
- If using key-based auth, ensure your public key is on the IO system
199+
200+
**Error: `No such file or directory`**
201+
- Verify the results directory path on the IO system
202+
- Connect to IO system to check: `ssh intel@10.34.106.119 ls -la /home/intel/dockerMount/IO_PLC_01/data/codesyscontrol/PlcLogic/Results`
203+
- Results may be in a different location; check the benchmark output logs
204+
205+
**Slow SCP Transfer**
206+
- Results files can be large (20MB+)
207+
- Use compression: `scp -C intel@10.34.106.119:/path/to/results/*.csv ~/Results_testing/`
208+
209+
### Network Issues
210+
211+
**Error: `Failed to connect to github.com`**
212+
- Ensure proxy is configured: `git config --global http.proxy http://YOUR_PROXY_HOST:YOUR_PROXY_PORT`
213+
- Or set environment variables: `export http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT`
214+
215+
**Error: `Failed to fetch from pypi.org`**
216+
- Run with proxy environment variables:
217+
```bash
218+
sudo http_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT https_proxy=http://YOUR_PROXY_HOST:YOUR_PROXY_PORT $(which uv) ...
219+
```
220+
- Or configure permanently in git: `git config --global http.proxy http://YOUR_PROXY_HOST:YOUR_PROXY_PORT`
221+
222+
### SSH Authentication
223+
224+
**Error: `Authentication failed` for IO system**
225+
- Check SSH credentials in `conf/config.yaml`
226+
- Verify network connectivity to 10.34.106.119 (IO system IP Address)
227+
- Ensure SSH password is correct
228+
229+
### PQOS Issues
230+
231+
**Error: `pqos tool is not installed`**
232+
- The demo will skip PQOS if not available
233+
- Install with: `sudo apt-get install intel-cmt-cat`
234+
235+
**Error: `ROOT ACCESS REQUIRED`**
236+
- All operations require `sudo` for hardware access
237+
238+
### Docker Issues
239+
240+
**Error: `Container stuck in exited state`**
241+
- Restart Docker: `sudo systemctl restart docker`
242+
- Force remove: `sudo docker rm -f <container_id>`
243+
244+
## Output Format
245+
246+
### Statistics Output
247+
248+
The analysis script produces statistics in multiple formats:
249+
250+
**Console Output (verbose mode):**
251+
```
252+
#### T2-T1 ############################
253+
mean std min max
254+
IO_01 355.4 144.5 195.0 754.0
255+
```
256+
257+
**CSV Files:**
258+
- Statistics at percentiles: 0.9, 0.99, 0.999, 0.9999, 0.99999
259+
- One row per instance
260+
- Columns: mean, std, min, max, and percentiles
261+
262+
### Latency Components
263+
264+
- **T2-T1**: IO to Control transmission time
265+
- **T3-T2**: Control processing time
266+
- **T4-T3**: Control to IO transmission time
267+
- **T4-T1**: Total end-to-end latency
268+
- **Jitter**: PubSub cycle time variation
269+
270+
## Performance Metrics
271+
272+
The HDE2E benchmark measures:
273+
- **Latency**: Time for data to traverse from IO → Control → IO (in microseconds)
274+
- **Jitter**: Variation in PubSub task cycle time (in microseconds)
275+
276+
With PQOS optimizations enabled:
277+
- Real-time workload (Class 1) gets priority cache access
278+
- Background workers (Class 0) are isolated with limited bandwidth
279+
- Results in lower latency and reduced jitter variance
280+
281+
## References
282+
283+
- [CODESYS Documentation](https://help.codesys.com/)
284+
- [Intel CAT/MBA Documentation](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-rdt-cat-mba-linux-rtos-support.html)
285+
- [Docker Documentation](https://docs.docker.com/)

‎main.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ def main(cfg: DictConfig):
9595

9696
if cfg.demo.demo_mode:
9797
print("Running in demo mode. Skipping test execution.")
98+
setup_pqos(cfg)
9899
runner = DockerHDE2E(cfg)
99100
print("Starting demo HDE2E test...")
100101
print("Starting IO...")

‎pyproject.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,5 +28,11 @@ dependencies = [
2828
"tqdm>=4.67.1",
2929
]
3030

31+
[project.scripts]
32+
hde2e-analyze = "hde2e_results:main"
33+
3134
[tool.uv.sources]
3235
hurst = { git = "https://github.com/Mottl/hurst" }
36+
37+
[tool.uv]
38+
package = true

0 commit comments

Comments
 (0)