Skip to content

Commit b4fb279

Browse files
authored
Feature/jmeter update (#217)
* jmeter script update * Reducing report
1 parent 8da72c6 commit b4fb279

16 files changed

Lines changed: 522 additions & 65 deletions

.cursor/rules/151-java-performance-jmeter.mdc

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ show_usage() {
113113
echo ""
114114
echo "ENVIRONMENT VARIABLES:"
115115
echo " JMETER_LOOPS Override default number of loops"
116-
echo " JMETER_THREADS Override default number of threads"
116+
echo " JMETER_THREADS Override default number of threads"
117117
echo " JMETER_RAMP_UP Override default ramp-up period"
118118
echo ""
119119
echo "EXAMPLES:"
@@ -174,7 +174,7 @@ check_jmeter() {
174174
print_info "Download from: https://jmeter.apache.org/download_jmeter.cgi"
175175
exit 1
176176
fi
177-
177+
178178
JMETER_VERSION=$(jmeter -v 2>&1 | head -n 1 | grep -o 'Version [0-9.]*' | cut -d' ' -f2)
179179
print_info "Found JMeter version: $JMETER_VERSION"
180180
}
@@ -201,7 +201,7 @@ clean_previous_results() {
201201
rm "$RESULTS_FILE"
202202
print_info "Cleaned previous results file"
203203
fi
204-
204+
205205
if [[ -d "$REPORT_DIR" ]]; then
206206
rm -rf "$REPORT_DIR"
207207
print_info "Cleaned previous report directory"
@@ -215,7 +215,7 @@ run_jmeter_test() {
215215
print_info " - Loops: $JMETER_LOOPS"
216216
print_info " - Threads: $JMETER_THREADS"
217217
print_info " - Ramp-up: $JMETER_RAMP_UP seconds"
218-
218+
219219
# Run JMeter in non-GUI mode
220220
jmeter -n \
221221
-t "$TEST_PLAN" \
@@ -226,7 +226,7 @@ run_jmeter_test() {
226226
-Jthreads="$JMETER_THREADS" \
227227
-Jrampup="$JMETER_RAMP_UP" \
228228
-j "$LOG_FILE"
229-
229+
230230
if [[ $? -eq 0 ]]; then
231231
print_success "JMeter test completed successfully"
232232
else
@@ -240,13 +240,13 @@ run_jmeter_gui() {
240240
print_info "Opening JMeter GUI..."
241241
print_info "Test plan will be loaded: $TEST_PLAN"
242242
print_info "You can configure and run tests manually in the GUI"
243-
243+
244244
# Run JMeter in GUI mode with test plan loaded
245245
jmeter -t "$TEST_PLAN" \
246246
-Jloops="$JMETER_LOOPS" \
247247
-Jthreads="$JMETER_THREADS" \
248248
-Jrampup="$JMETER_RAMP_UP"
249-
249+
250250
print_success "JMeter GUI session completed"
251251
}
252252

@@ -256,12 +256,12 @@ show_results() {
256256
print_info "Results file: $RESULTS_FILE"
257257
print_info "HTML report: $REPORT_DIR/index.html"
258258
print_info "Log file: $LOG_FILE"
259-
259+
260260
if [[ -f "$RESULTS_FILE" ]]; then
261261
TOTAL_SAMPLES=$(tail -n +2 "$RESULTS_FILE" | wc -l)
262262
print_info "Total samples: $TOTAL_SAMPLES"
263263
fi
264-
264+
265265
# Try to open the HTML report
266266
if command -v open &> /dev/null; then
267267
print_info "Opening HTML report in browser..."
@@ -278,10 +278,10 @@ show_results() {
278278
main() {
279279
print_info "JMeter Load Test Script"
280280
print_info "======================="
281-
281+
282282
check_jmeter
283283
check_test_plan
284-
284+
285285
if [[ "$GUI_MODE" == "true" ]]; then
286286
# GUI mode - just open JMeter with the test plan
287287
run_jmeter_gui
@@ -297,8 +297,7 @@ main() {
297297
}
298298

299299
# Execute main function
300-
main "$@"
301-
300+
main "$@"
302301
```
303302

304303
#### Step Constraints

.cursor/rules/161-java-profiling-detect.mdc

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,8 @@ OPTIONS:
103103
-c, --class CLASS Main class to run (default: info.jab.ms.MainApplication)
104104
-h, --heap SIZE Heap size (default: 512m)
105105
-p, --profile PROFILE Profile to activate (default: default)
106+
For Spring Boot: sets spring.profiles.active
107+
For Quarkus: sets quarkus.profile
106108
--jfr Enable Java Flight Recorder (disabled by default)
107109
--gc-log Enable GC logging (disabled by default)
108110
--help Show this help message
@@ -122,6 +124,12 @@ EXAMPLES:
122124
# Force Quarkus with custom heap size and profile
123125
$0 -f quarkus -h 1g -p dev
124126

127+
# Run Spring Boot with virtual threads profile
128+
$0 -f springboot -p vt
129+
130+
# Run Spring Boot with development profile
131+
$0 -f springboot -p dev
132+
125133
# Run with wall-clock profiling
126134
$0 -m wall
127135

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,4 @@
22
target/
33
.idea/
44
.vscode/
5-
*.log
65
.classpath

README.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ With this structure in mind, the project uses an XML Schema to define the way th
5151

5252
## Cursor Rules
5353

54-
Read the generated list of cursor rules for Java [here](./CURSOR-RULES-JAVA.md). The set of cursor rules covers aspects like `Build system based on Maven`, `Design`, `Coding`, `Testing`, `Refactoring`, `Performance testing with JMeter` & `Profiling with Async Profiler`.
54+
Read the generated list of cursor rules for Java [here](./CURSOR-RULES-JAVA.md). The set of cursor rules covers aspects like `Build system based on Maven`, `Design`, `Coding`, `Testing`, `Refactoring`, `Performance testing with JMeter` & `Profiling with Async profiler, jps, jstack, jcmd & jstat`.
5555

5656
## Constraints, Output format & Safety guards
5757

@@ -61,9 +61,15 @@ The **constraints** act as gate-keeping mechanisms that define hard requirements
6161

6262
This framework transforms AI from a general assistant into a specialized consultant with built-in quality controls and safety measures, making it particularly suitable for critical applications like Java software development. By embedding domain-specific expertise directly into the prompt structure, the cursor rules provide predictable, comprehensive, and safe interactions while reducing cognitive load for developers and ensuring system integrity throughout the development process.
6363

64-
## Limitations & Opportunities
64+
## Limitations
6565

66-
From the beginning, you need to know that results provided by the interaction with the different `Cursor rules` are not deterministic due to the models' nature, but this fact should not be considered in a negative, Software engineers not always has the same idea to solve a problem, you could find an analogy on this fact.
66+
### Lack of determinism
67+
68+
From the beginning, you need to know that results provided by the interaction with the different `Cursor rules` are not deterministic due to the models' nature, but this fact should not be considered negative. Software engineers do not always have the same idea to solve a problem, and you could find an analogy in this fact.
69+
70+
### Limits of interactions with Models
71+
72+
Models are able to generate code but they are not able to run code with your local data. In order to solve that limitation you can observe that a few prompts provide scripts to fix the gaps on the model side.
6773

6874
## Contribute
6975

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# JMeter Performance Comparison Analysis - July 20, 2025
2+
3+
## Executive Summary
4+
- **Analysis Objective**: Compare performance between two JMeter test executions on Spring Boot Memory Leak Demo
5+
- **Test Executions**: 223808 vs 223856 (timestamps)
6+
- **Overall Result**: ✅ **Test 1 (223808) demonstrates superior performance**
7+
- **Key Performance Gain**: 19.3% faster average response times, 4.3% higher throughput
8+
9+
## Test Execution Overview
10+
11+
| Metric | Test 1 (223808) | Test 2 (223856) | Performance Winner |
12+
|--------|------------------|------------------|-------------------|
13+
| **Execution Start Time** | 2025-07-20 22:38:08 | 2025-07-20 22:38:56 | - |
14+
| **Test Duration** | ~10.18 seconds | ~10.62 seconds | Test 1 ✓ |
15+
| **Total Requests** | 60 | 60 | Tie |
16+
| **Error Rate** | 0% | 0% | Tie |
17+
| **Success Rate** | 100% | 100% | Tie |
18+
19+
## Detailed Performance Metrics
20+
21+
### Response Time Analysis
22+
23+
#### GET /api/v1/objects/create
24+
| Metric | Test 1 (223808) | Test 2 (223856) | Improvement |
25+
|--------|------------------|------------------|-------------|
26+
| **Mean Response Time** | 7.45ms | 9.85ms | **24.3% faster**|
27+
| **Median Response Time** | 5.0ms | 6.5ms | **23.1% faster**|
28+
| **Min Response Time** | 2.0ms | 2.0ms | Tie |
29+
| **Max Response Time** | 58.0ms | 83.0ms | **30.1% faster**|
30+
| **90th Percentile** | 7.9ms | 12.6ms | **37.3% faster**|
31+
| **95th Percentile** | 55.5ms | 79.5ms | **30.2% faster**|
32+
33+
#### GET /api/v1/threads/create
34+
| Metric | Test 1 (223808) | Test 2 (223856) | Improvement |
35+
|--------|------------------|------------------|-------------|
36+
| **Mean Response Time** | 5.05ms | 5.7ms | **11.4% faster**|
37+
| **Median Response Time** | 5.0ms | 6.0ms | **16.7% faster**|
38+
| **Min Response Time** | 2.0ms | 3.0ms | **33.3% faster**|
39+
| **Max Response Time** | 8.0ms | 9.0ms | **11.1% faster**|
40+
| **90th Percentile** | 7.9ms | 8.0ms | **1.3% faster**|
41+
42+
#### GET /actuator/health
43+
| Metric | Test 1 (223808) | Test 2 (223856) | Improvement |
44+
|--------|------------------|------------------|-------------|
45+
| **Mean Response Time** | 8.45ms | 10.4ms | **18.8% faster**|
46+
| **Median Response Time** | 7.0ms | 8.0ms | **12.5% faster**|
47+
| **Min Response Time** | 2.0ms | 3.0ms | **33.3% faster**|
48+
| **Max Response Time** | 46.0ms | 63.0ms | **27.0% faster**|
49+
| **90th Percentile** | 8.9ms | 10.8ms | **17.6% faster**|
50+
51+
### Throughput Analysis
52+
53+
| Endpoint | Test 1 (223808) | Test 2 (223856) | Performance Gain |
54+
|----------|------------------|------------------|------------------|
55+
| **GET /api/v1/objects/create** | 2.067 req/sec | 1.956 req/sec | +5.7% ✓ |
56+
| **GET /api/v1/threads/create** | 2.108 req/sec | 1.997 req/sec | +5.6% ✓ |
57+
| **GET /actuator/health** | 2.106 req/sec | 2.011 req/sec | +4.7% ✓ |
58+
| **Overall Throughput** | 5.889 req/sec | 5.644 req/sec | **+4.3%**|
59+
60+
### Data Transfer Metrics
61+
62+
| Metric | Test 1 (223808) | Test 2 (223856) | Performance Gain |
63+
|--------|------------------|------------------|------------------|
64+
| **Received KB/sec** | 1.160 KB/sec | 1.112 KB/sec | +4.3% ✓ |
65+
| **Sent KB/sec** | 0.776 KB/sec | 0.744 KB/sec | +4.3% ✓ |
66+
67+
## Overall Performance Summary
68+
69+
| Performance Metric | Test 1 (223808) | Test 2 (223856) | Improvement |
70+
|-------------------|------------------|------------------|-------------|
71+
| **Overall Mean Response Time** | 6.98ms | 8.65ms | **19.3% faster**|
72+
| **Overall Median Response Time** | 6.0ms | 7.0ms | **14.3% faster**|
73+
| **Overall Min Response Time** | 2.0ms | 2.0ms | Tie |
74+
| **Overall Max Response Time** | 58.0ms | 83.0ms | **30.1% faster**|
75+
| **Overall Throughput** | 5.889 req/sec | 5.644 req/sec | **4.3% higher**|
76+
77+
## Key Performance Insights
78+
79+
### 🏆 Test 1 (223808) Advantages
80+
81+
1. **Consistent Superior Performance**: Test 1 outperformed Test 2 across ALL measured metrics
82+
2. **Better Response Time Consistency**: Lower maximum response times indicate more predictable performance
83+
3. **Higher Efficiency**: Completed the same workload 4.1% faster
84+
4. **Improved Stability**: Better worst-case performance scenarios
85+
86+
### 📊 Most Significant Improvements
87+
88+
1. **Objects Endpoint**: 24.3% faster average response time (most critical improvement)
89+
2. **Maximum Response Times**: 30.1% improvement in worst-case scenarios
90+
3. **90th Percentile Performance**: Up to 37.3% faster for objects endpoint
91+
92+
### 🔍 Performance Patterns
93+
94+
#### Response Time Distribution
95+
- **Test 1**: More consistent performance with tighter distribution
96+
- **Test 2**: Higher variability, especially in maximum response times
97+
98+
#### Endpoint-Specific Analysis
99+
- **Most Improved**: `/api/v1/objects/create` (24.3% faster)
100+
- **Most Consistent**: `/api/v1/threads/create` (11.4% faster)
101+
- **Health Check**: `/actuator/health` (18.8% faster)
102+
103+
## Technical Analysis
104+
105+
### Possible Performance Factors
106+
107+
#### Test 1 (223808) Advantages
108+
1. **JVM Warm-up**: Better Just-In-Time (JIT) compilation optimization
109+
2. **Memory Management**: More efficient garbage collection patterns
110+
3. **System Resources**: Lower system contention during execution
111+
4. **Application State**: Better internal caching or connection pooling
112+
113+
#### Test 2 (223856) Challenges
114+
1. **Cold Start Effects**: Less optimal JVM state at test initiation
115+
2. **Resource Contention**: Potential system load from other processes
116+
3. **Memory Pressure**: Possible GC overhead affecting performance
117+
4. **Network Conditions**: Minor network latency variations
118+
119+
### Statistical Significance
120+
121+
#### Sample Analysis (60 requests total, 20 per endpoint)
122+
- **Sample Size**: Adequate for trend identification
123+
- **Error Rate**: 0% (excellent test validity)
124+
- **Consistency**: Multiple endpoints show same performance pattern
125+
126+
#### Confidence Level
127+
- **High Confidence**: Consistent improvement across all metrics
128+
- **Reproducible Pattern**: All three endpoints show Test 1 superiority
129+
- **Significant Margins**: 19.3% overall improvement indicates real performance difference
130+
131+
## Environmental Context
132+
133+
### Test Environment Factors
134+
- **Time Gap**: 48 seconds between test executions
135+
- **System State**: Potential differences in JVM state, memory usage, system load
136+
- **Application State**: Possible cache warming or connection pool optimization
137+
138+
### Memory Leak Context
139+
Based on related profiling documentation, these tests were conducted during memory leak investigation:
140+
- **Controller**: Tests likely used `CocoController` (with known memory leaks)
141+
- **Memory State**: System may have been under memory pressure
142+
- **Performance Impact**: Memory leaks could affect test-to-test consistency
143+
144+
## Recommendations
145+
146+
### ✅ Immediate Actions
147+
1. **Adopt Test 1 Configuration**: Use whatever conditions produced Test 1 results as baseline
148+
2. **Investigate Environmental Factors**: Identify what made Test 1 perform better
149+
3. **Establish Performance Baseline**: Use Test 1 metrics as target performance
150+
151+
### 🔄 Performance Optimization
152+
1. **JVM Tuning**: Optimize JVM parameters for consistent performance like Test 1
153+
2. **Warm-up Strategy**: Implement application warm-up procedures
154+
3. **Monitoring**: Establish continuous performance monitoring using Test 1 as baseline
155+
156+
### 📊 Future Testing
157+
1. **Extended Testing**: Run longer-duration tests to validate consistency
158+
2. **Load Variation**: Test with different load patterns
159+
3. **Environmental Control**: Standardize test environment conditions
160+
161+
## Validation and Next Steps
162+
163+
### Performance Validation Checklist
164+
- [x] Response time analysis completed
165+
- [x] Throughput analysis completed
166+
- [x] Error rate verification completed
167+
- [x] Statistical significance assessed
168+
- [x] Environmental factors considered
169+
170+
### Next Steps
171+
1. **Root Cause Analysis**: Investigate why Test 1 performed better
172+
2. **Performance Tuning**: Apply lessons learned to optimize for Test 1-like performance
173+
3. **Monitoring Setup**: Implement alerting for performance regression
174+
4. **Load Testing**: Validate findings with sustained load tests
175+
176+
## Conclusion
177+
178+
### 🎯 Clear Winner: Test 1 (223808)
179+
180+
**Test 1 demonstrates superior performance across every measured metric**, with particularly significant improvements in:
181+
- **19.3% faster overall response times**
182+
- **4.3% higher throughput**
183+
- **30.1% better worst-case performance**
184+
185+
### 📈 Business Impact
186+
- **Better User Experience**: Faster response times improve customer satisfaction
187+
- **Higher Capacity**: 4.3% throughput improvement supports more concurrent users
188+
- **Improved Reliability**: Better worst-case performance reduces timeout risks
189+
190+
### 🔍 Technical Recommendation
191+
**Use Test 1 (223808) configuration and conditions as the performance baseline** for production deployment and future optimization efforts.
192+
193+
---
194+
195+
**Analysis Date**: July 20, 2025
196+
**Test Data Sources**:
197+
- `jmeter-result-20250720-223808.jtl`
198+
- `jmeter-result-20250720-223856.jtl`
199+
- `statistics.json` files from both test reports
200+
201+
**Status**: ✅ **Analysis Complete - Test 1 Recommended**
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[92.858s][info][gc] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 298M->49M(512M) 38.190ms
2+
[188.772s][info][gc] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 338M->77M(512M) 69.008ms
3+
[282.978s][info][gc] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 361M->105M(512M) 74.681ms
4+
[376.662s][info][gc] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 395M->135M(512M) 76.817ms
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[291.583s][info][gc] GC(7) Pause Young (Prepare Mixed) (G1 Evacuation Pause) 311M->22M(512M) 20.482ms
2+
[570.492s][info][gc] GC(8) Pause Young (Mixed) (G1 Evacuation Pause) 287M->22M(512M) 16.636ms

0 commit comments

Comments
 (0)