Skip to content

Commit d15028b

Browse files
Nolin NaidooNolin Naidoo
Nolin Naidoo
authored and
Nolin Naidoo
committed
Add performance monitoring and testing documentation
- Add docs/PERFORMANCE.md - Performance monitoring documentation - Add docs/TESTING.md - Testing guidelines with core principle - Update README.md: - Link to PERFORMANCE.md in Performance section - Add core principle: 'No broken or failed tests in commits' - Link to TESTING.md in Testing section - All tests must pass before committing/merging - CI/CD validates on all platforms (Ubuntu, macOS, Windows)
1 parent 6bf4ace commit d15028b

3 files changed

Lines changed: 338 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,8 @@ For the complete list of available settings, open VS Code Settings and search fo
242242

243243
Secrets-LE includes built-in performance monitoring and configurable thresholds to help track operation speed and resource usage.
244244

245+
For detailed information, see [Performance Monitoring](docs/PERFORMANCE.md).
246+
245247
## 🔧 Troubleshooting
246248

247249
**Not detecting secrets?**
@@ -271,13 +273,19 @@ High accuracy with configurable sensitivity to reduce false positives
271273

272274
**17 unit tests across 1 test file** • Powered by Vitest • Run with `bun run test:coverage`
273275

276+
### Core Principle
277+
278+
**No broken or failed tests are allowed in commits.** All tests must pass before code can be committed or merged.
279+
274280
### Test Suite Highlights
275281

276282
- **Comprehensive secret detection** across 15+ types
277283
- **Sanitization validation** with replacement verification
278284
- **Error handling** with graceful degradation
279285
- **Security-focused** testing for edge cases
280286

287+
For detailed testing guidelines, see [Testing Guidelines](docs/TESTING.md).
288+
281289
---
282290

283291
Copyright © 2025

‎docs/PERFORMANCE.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Secrets-LE Performance Monitoring
2+
3+
Secrets-LE includes built-in performance monitoring capabilities to track operation metrics and ensure optimal performance during secret detection and sanitization.
4+
5+
## Overview
6+
7+
Performance monitoring is automatically enabled for all detection and sanitization operations. The system tracks:
8+
9+
- **Execution Time** - How long operations take to complete
10+
- **Memory Usage** - Heap memory consumed during operations
11+
- **CPU Usage** - System and user CPU time
12+
- **Operation Metrics** - Secrets detected, errors, warnings
13+
14+
## Configuration
15+
16+
Performance monitoring can be configured in VS Code settings:
17+
18+
```json
19+
{
20+
"secrets-le.performance.enabled": true,
21+
"secrets-le.performance.maxDuration": 5000,
22+
"secrets-le.performance.maxMemoryUsage": 104857600,
23+
"secrets-le.performance.maxCpuUsage": 1000000
24+
}
25+
```
26+
27+
### Settings
28+
29+
- **`secrets-le.performance.enabled`** - Enable/disable performance monitoring (default: `true`)
30+
- **`secrets-le.performance.maxDuration`** - Maximum operation duration in milliseconds (default: `5000`)
31+
- **`secrets-le.performance.maxMemoryUsage`** - Maximum memory usage in bytes (default: `104857600` = 100MB)
32+
- **`secrets-le.performance.maxCpuUsage`** - Maximum CPU usage in microseconds (default: `1000000` = 1 second)
33+
34+
## How It Works
35+
36+
### Automatic Monitoring
37+
38+
When you run detection or sanitization commands, performance monitoring automatically:
39+
40+
1. **Tracks start time** and initial resource usage
41+
2. **Monitors operation progress** during execution
42+
3. **Captures end metrics** when operations complete
43+
4. **Compares against thresholds** to detect performance issues
44+
5. **Reports metrics** via telemetry (if enabled)
45+
46+
### Pattern Matching Efficiency
47+
48+
Secret detection uses optimized pattern matching:
49+
50+
- **Multiple detector types** run in parallel where possible
51+
- **Early termination** for certain patterns
52+
- **Efficient regex matching**
53+
- **Memory-efficient processing** for large files
54+
55+
## Performance Reports
56+
57+
Performance metrics are displayed in command results:
58+
59+
- Detection operations show processing time and secret count
60+
- Sanitization operations show replacement time
61+
- Large file warnings when approaching limits
62+
63+
## Best Practices
64+
65+
1. **Monitor large files** - Enable performance monitoring when scanning large codebases
66+
2. **Check thresholds** - Adjust limits based on your system capabilities
67+
3. **Review metrics** - Use performance data to optimize scanning patterns
68+
4. **Watch for warnings** - System will warn if operations exceed thresholds
69+
70+
## Troubleshooting
71+
72+
### Operations Taking Too Long
73+
74+
If operations exceed `maxDuration`:
75+
- Check file size - large files take longer to scan
76+
- Review sensitivity level - higher sensitivity scans more patterns
77+
- Consider scanning specific file types only
78+
79+
### High Memory Usage
80+
81+
If operations exceed `maxMemoryUsage`:
82+
- Large input files consume more memory
83+
- Many detected secrets increase memory usage
84+
- Consider processing files in smaller batches
85+
86+
### CPU Usage Warnings
87+
88+
If operations exceed `maxCpuUsage`:
89+
- Multiple pattern detectors increase CPU usage
90+
- Higher sensitivity levels require more CPU
91+
- Consider limiting concurrent operations
92+
93+
## Telemetry
94+
95+
When telemetry is enabled, performance metrics are logged to the Output panel for debugging:
96+
97+
- Operation duration
98+
- Memory usage
99+
- CPU usage
100+
- Secret counts
101+
- Errors and warnings
102+
103+
This helps identify performance patterns over time.
104+
105+
## Related Documentation
106+
107+
- [Commands](../README.md#commands) - Available commands and their usage
108+
- [Configuration](../README.md#configuration) - Full configuration options
109+
- [Detection Types](../README.md#detecting-api-keys--credentials) - Supported secret types
110+

‎docs/TESTING.md‎

Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
# Secrets-LE Testing Guidelines
2+
3+
This document outlines testing practices and policies for Secrets-LE development.
4+
5+
## Core Principle
6+
7+
**No broken or failed tests are allowed in commits.**
8+
9+
All tests must pass before code can be committed or merged. This ensures code quality and prevents regressions.
10+
11+
## Running Tests
12+
13+
### Run All Tests
14+
15+
```bash
16+
bun run test
17+
```
18+
19+
### Run Tests with Coverage
20+
21+
```bash
22+
bun run test:coverage
23+
```
24+
25+
### Run Tests in Watch Mode
26+
27+
```bash
28+
bun run test:watch
29+
```
30+
31+
### Run Tests for Specific File
32+
33+
```bash
34+
bun x vitest run src/extraction/detectors.test.ts
35+
```
36+
37+
## Test Structure
38+
39+
### Unit Tests
40+
41+
Located in `src/**/*.test.ts` and `src/**/*.spec.ts`:
42+
43+
- **Detector tests** - Test secret detection logic
44+
- **Utility tests** - Test helper functions
45+
- **Configuration tests** - Test config validation
46+
47+
### Integration Tests
48+
49+
Located in `src/sampleFiles.spec.ts`:
50+
51+
- **Sample file tests** - Test detection against real file formats
52+
- **Cross-platform tests** - Ensure case-sensitivity compatibility
53+
- **End-to-end workflows** - Test complete command flows
54+
55+
## Test Coverage Requirements
56+
57+
- **Minimum Coverage**: Maintain reasonable coverage across core functionality
58+
- **Critical Paths**: All detection logic must be tested
59+
- **Error Handling**: All error paths must be covered
60+
- **Edge Cases**: Boundary conditions must be tested
61+
62+
## Before Committing
63+
64+
### Checklist
65+
66+
- [ ] All tests pass (`bun run test`)
67+
- [ ] No broken tests
68+
- [ ] No skipped tests (unless intentionally)
69+
- [ ] Type checking passes (`bun x tsc -p ./`)
70+
- [ ] Linting passes (`bun run lint`)
71+
72+
### CI/CD Validation
73+
74+
The CI pipeline automatically:
75+
76+
1. Runs all tests on Ubuntu, macOS, and Windows
77+
2. Generates coverage reports
78+
3. Verifies all tests pass
79+
4. Fails the build if any tests fail
80+
81+
## Fixing Failed Tests
82+
83+
### When a Test Fails
84+
85+
1. **Don't commit the failure** - Fix the test or the code
86+
2. **Run locally first** - Verify fix works before pushing
87+
3. **Check all platforms** - Ensure fix works on Linux/Windows (case sensitivity, etc.)
88+
4. **Update test if needed** - If behavior changed intentionally, update test
89+
90+
### Common Issues
91+
92+
- **Case sensitivity** - Use exact case for file references (`README.md` not `readme.md`)
93+
- **Mock issues** - Ensure mocks are properly reset in `beforeEach`
94+
- **Pattern matching** - Test with realistic secret patterns
95+
96+
## Test Best Practices
97+
98+
### 1. Use Descriptive Test Names
99+
100+
```typescript
101+
// ✅ Good
102+
it('should detect AWS access keys', () => {
103+
// ...
104+
});
105+
106+
// ❌ Bad
107+
it('works', () => {
108+
// ...
109+
});
110+
```
111+
112+
### 2. Test One Thing Per Test
113+
114+
```typescript
115+
// ✅ Good - separate tests
116+
it('should detect AWS keys', () => { /* ... */ });
117+
it('should detect GitHub tokens', () => { /* ... */ });
118+
119+
// ❌ Bad - multiple concerns
120+
it('should detect AWS keys and GitHub tokens', () => { /* ... */ });
121+
```
122+
123+
### 3. Use Arrange-Act-Assert Pattern
124+
125+
```typescript
126+
it('should detect secret with high confidence', () => {
127+
// Arrange
128+
const content = 'const apiKey = "AKIAIOSFODNN7EXAMPLE"';
129+
130+
// Act
131+
const result = detectSecretsInContent(content, {
132+
includeApiKeys: true,
133+
sensitivity: 'high',
134+
});
135+
136+
// Assert
137+
expect(result.success).toBe(true);
138+
expect(result.secrets.length).toBeGreaterThan(0);
139+
});
140+
```
141+
142+
### 4. Clean Up Mocks
143+
144+
```typescript
145+
beforeEach(() => {
146+
vi.clearAllMocks();
147+
// Reset mocks to default state
148+
});
149+
```
150+
151+
## Cross-Platform Testing
152+
153+
### Case Sensitivity
154+
155+
Always use exact case for file references:
156+
157+
```typescript
158+
// ✅ Good - works on all platforms
159+
const content = readSampleFile('README.md');
160+
161+
// ❌ Bad - fails on Linux
162+
const content = readSampleFile('readme.md');
163+
```
164+
165+
### Path Separators
166+
167+
Use platform-agnostic path handling:
168+
169+
```typescript
170+
import { join } from 'path';
171+
const filePath = join(SAMPLE_DIR, filename);
172+
```
173+
174+
## Coverage Reports
175+
176+
Coverage reports are generated automatically:
177+
178+
- **Location**: `coverage/index.html`
179+
- **Format**: HTML, LCOV, JSON
180+
- **CI/CD**: Coverage uploaded as artifact
181+
182+
## Continuous Integration
183+
184+
### GitHub Actions
185+
186+
Tests run automatically on:
187+
188+
- **Ubuntu** (latest)
189+
- **macOS** (latest)
190+
- **Windows** (latest)
191+
192+
All platforms must pass for the build to succeed.
193+
194+
### Pre-commit Hooks
195+
196+
Consider setting up pre-commit hooks to run tests before commits:
197+
198+
```bash
199+
# Install husky (if needed)
200+
bun add -d husky
201+
202+
# Add pre-commit hook
203+
echo "bun run test" > .husky/pre-commit
204+
```
205+
206+
## Reporting Test Issues
207+
208+
If you encounter test failures:
209+
210+
1. **Run locally** - Verify it fails consistently
211+
2. **Check CI logs** - See platform-specific errors
212+
3. **Reproduce** - Document steps to reproduce
213+
4. **Fix or report** - Either fix or create an issue
214+
215+
## Related Documentation
216+
217+
- [Performance Monitoring](PERFORMANCE.md) - Performance testing and benchmarks
218+
- [Architecture](../README.md#technical-implementation) - Code structure
219+
- [Contributing](../README.md#contributing) - Contribution guidelines
220+

0 commit comments

Comments
 (0)