Skip to content

Commit cbd83aa

Browse files
afarberafarber
committed
Reduce required test coverage for now
1 parent 615e5f4 commit cbd83aa

8 files changed

Lines changed: 191 additions & 82 deletions

File tree

‎.github/workflows/_instrumentation.yml‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Instrumentation Tests
2+
3+
on:
4+
workflow_call:
5+
6+
permissions:
7+
contents: read
8+
9+
jobs:
10+
instrumented-test:
11+
name: Run Instrumentation Tests
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
include:
16+
- device: phone
17+
profile: Nexus 6
18+
- device: automotive
19+
profile: automotive_1024p_landscape
20+
21+
steps:
22+
- name: Checkout repository
23+
uses: actions/checkout@v4
24+
25+
- name: Set up JDK 17
26+
uses: actions/setup-java@v4
27+
with:
28+
distribution: temurin
29+
java-version: 17
30+
31+
- name: Cache Gradle
32+
uses: actions/cache@v4
33+
with:
34+
path: |
35+
~/.gradle/caches
36+
~/.gradle/wrapper
37+
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
38+
restore-keys: |
39+
${{ runner.os }}-gradle-
40+
41+
- name: Grant execute permission for Gradle wrapper
42+
run: chmod +x gradlew
43+
44+
- name: Enable KVM group permissions
45+
run: |
46+
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
47+
sudo udevadm control --reload-rules
48+
sudo udevadm trigger --name-match=kvm
49+
50+
- name: Run instrumentation tests on ${{ matrix.device }}
51+
uses: reactivecircus/android-emulator-runner@v2
52+
with:
53+
api-level: 29
54+
target: default
55+
arch: x86_64
56+
profile: ${{ matrix.profile }}
57+
script: ./gradlew :openmapview:connectedDebugAndroidTest --continue
58+
59+
- name: Upload test results
60+
if: always()
61+
uses: actions/upload-artifact@v4
62+
with:
63+
name: instrumentation-test-results-${{ matrix.device }}
64+
path: '**/build/outputs/androidTest-results/connected/**/*.xml'
65+
retention-days: 30
66+
67+
- name: Upload test reports
68+
if: always()
69+
uses: actions/upload-artifact@v4
70+
with:
71+
name: instrumentation-test-reports-${{ matrix.device }}
72+
path: '**/build/reports/androidTests/connected/**'
73+
retention-days: 30

‎.github/workflows/ci.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,8 @@ jobs:
3838
needs: [format, copyright, test, coverage]
3939
uses: ./.github/workflows/_build-examples.yml
4040

41+
instrumented-test:
42+
name: Instrumentation Tests (PR only)
43+
if: github.event_name == 'pull_request'
44+
uses: ./.github/workflows/_instrumentation.yml
45+

‎.github/workflows/daily.yml‎

Lines changed: 1 addition & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -12,65 +12,4 @@ permissions:
1212
jobs:
1313
instrumented-test:
1414
name: Run Instrumentation Tests
15-
runs-on: ubuntu-latest
16-
strategy:
17-
matrix:
18-
include:
19-
- device: phone
20-
profile: Nexus 6
21-
- device: automotive
22-
profile: automotive_1024p_landscape
23-
24-
steps:
25-
- name: Checkout repository
26-
uses: actions/checkout@v4
27-
28-
- name: Set up JDK 17
29-
uses: actions/setup-java@v4
30-
with:
31-
distribution: temurin
32-
java-version: 17
33-
34-
- name: Cache Gradle
35-
uses: actions/cache@v4
36-
with:
37-
path: |
38-
~/.gradle/caches
39-
~/.gradle/wrapper
40-
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
41-
restore-keys: |
42-
${{ runner.os }}-gradle-
43-
44-
- name: Grant execute permission for Gradle wrapper
45-
run: chmod +x gradlew
46-
47-
- name: Enable KVM group permissions
48-
run: |
49-
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
50-
sudo udevadm control --reload-rules
51-
sudo udevadm trigger --name-match=kvm
52-
53-
- name: Run instrumentation tests on ${{ matrix.device }}
54-
uses: reactivecircus/android-emulator-runner@v2
55-
with:
56-
api-level: 29
57-
target: default
58-
arch: x86_64
59-
profile: ${{ matrix.profile }}
60-
script: ./gradlew :openmapview:connectedDebugAndroidTest --continue
61-
62-
- name: Upload test results
63-
if: always()
64-
uses: actions/upload-artifact@v4
65-
with:
66-
name: instrumentation-test-results-${{ matrix.device }}
67-
path: '**/build/outputs/androidTest-results/connected/**/*.xml'
68-
retention-days: 30
69-
70-
- name: Upload test reports
71-
if: always()
72-
uses: actions/upload-artifact@v4
73-
with:
74-
name: instrumentation-test-reports-${{ matrix.device }}
75-
path: '**/build/reports/androidTests/connected/**'
76-
retention-days: 30
15+
uses: ./.github/workflows/_instrumentation.yml

‎docs/CONTRIBUTING.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ Run unit tests:
102102

103103
### Test Coverage
104104

105-
The project enforces a minimum test coverage of 50% for all code. Coverage is measured using JaCoCo.
105+
The project enforces a minimum test coverage of 20% for all code. Coverage is measured using JaCoCo.
106106

107107
Generate coverage report:
108108

@@ -121,7 +121,7 @@ Check coverage meets minimum threshold:
121121
./scripts/check-coverage.sh
122122
```
123123

124-
The CI pipeline will fail if coverage drops below 50%. Focus on testing:
124+
The CI pipeline will fail if coverage drops below 20%. Focus on testing:
125125
- Core business logic
126126
- Public API methods
127127
- Edge cases and error handling
@@ -159,7 +159,7 @@ The CI pipeline runs automatically on all pull requests and includes:
159159
1. **Format Check** - Verifies Spotless formatting
160160
2. **Copyright Check** - Verifies MIT license headers
161161
3. **Unit Tests** - Runs all JVM unit tests
162-
4. **Test Coverage** - Ensures minimum 50% code coverage
162+
4. **Test Coverage** - Ensures minimum 20% code coverage
163163
5. **Build Library** - Builds the OpenMapView AAR
164164
6. **Build Examples** - Builds all example applications
165165

‎docs/GITHUB_WORKFLOWS.md‎

Lines changed: 104 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,25 @@ The workflows are designed using a **modular, reusable component architecture**
2222

2323
Located in `.github/workflows/`:
2424

25+
#### `_copyright.yml`
26+
- **Purpose**: Verify MIT license headers in all `.kt` files
27+
- **Runs**: `./scripts/check-copyright.sh`
28+
- **Usage**: Called by CI and Release workflows
29+
- **Fast**: ~30 seconds
30+
2531
#### `_format.yml`
2632
- **Purpose**: Verify code formatting with Spotless
2733
- **Runs**: `./gradlew spotlessCheck`
2834
- **Usage**: Called by CI and Release workflows
2935
- **Fast**: ~30 seconds
3036

37+
#### `_coverage.yml`
38+
- **Purpose**: Check test coverage meets minimum threshold
39+
- **Runs**: `./scripts/check-coverage.sh` (minimum 20% coverage)
40+
- **Artifacts**: Uploads coverage reports to Codecov and as artifacts
41+
- **Usage**: Called by CI workflow
42+
- **Duration**: ~1-2 minutes
43+
3144
#### `_test.yml`
3245
- **Purpose**: Run unit tests
3346
- **Runs**: `./gradlew :openmapview:test --continue`
@@ -49,6 +62,13 @@ Located in `.github/workflows/`:
4962
- **Usage**: Called by CI and Release workflows
5063
- **Duration**: ~3-4 minutes
5164

65+
#### `_instrumentation.yml`
66+
- **Purpose**: Run instrumentation tests on Android emulators
67+
- **Runs**: Tests on both phone (Nexus 6) and automotive (automotive_1024p_landscape) profiles
68+
- **Artifacts**: Uploads test results and reports for both device types
69+
- **Usage**: Called by daily.yml and ci.yml (PR only)
70+
- **Duration**: ~10-15 minutes (may vary due to emulator startup and potential flakiness)
71+
5272
### Main Workflows
5373

5474
#### `ci.yml` - Continuous Integration
@@ -61,28 +81,47 @@ Located in `.github/workflows/`:
6181
format (Check formatting)
6282
|
6383
v
64-
test (Run unit tests)
84+
copyright (Check headers)
6585
|
6686
v
6787
+----------------+----------------+
6888
| |
6989
v v
90+
test (Unit tests) coverage (Test coverage)
91+
| |
92+
+----------------+----------------+
93+
|
94+
+----------------+----------------+
95+
| |
96+
v v
7097
build-library build-examples
7198
(Build AAR) (Build 3 APKs)
99+
100+
For Pull Requests only:
101+
v
102+
instrumented-test (Phone + Automotive)
103+
(Runs in parallel, not blocking)
72104
```
73105

74106
**Jobs**:
75107
1. **format** - Runs Spotless formatting check
76-
2. **test** - Runs unit tests (depends on format)
77-
3. **build-library** - Builds library AAR (depends on format + test, runs in parallel with build-examples)
78-
4. **build-examples** - Builds example APKs (depends on format + test, runs in parallel with build-library)
108+
2. **copyright** - Verifies MIT license headers
109+
3. **test** - Runs unit tests (depends on format + copyright)
110+
4. **coverage** - Checks test coverage meets 20% minimum (depends on format + copyright)
111+
5. **build-library** - Builds library AAR (depends on format + copyright + test + coverage, runs in parallel with build-examples)
112+
6. **build-examples** - Builds example APKs (depends on format + copyright + test + coverage, runs in parallel with build-library)
113+
7. **instrumented-test** - Runs instrumentation tests on phone and automotive emulators (PR only, runs independently)
79114

80-
**Total Duration**: ~3-4 minutes (with parallel builds)
115+
**Total Duration**:
116+
- **Maintainer pushes to main**: ~3-4 minutes (no instrumentation tests)
117+
- **External PRs**: ~10-15 minutes (includes instrumentation tests on phone + automotive)
81118

82119
**Benefits**:
83120
- Catches formatting issues early (before expensive builds)
84121
- Parallel builds save time
85122
- PR checks ensure code quality before merge
123+
- Instrumentation tests only run on PRs, reducing CI time for maintainer's direct commits
124+
- External contributors' PRs are thoroughly tested on both phone and automotive platforms
86125

87126
#### `release.yml` - Maven Central Release
88127
**Trigger**: Push of version tag matching `v*.*.*` pattern (e.g., `v0.2.0`)
@@ -131,6 +170,38 @@ build-library build-examples
131170

132171
**Secrets Required**: See "Required GitHub Secrets" section below
133172

173+
#### `daily.yml` - Daily Instrumentation Tests
174+
**Trigger**: Scheduled daily at 09:45 UTC, or manual trigger via workflow_dispatch
175+
176+
**Purpose**: Run comprehensive instrumentation tests on both phone and automotive platforms daily to catch potential issues early
177+
178+
**Execution Flow**:
179+
```
180+
Scheduled trigger (09:45 UTC daily)
181+
|
182+
v
183+
instrumented-test (calls _instrumentation.yml)
184+
|
185+
v
186+
+------------------+------------------+
187+
| |
188+
v v
189+
Phone Tests Automotive Tests
190+
(Nexus 6 profile) (automotive_1024p_landscape)
191+
```
192+
193+
**Jobs**:
194+
1. **instrumented-test** - Calls the reusable `_instrumentation.yml` workflow
195+
196+
**Total Duration**: ~10-15 minutes (may vary due to emulator startup and flakiness)
197+
198+
**Benefits**:
199+
- Catches device-specific regressions early
200+
- Tests on both phone and automotive form factors
201+
- Runs independently from main CI pipeline
202+
- Manual trigger available for on-demand testing
203+
- Acceptable flakiness since it doesn't block development workflow
204+
134205
## Required GitHub Secrets
135206

136207
For the release workflow to function, the project owner must configure these secrets in the GitHub repository:
@@ -266,19 +337,34 @@ GitHub triggers ci.yml workflow
266337
format job: Check Spotless formatting (30 sec) - PASS
267338
|
268339
v
269-
test job: Run unit tests (1 min) - PASS
340+
copyright job: Check license headers (30 sec) - PASS
270341
|
271342
v
272343
+-------------+--------------+
273344
| |
274345
v v
346+
test job: Unit tests coverage job: Test coverage
347+
(1 min) - PASS (1 min) - PASS (>20%)
348+
| |
349+
+-------------+--------------+
350+
|
351+
+-------------+--------------+
352+
| |
353+
v v
275354
build-library: Build AAR build-examples: Build 3 APKs
276355
(2 min) - PASS (3 min) - PASS
277356
| |
278357
+-------------+--------------+
279358
|
280359
v
281-
All checks pass -> PR is ready to merge
360+
All core checks pass (3-4 min)
361+
362+
Meanwhile (in parallel, PR only):
363+
instrumented-test: Phone + Automotive tests
364+
(10-15 min) - PASS (may be flaky)
365+
|
366+
v
367+
All checks complete -> PR is ready to merge
282368
```
283369

284370
### Example: Release to Maven Central
@@ -331,15 +417,22 @@ To upgrade Java version or change Gradle cache settings:
331417
- Automatically applies to CI and Release workflows
332418

333419
### 4. Composability
334-
Easy to add new workflows:
420+
The reusable `_instrumentation.yml` workflow demonstrates this principle:
335421
```yaml
336-
# Future: instrumentation-tests.yml
422+
# Used by both daily.yml and ci.yml
337423
jobs:
338424
instrumentation-tests:
339-
uses: ./.github/workflows/_instrumentation-tests.yml
425+
uses: ./.github/workflows/_instrumentation.yml
340426
```
427+
This single workflow definition powers both daily scheduled tests and PR validation tests.
428+
429+
### 5. Flexible Testing Strategy
430+
- **Maintainer commits to main**: Fast feedback (~3-4 min) without instrumentation delays
431+
- **External PRs**: Comprehensive testing (~10-15 min) including phone and automotive instrumentation tests
432+
- **Daily scheduled runs**: Proactive regression detection on both platforms
433+
- **Test flakiness handled gracefully**: Instrumentation tests run independently and don't block merges
341434
342-
### 5. Clear Separation of Concerns
435+
### 6. Clear Separation of Concerns
343436
- Format checks catch simple issues fast
344437
- Tests verify correctness
345438
- Builds ensure compilation works

‎docs/TESTING_UNIT.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -373,7 +373,7 @@ Check Robolectric version compatibility with CI's JDK version. The project uses:
373373

374374
### Overview
375375

376-
OpenMapView enforces a minimum test coverage of 50% using JaCoCo. Coverage is measured automatically on every CI run and displayed via a badge in the README.
376+
OpenMapView enforces a minimum test coverage of 20% using JaCoCo. Coverage is measured automatically on every CI run and displayed via a badge in the README.
377377

378378
[![codecov](https://codecov.io/gh/afarber/OpenMapView/branch/main/graph/badge.svg)](https://codecov.io/gh/afarber/OpenMapView)
379379

@@ -405,7 +405,7 @@ JaCoCo is configured in `openmapview/build.gradle.kts` with the following exclus
405405
- Test files
406406
- Android framework classes
407407

408-
The minimum coverage threshold is set to 50% and enforced in CI.
408+
The minimum coverage threshold is set to 20% and enforced in CI.
409409

410410
### Current Coverage Areas
411411

@@ -429,7 +429,7 @@ The `.github/workflows/_coverage.yml` workflow:
429429
1. Runs all unit tests
430430
2. Generates JaCoCo coverage report
431431
3. Uploads coverage to Codecov
432-
4. Checks coverage meets 50% minimum
432+
4. Checks coverage meets 20% minimum
433433
5. Uploads HTML and XML reports as artifacts
434434

435435
Coverage results are available:

0 commit comments

Comments
 (0)