Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .github/codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,19 @@ coverage:
precision: 2
round: down
range: "50...100"
patch: no
changes: no
# Under status, which is where Codecov reads them. Written directly under coverage, as they
# were from 2021, they made the file invalid, and for an invalid file Codecov "keep[s] using
# the last valid YAML provided" (https://docs.codecov.com/docs/codecov-yaml). This one had
# never been valid, so none of it applied, the flags below included. Check an edit with
# curl -s -X POST --data-binary @.github/codecov.yml https://api.codecov.io/validate
status:
patch: no
changes: no

# Flags are declared so that each one is scoped to the code it actually measures and is
# carried forward when its leg does not run. Both matter here: every flag is uploaded from
# exactly one operating system of its matrix (csharp from windows, fsharp and interactive
# from ubuntu), so on any run where that leg is skipped an undeclared flag reports as
# exactly one operating system of its matrix (csharp from windows, fsharp, interactive and
# cpp from ubuntu), so on any run where that leg is skipped an undeclared flag reports as
# missing rather than unchanged, and the report reads as a coverage drop that did not
# happen. https://github.com/asc-community/AngouriMath/issues/397
flags:
Expand All @@ -28,6 +34,10 @@ flags:
- Sources/Wrappers/AngouriMath.Interactive/
- Sources/Terminal/
carryforward: true
cpp:
paths:
- Sources/Wrappers/AngouriMath.CPP.Importing/
carryforward: true

parsers:
gcov:
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/CPPTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ jobs:

runs-on: ${{ matrix.os }}

# id-token is what the codecov step signs in with. Naming one permission sets every other to
# none, so contents: read is there for the checkout.
permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v6

Expand Down Expand Up @@ -130,6 +136,12 @@ jobs:
# reports -- the whole of googletest and of /usr/include/c++/13 -- so the flag would have
# carried everything the filter above exists to exclude. `disable_search` and `plugins: noop`
# keep it to the one report gcovr wrote.
# Codecov takes an upload from a branch of this repository only with a token, and without one
# it rejected them while this step stayed green: master's newest report there is from
# 2022-11-28. use_oidc gets a token for the run; on a fork's pull request the action skips it,
# and Codecov takes that upload without one. A rejected upload fails the run on a push to
# master only, so that an outage at Codecov does not fail a pull request. See
# Sources/AngouriMath/Docs/Contributing/Coverage.md.
- name: 'Send to codecov'
if: ${{ matrix.os == 'ubuntu-latest' }}
uses: codecov/codecov-action@v5
Expand All @@ -138,6 +150,8 @@ jobs:
disable_search: true
plugins: noop
flags: cpp
use_oidc: true
fail_ci_if_error: ${{ github.event_name == 'push' }}

- name: 'Upload logs'
if: failure()
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/CSharpTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ jobs:

runs-on: ${{ matrix.os }}

# id-token is what the codecov step signs in with. Naming one permission sets every other to
# none, so contents: read is there for the checkout.
permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v6
with:
Expand Down Expand Up @@ -95,9 +101,17 @@ jobs:
# `files`, not `file`. v5 of the action does not take `file`: it warns about an unexpected
# input, ignores the path, and searches the workspace instead, so the named report was never
# what made this flag.
# Codecov takes an upload from a branch of this repository only with a token, and without one
# it rejected them while this step stayed green: master's newest report there is from
# 2022-11-28. use_oidc gets a token for the run; on a fork's pull request the action skips it,
# and Codecov takes that upload without one. A rejected upload fails the run on a push to
# master only, so that an outage at Codecov does not fail a pull request. See
# Sources/AngouriMath/Docs/Contributing/Coverage.md.
- name: 'Send to codecov'
if: ${{ matrix.os == 'windows-latest' }}
uses: codecov/codecov-action@v5
with:
files: ./Sources/Tests/UnitTests/coverage.opencover.xml
flags: csharp
use_oidc: true
fail_ci_if_error: ${{ github.event_name == 'push' }}
14 changes: 14 additions & 0 deletions .github/workflows/FSharpTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ jobs:

runs-on: ${{ matrix.os }}

# id-token is what the codecov step signs in with. Naming one permission sets every other to
# none, so contents: read is there for the checkout.
permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v6
with:
Expand Down Expand Up @@ -43,9 +49,17 @@ jobs:
run: |
dotnet test "Sources/Tests/FSharpWrapperUnitTests" -c Release -l GitHubActions --collect:"XPlat Code Coverage" --results-directory ./coverage -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover

# Codecov takes an upload from a branch of this repository only with a token, and without one
# it rejected them while this step stayed green: master's newest report there is from
# 2022-11-28. use_oidc gets a token for the run; on a fork's pull request the action skips it,
# and Codecov takes that upload without one. A rejected upload fails the run on a push to
# master only, so that an outage at Codecov does not fail a pull request. See
# Sources/AngouriMath/Docs/Contributing/Coverage.md.
- name: 'Send to codecov'
if: ${{ matrix.os == 'ubuntu-latest' }}
uses: codecov/codecov-action@v5
with:
directory: ./coverage
flags: fsharp
use_oidc: true
fail_ci_if_error: ${{ github.event_name == 'push' }}
14 changes: 14 additions & 0 deletions .github/workflows/InteractiveTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ jobs:

runs-on: ${{ matrix.os }}

# id-token is what the codecov step signs in with. Naming one permission sets every other to
# none, so contents: read is there for the checkout.
permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v6
with:
Expand Down Expand Up @@ -58,9 +64,17 @@ jobs:
dotnet test "Sources/Tests/TerminalUnitTests" -c Release --collect:"XPlat Code Coverage" --results-directory ./coverage -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
dotnet test "Sources/Tests/McpUnitTests" -c Release --collect:"XPlat Code Coverage" --results-directory ./coverage -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover

# Codecov takes an upload from a branch of this repository only with a token, and without one
# it rejected them while this step stayed green: master's newest report there is from
# 2022-11-28. use_oidc gets a token for the run; on a fork's pull request the action skips it,
# and Codecov takes that upload without one. A rejected upload fails the run on a push to
# master only, so that an outage at Codecov does not fail a pull request. See
# Sources/AngouriMath/Docs/Contributing/Coverage.md.
- name: 'Send to codecov'
if: ${{ matrix.os == 'ubuntu-latest' }}
uses: codecov/codecov-action@v5
with:
directory: ./coverage
flags: interactive
use_oidc: true
fail_ci_if_error: ${{ github.event_name == 'push' }}
33 changes: 33 additions & 0 deletions Sources/AngouriMath/Docs/Contributing/Coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,36 @@ The filter is the honest part. Without it the report is mostly vendored googlete

To reproduce it locally, publish the exporting project as `CPPTest.yml` does, then configure the
tests with the two cmake flags above and run `gcovr` from the build directory.

## How a report reaches Codecov

Codecov takes an upload without a token only from a fork's branch. From a branch of this
repository, `master` included, it answers `Token required because branch is protected`, and
`codecov-action` logs that as an error and finishes green unless `fail_ci_if_error` is set. No
leg had a token or set it, so the four legs collected coverage and Codecov kept none of it: the
newest `master` commit it holds is from 2022-11-28, and the newest pull request upload came from
a fork on 2026-08-04.

Each upload step now sets `use_oidc: true`, in a job granted `id-token: write`. The job asks
GitHub for a token that lasts for the run, so no secret is stored anywhere. On a fork's pull
request the action skips it, since GitHub gives a fork no id-token, and Codecov takes that upload
without one. `fail_ci_if_error` is set on pushes to `master` only, so a rejected upload turns
`master` red, and an outage at Codecov does not fail a pull request.

`.github/codecov.yml` itself had not been valid since it was added: `patch` and `changes` sat
directly under `coverage` instead of under `coverage.status`. Codecov keeps using the last valid
version of an invalid file, and there was none, so the flags it declares had never applied.
Check an edit to it with

```
curl -s -X POST --data-binary @.github/codecov.yml https://api.codecov.io/validate
```

which answers `Valid!` or names the field it rejects.

To check a run, read the upload step's log. `Upload queued for processing complete` is followed
by `Upload queued for processing failed: 鈥 when Codecov refused it. To see what Codecov holds:

```
curl -s 'https://api.codecov.io/api/v2/github/asc-community/repos/AngouriMath/commits/?branch=master'
```
Loading