Skip to content

Add RSZ/ONSS employment by daily-wage class and annual wage key figures - #388

Merged
MaxGhenis merged 8 commits into
mainfrom
be/b2f-rsz-wage-classes
Oct 24, 2026
Merged

MaxGhenis merged 8 commits into
mainfrom
be/b2f-rsz-wage-classes

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #387; merge after it. Belgian transport port (ruling d048), source package B2f: a Belgian wage distribution from the social security office (RSZ/ONSS).

What it adds

packages/rsz/employment_daily_wage_class_2023_2026 (alias rsz-employment-daily-wage-class-2023-2026, 34,629 facts), from RSZ "Arbeidsmarktanalyse: gedetailleerde kwartaalgegevens":

  • T17: daily-wage class × age × sex, 2023Q1–2026Q1, 5,616 facts. 2023 comes from the 2023–2024 workbook, 2024 onward from the 2024–2026 workbook.
  • T7: status × sex × NACE 2025 × daily-wage class, 2024Q1–2026Q1, 29,013 facts.
  • Measures:
    • jobs (arbeidsplaatsen): a quarter-end stock, entity person, role employee_job;
    • workers deduplicated by INSZ, carrying their principal job: a quarter-end stock, role employee;
    • FTE (VTE): volume over the whole quarter, role full_time_equivalent_employee.
    • Each measure has its own record set and coverage; table spec ids and record-id prefixes are shared.
  • Wage classes: published as <100, 100 - 149, 150 - 199, 200 - 299, >= 300 and Onbekend-Inconnu, in EUR per day. Numeric endpoints are not encoded as constraints, because their inclusivity is not published.
  • Daily wage (RSZ common criteria page): normal gross pay ÷ paid days for full-time workers, and hourly pay × 7.6 for part-time workers. It excludes double holiday pay, premiums, the 13th month and severance. It ranks pay rates, not annual earnings.

packages/rsz/annual_wages_key_figures_2020_2025 (alias rsz-annual-wages-key-figures-2020-2025, 252 facts):

  • the "Kerncijfers-Chiffres Clés" sheet of "Lonen en bezoldigde prestaties", the October 2026 pre-publication release;
  • worker type × sex × six measures, 2020–2025, in the published units (thousands of EUR, days, thousands of hours), plus its 36 typed total cells.

Licence: CC BY 4.0, from the captured RSZ open-data page. Attribution is in each manifest and README.

Precision, overlap and totals

  • Every FTE and annual cell displays one decimal (#,##0.0); jobs and workers are integers. Values are the cached value rounded half-up at the displayed precision. The cached value and number format stay in each locator.
  • 2024 overlap between the two quarterly workbooks: 1,728 T17 cells agree at two decimals. 285 FTE caches differ by float noise; 15 cross a half-tenth when rounded to one decimal. All are listed in workbook_audit.json, and the newer workbook supplies every 2024 fact.
  • Quarterly totals: every Totaal row is a SUBTOTAL formula and is excluded. Tests check that the cached details sum to the cached subtotals. Rounded facts need not sum to a rounded subtotal.
  • Annual totals: the 36 annual total cells are typed numbers, not formulas, so they are kept as published totals.

Invariants

  • Fidelity: one fact per selected published numeric cell. Each value equals the cell's cached value at its displayed precision.
  • No constructed totals: the facts contain no derived totals.
  • Determinism: both generators reproduce their CSV, YAML and manifests byte for byte.
  • Lineage: coverage is 1.0, with 0 acceptance errors.

Checks

Check Result
validate-package, both packages valid, 0 errors, 0 warnings
build-suite daily --year 2026 valid; 34,629 facts, 132,900 constraints, 0 acceptance errors, lineage 1.0
build-suite annual --year 2025 valid; 252 facts, 684 constraints, 0 acceptance errors, lineage 1.0
Spot check: 40 random daily rows (120 values plus printed dimensions) re-read from the workbooks 0 mismatches
ruff check / ruff format --check pass
R2 ledger-raw, all 15 manifest files (daily CSV now b4f2f134…) uploaded, SHA-256 readback verified 2026-10-24
Focused tests, annual (tests/test_source_rsz_annual_wages_key_figures.py) 7/7 pass when called directly at 7e447cb; the pytest run waits on the host reservation, and CI's Chronicle surface job runs the file
Focused tests, daily (tests/test_source_rsz_employment_daily_wage_class.py) 5/5 pass when called directly at 23c0bd5; the pytest run waits on the host reservation, and CI's Chronicle surface job runs the file
Bundle pins in tests/test_chronicle_bundle.py (099a4c1, on top of #387's) +34,881 person facts at country:BE; two sources, two source tables and two built packages; quarter 2023-Q1 to 2026-Q1 and calendar years 2020 to 2025 incremented; no new geography, period or entity key; no warning. Measured with Chronicle's bundle functions and confirmed against #388's CI bundle on the old base. The full bundle test, replayed against #385's CI bundle plus #386's skipped packages, #387's package and these two, passes; #387's test fails against it. CI runs the full bundle test on this head
Independent review (GPT-6.1 Sol) 9da0e3b: REQUEST CHANGES for one broken annual test (consumer-contract validator given exported dicts), fixed in 7e447cb. Its independent full re-read of all 34,881 facts found 0 value, precision, dimension or locator mismatches, and 0 omissions or extras. It confirmed coverage, definitions, units, overlap and subtotal claims. 7e447cb: APPROVE, no findings. 23c0bd5 merges the T7 sex case variants (Man-homme/Vrouw-femme through 2025Q3) into one value per sex, as for Onbekend-Inconnu, so T7 series no longer split at 2025Q4. Re-review of 23c0bd5 pending

Source-only coverage:

  • 34,881 facts in total;
  • 0 new period keys: every quarter:2023-Q1…2026-Q1 and calendar_year:2020…2025 key already exists;
  • 0 new geography keys: country:BE;
  • no label conflicts.

Stacking and rebase

Rebased from 200bf6d onto #387's head 9323e9e, which is on main ba2509a. git range-diff shows all three commits unchanged. Every file is byte-identical to 23c0bd5 except the alias registry, which differs by exactly this PR's two RSZ aliases.

This PR and #389 are stacked so their CI runs on the trees that will land. They merge as merge commits in order: #387, then #388, then #389. Until #387 merges, this PR's diff on GitHub also shows #387's commits.

Shared code

Only the two SOURCE_PACKAGE_ALIASES registration lines in chronicle/source_package.py.

🤖 Generated with Claude Code

MaxGhenis and others added 8 commits October 24, 2026 04:16
Preserve all 565 commune rates printed in FPS Finance's assessment-year 2026
PDF, with exact page, column-block and text-line locators and a bijective
mapping to Statbel's NIS 2025 communes (558 exact, 7 orthographic).

Key the facts as tax_year 2025, the income year these rates tax, with
income_year == 2025 and assessment_year == 2026 constraints evidenced by
source-row columns and the legal vintage income_year_2025_assessment_year_2026,
as in statbel/fiscal_income_distribution_2023. The printed assessment-year
label is kept verbatim. The CSV and PDF are in R2 ledger-raw, verified by
SHA-256 readback.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address the review of 875dea6:
- Each fact's geography name is now the commune name SPF prints, as the
  harness requires of every publisher (chronicle#266). Nine differ from
  Statbel's (Bruxelles/Brussels, Waimes/Weismes and seven spelling or
  punctuation variants); the mapping audit keeps Statbel's name beside
  each, and the bundle reports them as cross-package name warnings.
- Drop the per-row publisher_commune filter and constraint and its
  dimension label. The NIS code identifies the commune, so a spelling no
  longer changes aggregate or semantic fact keys. The printed label stays
  in the CSV, the source-row id and a guard cell.

The regenerated CSV is in R2 ledger-raw under its new hash, verified by
SHA-256 readback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For all 565 facts, respell the commune wherever the fact carries it
(geography name, filter and constraint values) as Statbel's name and
casefolded, uppercased and accent- and punctuation-stripped forms; both
aggregate and semantic keys must not change. Moving a fact to another
NIS code must change both. The respelling fails on the 875dea6 shape,
where the printed name was a filter and constraint.

Hypothesis is not a Chronicle dependency, and adding one is a shared
dependency change, so the property is checked exhaustively over the
package's facts and a fixed set of respellings instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The package adds 565 government facts at tax_year:2025, one per existing
commune key, so geography_count is unchanged and commune:11002 goes from
9 to 10. It adds one source, one source table and one built package.
SPF spells nine communes unlike Statbel, and the package keeps SPF's
wording, so nine commune geography-name warnings are added (123 -> 132).

The deltas were measured with Chronicle's own bundle functions. A replay
of the full bundle test against #385's CI bundle, with #386's seven
skipped county packages and this package added, passes, and main's
unchanged test fails against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rsz-employment-daily-wage-class-2023-2026 (34,629 facts): RSZ quarterly
labour market analysis, T17 (daily-wage class x age x sex, 2023Q1-2026Q1;
2023 from the 2023-2024 workbook, 2024 onward from the 2024-2026 workbook)
and T7 (status x sex x NACE 2025 x daily-wage class, 2024Q1-2026Q1). Jobs
and deduplicated workers are quarter-end stocks; FTE covers the whole
quarter. Values are recorded at the publisher's displayed precision;
SUBTOTAL formula rows are excluded and tested against the details.

rsz-annual-wages-key-figures-2020-2025 (252 facts): the Kerncijfers sheet
of the October 2026 pre-publication workbook, in the published thousands
of EUR and hours, including its 36 typed total cells.

Both are CC BY 4.0 with captured licence and methodology pages. All raw
files are in R2 ledger-raw, verified by SHA-256 readback.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntract

The per-fact annual test passed consumer_fact_rows() dicts to
validate_consumer_fact_contract(), which takes AggregateFact objects, so it
raised AttributeError before any per-fact assertion ran. Validate the facts
and check the exported rows' lineage and R2 keys separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
T7 prints Man-homme/Vrouw-femme through 2025Q3 and Man-Homme/Vrouw-Femme
from 2025Q4; T17 always prints the capitalized forms. Keeping both as
values split every T7 series at 2025Q4, so a consumer selecting
Man-Homme silently lost 2024-2025Q3. Map each to one value, as the
package already does for Onbekend-Inconnu; the printed form stays in
every source locator, and any other sex label stops the build. Tests
check both printed cases occur and that each sex is a single value.

The regenerated CSV is in R2 ledger-raw under its new hash, verified by
SHA-256 readback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ages

The two packages add 34,881 person facts at country:BE: 34,629 quarterly
daily-wage-class cells from 2023 Q1 to 2026 Q1, and 252 annual key
figures for 2020-2025. They add two sources, two source tables and two
built packages. They add no new geography, period or entity key and no
warning.

This stacks on #387's pins. The deltas were measured with Chronicle's
own bundle functions and confirmed against #388's CI bundle on the old
base. A replay of the full bundle test against #385's CI bundle, with
#386's skipped packages, #387's package and these two packages added,
passes, and #387's test fails against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the be/b2f-rsz-wage-classes branch from 23c0bd5 to 099a4c1 Compare October 24, 2026 08:20
MaxGhenis added a commit that referenced this pull request Oct 24, 2026
The package adds 35 calendar-2025 facts: 19 person, 2 benefit_unit
premium payments and 14 government expenditure. It adds one
statistical_scope geography, the German-speaking Community, plus one
source, one source table and one built package. It adds no warning.

This stacks on #388's pins. The deltas were measured with Chronicle's
own bundle functions and confirmed against #389's CI bundle on the old
base. A replay of the full bundle test against #385's CI bundle, with
#386's skipped packages and #387's, #388's and this package added,
passes, and #388's test fails against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 24, 2026 08:33
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Delta review at 099a4c1 (GPT-6.1 Sol, Subfleet run 20261024-042210-chronicle388-review-r4): APPROVE, no findings. ledger-source-fidelity: PASS. ledger-boundary: PASS.

  • Rebase onto Add SPF Finances municipal PIT surcharge rates, income year 2025 #387's head 9323e9e: git range-diff shows all three commits unchanged. All 30 non-registry files are byte-identical to the approved 23c0bd5, and the registry diff is exactly the two RSZ aliases.
  • Bundle pins (099a4c1): derived independently at bundle year 2023, and every changed pin matches. +34,881 person facts, country:BE +34,881, two sources, two tables and two packages; quarter 2023-Q1 to 2026-Q1 and calendar years 2020 to 2025 incremented as pinned; no change in geography, period or entity cardinality. Zero warnings of any kind: all 19 packages emitting country:BE name it "Belgium", and the RSZ dimension ids and labels conflict with no other package.
  • Earlier reviews: the first review REQUEST CHANGES, fixed in 7e447cb; r2 APPROVE; r3 at 23c0bd5 APPROVE.

This merges as a merge commit after #387, at 099a4c1, once CI passes.

@MaxGhenis
MaxGhenis merged commit de8bccf into main Oct 24, 2026
3 checks passed
@MaxGhenis
MaxGhenis deleted the be/b2f-rsz-wage-classes branch October 24, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant