[slee430]

Summary

The grade 3 and 4 sentence-structure rubrics reference avg_sentence_length, but the classifier receives avg_words_per_sentence. The notebook and Python SDK also truncate fractional features to integers before classification; the TypeScript SDK rounds them to whole numbers. For example, four subordinate clauses across ten sentences becomes 0 instead of 0.4 per sentence. Details are described in this doc.

This PR:

• Renames all eight outdated rubric references without changing thresholds.
• Preserves one decimal place in the notebook and both SDK classification paths.
• Adds a keyword-only cast_to_int option to the Python serializer, keeping its existing default for compatibility and explicitly disabling integer conversion in this evaluator.
• Updates rubric hashes and regenerates the Python contract bundle.
• Adds regression tests for fractional features reaching the classifier and the grade-specific rubric keys.

The precision change applies to all serialized numeric features, including percentages and sentence-length averages. It retains one-decimal rounding; Python/TypeScript tie-breaking differences remain outside this change.

Validation

• Repository check harness: all 6 checks passed.
• Python make verify: passed, including 1,705 tests; 107 skipped.
• TypeScript lint and typecheck: passed; 1,621 unit tests passed.
• Existing TypeScript sentence-structure live integration suite: 5 tests passed under its retry/acceptable-label rules.
• Controlled comparison of the 5 unique repository fixture/integration cases: no label changes across the four configurations; 4/5 exact matches and 5/5 within the documented acceptable labels.

Classification impact — review before release

A separate 402-row grade 3–4 evaluation used human Sentence Score labels, gpt-4o-2024-08-06, temperature 0, and shared stage-one counts across configurations. The baseline reproduces the original Python SDK behavior, not historical saved predictions.

Configuration	Correct	Accuracy
Original baseline	232/402	57.71%
Rubric rename only	236/402	58.71%
Preserve decimals only	228/402	56.72%
Both fixes	230/402	57.21%

With both fixes, 37 labels changed: 14 became correct, 16 became incorrect, and 7 remained incorrect. Grade 3 gained 5 correct cases; grade 4 lost 7. The overall paired accuracy difference was −0.50 percentage points (95% passage-clustered bootstrap interval: −3.21 to +2.22). Two additional baseline/fixed repeats on the changed cases and 40 unchanged controls found 9 of the initial regressions persisted across all three runs.

These are confirmed input-contract/precision defects, but the combined change is not established as performance-neutral. Please review whether the rubric rename and precision change should ship together or separately, particularly given the grade 4 regressions. Additional SDK changes and tests on this PR branch are welcome.

All relevant data and codes are in this google drive folder.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[adnanrhussain]

@slee430 - After reviewing the code, I think there are more inconsistencies between the SDKs. For example, I think the JSON.stringify to convert the features and send to the model, would not send a 10.0 and instead send a 10. This would be different behavior than the Python impl.

Since most of the code already truncates today, perhaps we could try the alternative path of -

• flooring the raw values to whole numbers
• rubric thresholds to use only < and >=, with ranges written as "≥ a and < b"
• the floor has to apply to the unrounded ratio

This is a subtle change, meeting the original intent of the rubric, but changing its current behavior, so will require re-running the tests you validated for this PR, but now with this change.

[adnanrhussain]

@slee430 - To confirm, the code was already sending avg_words_per_sentence, but the rubric was referencing avg_sentence_length. The model was inferring on this own, and silently resolving this bug?

[slee430]

@adnanrhussain Yes, the code was already sending avg_words_per_sentence, but the grade 3 and 4 rubrics referenced avg_sentence_length. The model may have inferred the connection, but we cannot confirm that it did so consistently. This fix makes the reference explicit.

[slee430]

Regarding your alternative approach, I’ll keep the current implementation for comparison and test your proposal in a separate branch across Python, TypeScript, and the notebook. I’ll keep the feature-name fix in both versions and make the rubric boundary changes explicit.
Then I’ll rerun the tests and compare the baseline, the current PR, and your proposal using the same extracted features. I’ll check the grade 4 results closely, since accuracy decreased there in the previous comparison. We can review the results before deciding which approach to use in this PR.