[thomaspinder]

Summary

get_data built its frame with pl.DataFrame(rows), and polars infers column types from the first 100 rows only. Long CBS tables hit this in two ways:

column is null for 100 rows, then 100     -> ComputeError (86103NED, 00372)
column is 1 for 100 rows, then 1.5        -> Int64, value silently becomes 1

The fix scans every row before choosing a type:

-  return pl.DataFrame(renamed_rows)
+  return pl.DataFrame(renamed_rows, infer_schema_length=None)

Evidence

• Before: Client().get_data("86103NED") and get_data("00372") raise ComputeError: could not append value: 100 of type: i64 to the builder.
  After: they return (197, 35) and (773, 39) frames in under half a second each.
• New tests in TestGetData:
  • test_column_null_for_first_hundred_rows: fails on main with the ComputeError and passes here.
  • test_float_after_hundred_ints_is_not_truncated: fails on main with assert 1 == 1.5 and passes here.
• All 55 tests pass. ruff check, ruff format --check and ty check are clean.

Merge Danger

Door: two-way

Blast Radius: dtypes

Columns that mix integers and floats beyond row 100 now come back as Float64 rather than truncated Int64. That is a change in output for any caller who got integers before, but those integers were wrong. Type inference now reads every row, which costs little next to the HTTP fetch.

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!