Fix: decode values()/values_list() columns to their field's Python type#13
Merged
Conversation
values()/values_list() returned raw driver values without applying the per-field read decoder that instance hydration uses. On SQLite (which stores NUMERIC as text) a DecimalField came back as str from values_list() but as Decimal from obj.field, so arithmetic on a projected decimal raised "TypeError: can't multiply sequence by non-int of type 'str'". Backends that don't express a type natively (MySQL CHAR(36) uuids, naive datetimes) were similarly under-converted. _fetch_columns now resolves each projected path to its terminal Field (new _terminal_field helper, mirroring _resolve_column's traversal — local columns, pk, forward-relation chains, and a single reverse-FK/M2M hop) and applies that field's dialect read decoder, so a values()/ values_list() value has the same type as the attribute on a hydrated instance. The decoder is idempotent where the driver already returns the native type (e.g. Postgres Decimal), so other backends are unaffected. Adds regression tests covering local and relation-traversed decimals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the code-review findings on the values()/values_list() decode change: - The decoder was applied only in _fetch_columns, so annotate()/group_by() + values() (the _values_grouped path) still returned raw driver values — the exact TypeError the fix set out to remove was reproducible whenever an annotation or group_by was present. _grouped_select_sql now yields the terminal field per output column (None for aggregates) and _values_grouped decodes field columns through the same read decoder. - The separate _terminal_field resolver mirrored _resolve_column's traversal but returned None for a trailing `pk` alias and for bare relation/M2M names, so those projections skipped decoding and diverged from the equivalent relation projection (visible as a raw-str uuid pk on MySQL/Oracle/SQL Server). Folded it into a single _resolve_column_field that returns (column, field) in one traversal; _resolve_column is now a thin wrapper and _terminal_field is gone, so the SELECT and its decoder can never drift. Shared _column_decoders/_decode_rows helpers back both projection paths. Adds regression tests for annotate()/group_by() + values() decode, relation-pk decode consistency (uuid pk), and a NULL nullable decoded column.
vsdudakov
added a commit
that referenced
this pull request
Jul 8, 2026
…pe (#13) * Fix: decode values()/values_list() columns to their field's Python type values()/values_list() returned raw driver values without applying the per-field read decoder that instance hydration uses. On SQLite (which stores NUMERIC as text) a DecimalField came back as str from values_list() but as Decimal from obj.field, so arithmetic on a projected decimal raised "TypeError: can't multiply sequence by non-int of type 'str'". Backends that don't express a type natively (MySQL CHAR(36) uuids, naive datetimes) were similarly under-converted. _fetch_columns now resolves each projected path to its terminal Field (new _terminal_field helper, mirroring _resolve_column's traversal — local columns, pk, forward-relation chains, and a single reverse-FK/M2M hop) and applies that field's dialect read decoder, so a values()/ values_list() value has the same type as the attribute on a hydrated instance. The decoder is idempotent where the driver already returns the native type (e.g. Postgres Decimal), so other backends are unaffected. Adds regression tests covering local and relation-traversed decimals. * fix(values): decode grouped/annotated projections and unify the resolver Addresses the code-review findings on the values()/values_list() decode change: - The decoder was applied only in _fetch_columns, so annotate()/group_by() + values() (the _values_grouped path) still returned raw driver values — the exact TypeError the fix set out to remove was reproducible whenever an annotation or group_by was present. _grouped_select_sql now yields the terminal field per output column (None for aggregates) and _values_grouped decodes field columns through the same read decoder. - The separate _terminal_field resolver mirrored _resolve_column's traversal but returned None for a trailing `pk` alias and for bare relation/M2M names, so those projections skipped decoding and diverged from the equivalent relation projection (visible as a raw-str uuid pk on MySQL/Oracle/SQL Server). Folded it into a single _resolve_column_field that returns (column, field) in one traversal; _resolve_column is now a thin wrapper and _terminal_field is gone, so the SELECT and its decoder can never drift. Shared _column_decoders/_decode_rows helpers back both projection paths. Adds regression tests for annotate()/group_by() + values() decode, relation-pk decode consistency (uuid pk), and a NULL nullable decoded column.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
values()/values_list()returned whatever the driver handed back, skipping the per-field read decoder that instance hydration applies. Instance hydration (_from_db_row) runs every column through the field's dialect decoder, but_fetch_columns— which backsvalues/values_list— returnedengine.fetch_rows(...)untouched.On SQLite (stores
NUMERICas text) this means aDecimalFieldcomes back asstrfromvalues_listbut asDecimalfromobj.balance:It's a general correctness bug — not decimal-specific. Backends that don't express a type natively (MySQL
CHAR(36)uuids, naive datetimes) were similarly under-converted throughvalues/values_list.Fix
python/yara_orm/queryset.py:_terminal_field(path)— new helper resolving avalues()path to theFieldit selects, mirroring_resolve_column's traversal (local column,pk, a relation name, a forward-relation chain, or a single reverse-FK/M2M hop). ReturnsNonefor anything without a single owning field, so those values pass through untouched._fetch_columnsnow appliesdialect.read_decoder(field)per column — the same decoder instance hydration uses. Computed once per query, applied in place. It's a no-op where the driver already returns the native type (e.g. PostgresDecimal), sinceto_pythonis idempotent, so other backends are unaffected.Result: a
values()/values_list()value now has the same Python type as the attribute on a hydrated instance, including values that traverse a relation (ledger__balance).Tests
tests/test_values_and_traversal.pycovering local and relation-traversed decimals (values(),values_list()tuple +flat=True), parametrized across all backends.ruff check/ruff formatclean.