Skip to content

fix(go): scan nullable type/payload in receive#285

Draft
NikolayS wants to merge 1 commit into
mainfrom
claude/fix-go-nullable-payload-oqxpbr
Draft

fix(go): scan nullable type/payload in receive#285
NikolayS wants to merge 1 commit into
mainfrom
claude/fix-go-nullable-payload-oqxpbr

Conversation

@NikolayS

Copy link
Copy Markdown
Owner

Bug

pgque.message.type and .payload are nullable text (sql/pgque-api/receive.sql), and PgQ legally stores NULL ev_type/ev_data (direct pgque.insert_event calls, trigger-based producers). The Go client declared Message.Type and Message.Payload as non-nullable string and scanned the columns directly into them in both Receive and ReceiveCoop.

Scanning a NULL into Go string errors (cannot scan NULL into *string), which fails the WHOLE batch: Receive returns an error, the batch is never acked, and next_batch returns the same batch on every poll — a poison-message livelock from a single NULL event.

Fix

Both scan loops now go through a shared scanMessage helper that scans type/payload into nullable *string temporaries and maps NULL to "".

Approach rationale (over switching the fields to *string like Extra1..4):

  • The public API is firmly built around plain string: consumer handler dispatch is keyed by msg.Type (map lookup, string concat in nack reasons), and the documented payload pattern is json.Unmarshal([]byte(msg.Payload), ...). Pointer fields would break every caller for a rare edge case.
  • A NULL type maps to "", which has no registered handler, so the Consumer routes it through the existing unknown-handler policy path (Nack by default, or skip with AckUnknown) — no crash, no consumer.go changes needed.
  • Server-side pgque.nack re-queries the canonical event row by msg_id from the active batch, so the client-side NULL -> "" mapping cannot corrupt retry/DLQ data.

The mapping is documented on the Message struct and on scanMessage.

Verification

TDD: the two new integration tests (clients/go/null_event_test.go) were written first and confirmed red on unfixed code, failing exactly on the bug:

$ PGQUE_TEST_DSN="postgresql:///pgque_gonull?host=/var/run/postgresql" go test -run 'NullTypeAndPayload' -v .
--- FAIL: TestReceive_NullTypeAndPayload
    receive with NULL type/payload: pgque: scan message: can't scan into dest[2] (col: type): cannot scan NULL into *string
--- FAIL: TestReceiveCoop_NullTypeAndPayload
    receive_coop with NULL type/payload: pgque: scan message: can't scan into dest[2] (col: type): cannot scan NULL into *string

The tests install an event with NULL type/data via select pgque.insert_event(queue, null, null), tick, then assert Receive/ReceiveCoop succeed, return the message with Type == "" and Payload == "", and the batch is ackable.

After the fix (scratch DB pgque_gonull, fresh install of sql/pgque.sql on local PostgreSQL 16):

$ cd clients/go
$ PGQUE_TEST_DSN="postgresql:///pgque_gonull?host=/var/run/postgresql" go test -run 'NullTypeAndPayload' -v .
--- PASS: TestReceive_NullTypeAndPayload (0.06s)
--- PASS: TestReceiveCoop_NullTypeAndPayload (0.07s)

$ PGQUE_TEST_DSN="postgresql:///pgque_gonull?host=/var/run/postgresql" go test ./...
ok  	github.com/NikolayS/pgque-go	16.089s

$ go vet ./...
(clean)

Addresses finding B4 of #283

https://claude.ai/code/session_01KAaEGkQZmey1D1xCsVGmqv


Generated by Claude Code

pgque.message.type and .payload are nullable text, and PgQ legally
stores NULL ev_type/ev_data (direct pgque.insert_event calls,
trigger-based producers). Receive and ReceiveCoop scanned these
columns straight into Go string, so one NULL event errored the whole
batch; the batch was never acked and next_batch redelivered it
forever -- a poison-message livelock.

Scan through nullable temporaries in a shared scanMessage helper and
map NULL to "". Keeps the Message API (handler dispatch by Type,
json.Unmarshal of Payload) unchanged; a NULL type dispatches as ""
and routes through the unknown-handler policy. Server-side nack
re-queries canonical event data by msg_id, so the mapping cannot
corrupt retry/DLQ rows.

Addresses finding B4 of #283.

https://claude.ai/code/session_01KAaEGkQZmey1D1xCsVGmqv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants