fix: prevent heap overflow in Q decompression

[belowzeroff]

Problem

q_decompress() (q.c) sizes the output buffer from the frame's declared
uncompressed size, but never checks that a back-reference's expansion — 2
literal bytes plus up to 255 copied bytes — stays within it. A malformed or
hostile frame can declare a tiny out_size yet emit opcodes that write past the
allocation: a heap buffer overflow.

This is remotely reachable on the server. q_read_body() in q_server.c
passes the client-controlled compressed header flag straight into
q_decode(), so a peer can select the decompression path with a crafted body.
The rayforce -q server also performs no authentication, so the path is
pre-auth.

Fix

Read the back-reference index and length first, then reject the frame when
s + 2 + n would exceed out_size, instead of writing past result. The
source reads are already in bounds (r < s <= out_size), so only the output
write needed guarding. Valid frames are unaffected — the write sequence is
unchanged, only an out-of-bounds expansion is now refused.

Validation

Reproduced with an AddressSanitizer harness driving q_decompress directly
(the function has no rayforce dependencies). A frame declaring out_size = 2
with a back-reference of n = 255:

• before: AddressSanitizer: heap-buffer-overflow ... in q_decompress
• after: the frame is rejected (rc = -1), no out-of-bounds access.

The full server test suite (make test, server legs) still passes 32/32.

🤖 Generated with Claude Code