Recompile with a recorded profile produces a parser that prepends a spurious empty element to repeated string fields

[jfberry]

Summary

A MessageType recompiled via ty.Recompile(profile) — after recording a profile with WithRecordProfile — mis-parses repeated string fields: the resulting list contains a spurious empty string at index 0 followed by the correct element(s). The baseline compiled type parses the same bytes correctly. Parsing succeeds with no error, so the corruption is silent.

Versions / platforms

• buf.build/go/hyperpb v0.1.3
• google.golang.org/protobuf v1.36.9
• Go 1.26.3
• Reproduced on darwin/arm64 and linux/amd64

Reproducer

Fully standalone (programmatic descriptor, no codegen): go mod init repro && go get buf.build/go/hyperpb@v0.1.3 && go run .

// message Fort { string id = 1; repeated string image_url = 2; }
package main

import (
	"fmt"

	"buf.build/go/hyperpb"
	"google.golang.org/protobuf/encoding/protowire"
	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/reflect/protodesc"
	"google.golang.org/protobuf/reflect/protoreflect"
	"google.golang.org/protobuf/types/descriptorpb"
)

func main() {
	fdp := &descriptorpb.FileDescriptorProto{
		Name:    proto.String("repro.proto"),
		Package: proto.String("repro"),
		Syntax:  proto.String("proto3"),
		MessageType: []*descriptorpb.DescriptorProto{{
			Name: proto.String("Fort"),
			Field: []*descriptorpb.FieldDescriptorProto{
				{Name: proto.String("id"), Number: proto.Int32(1),
					Type:  descriptorpb.FieldDescriptorProto_TYPE_STRING.Enum(),
					Label: descriptorpb.FieldDescriptorProto_LABEL_OPTIONAL.Enum()},
				{Name: proto.String("image_url"), Number: proto.Int32(2),
					Type:  descriptorpb.FieldDescriptorProto_TYPE_STRING.Enum(),
					Label: descriptorpb.FieldDescriptorProto_LABEL_REPEATED.Enum()},
			},
		}},
	}
	fd, err := protodesc.NewFile(fdp, nil)
	if err != nil {
		panic(err)
	}
	md := fd.Messages().Get(0)
	imageFd := md.Fields().ByName("image_url")

	// id="FORT_1", image_url=["https://example.com/one.png"]
	var payload []byte
	payload = protowire.AppendTag(payload, 1, protowire.BytesType)
	payload = protowire.AppendString(payload, "FORT_1")
	payload = protowire.AppendTag(payload, 2, protowire.BytesType)
	payload = protowire.AppendString(payload, "https://example.com/one.png")

	ty := hyperpb.CompileMessageDescriptor(md)

	parse := func(ty *hyperpb.MessageType) protoreflect.List {
		shared := new(hyperpb.Shared)
		msg := shared.NewMessage(ty)
		if err := msg.Unmarshal(payload); err != nil {
			panic(err)
		}
		return msg.Get(imageFd).List()
	}

	base := parse(ty)
	fmt.Printf("baseline:   len=%d %q\n", base.Len(), base.Get(0).String())

	profile := ty.NewProfile()
	shared := new(hyperpb.Shared)
	for i := 0; i < 300; i++ {
		msg := shared.NewMessage(ty)
		if err := msg.Unmarshal(payload, hyperpb.WithRecordProfile(profile, 1.0)); err != nil {
			panic(err)
		}
		shared.Free()
	}
	recompiled := ty.Recompile(profile)

	rec := parse(recompiled)
	fmt.Printf("recompiled: len=%d", rec.Len())
	for i := 0; i < rec.Len(); i++ {
		fmt.Printf(" [%d]=%q", i, rec.Get(i).String())
	}
	fmt.Println()
}

Observed

baseline:   len=1 "https://example.com/one.png"
recompiled: len=2 [0]="" [1]="https://example.com/one.png"

Expected

The recompiled type parses identically to the baseline type: len=1, no empty element.

Notes

• The spurious element is always an empty string at index 0, suggesting the profile-specialized repeated-string path appends a zero-value element before parsing the real one.
• Only the recompiled type is affected; the baseline CompileMessageDescriptor output is correct on the same bytes across everything we've thrown at it.
• We found this in production via a shadow-verification harness that dual-decodes sampled traffic with hyperpb and protobuf-go and compares field digests — the divergence appeared only after our runtime PGO warmup completed, which is what isolated Recompile as the variable.
• Happy to test a fix; we currently work around it by not calling Recompile.

[mcy]

Hi! There is a fix at #40, feel free to cherry-pick that. I plan to merge it shortly.