Implement async byte latent transformer with entropy-based patching

[laerdon]

overview

this pr implements a complete asynchronous byte latent transformer (blt) architecture with entropy-based dynamic patching, based on the facebook research paper "byte latent transformer: patches scale better than tokens".

key features

byte encoder

• dynamically sized patches based on next-byte prediction entropy
• higher entropy regions get more compute allocation
• configurable patch size bounds (min/max)
• no tokenization required - works directly with raw bytes

async patch processor

• parallel processing of variable-length patches
• async attention layers for improved throughput
• configurable worker pool for concurrent operations
• supports both sync and async execution modes

transformer architecture

• multi-head self-attention with residual connections
• feed-forward networks with gelu activation
• layer normalization and positional encoding
• configurable depth and width

training infrastructure

• async training loop with gradient accumulation
• cosine annealing learning rate schedule
• checkpoint saving and loading
• validation support

testing

all components have comprehensive tests:

• byte encoder tests: entropy computation, patch segmentation, batch encoding
• async processor tests: parallel processing, attention mechanisms
• blt model tests: forward pass, generation, parameter counting

all tests pass successfully:

[PASS] all byte encoder tests passed
[PASS] all async processor tests passed
[PASS] all blt model tests passed

examples

included example scripts demonstrate:

• inference and text generation
• training with custom data
• model configuration options

usage

from blt.models.blt_model import ByteLatentTransformer

model = ByteLatentTransformer(
    d_model=512,
    num_layers=6,
    num_heads=8,
    use_async=True,
)

# process bytes directly
byte_tensor = torch.tensor(list(text.encode('utf-8')), dtype=torch.long)
logits, patch_sizes = model(byte_tensor)

implementation details

• entropy-based segmentation allocates compute based on data complexity
• async processing enables parallel patch handling
• supports both single sequence and batch processing
• includes comprehensive documentation in readme and implementation.md

files changed

• blt/models/byte_encoder.py: entropy-based byte encoder
• blt/models/patch_processor.py: async patch processing
• blt/models/blt_model.py: main transformer architecture
• blt/utils/trainer.py: async training loop
• blt/utils/data_loader.py: byte sequence data loading
• examples/: training and inference examples
• tests/: comprehensive test suite
• README.md: complete documentation
• IMPLEMENTATION.md: detailed implementation summary

Slack Thread

  

[laerdon]

Comment: this was just seeing if Opus 4.6 would one-shot it for fun