What's Changed
- Feat: Generator fan-out in transform pipeline by @bpolaszek in #59
- Feat: Batch transform support (BatchTransformerInterface) by @bpolaszek in #60
Full Changelog: 4.1...4.2.0
New Features
Generator fan-out in transform pipeline (#59)
Chained transformers now support flatMap semantics: when a transformer yields multiple items via a Generator, each item is passed individually to the next transformer in the chain instead of passing the whole generator (see BC breaks below).
$etl = (new EtlExecutor())
->transformWith(
fn (string $item) => strrev($item),
function (string $item): Generator {
yield $item;
yield strtoupper($item);
},
fn (string $item) => "({$item})",
);
$report = $etl->process(['foo', 'bar']);
// Output: ['(oof)', '(OOF)', '(rab)', '(RAB)']Batch transform support (#60)
New BatchTransformerInterface allows transforming multiple items at once — ideal for concurrent HTTP requests or any batchable operation.
$executor = (new EtlExecutor())
->transformWith(new CallableBatchTransformer(
function (array $items, EtlState $state): array {
return $httpClient->sendConcurrent(
array_map(fn ($item) => new Request('GET', $item['url']), $items)
);
}
))
->withOptions(new EtlConfiguration(batchSize: 10))
->process($source);EtlConfigurationgains abatchSizeoption (default:1, no batching)- Items are chunked automatically and passed as an array to the batch transformer
bentools/iterable-functionsis now a production dependency
Breaking Changes
| Impact | Description |
|---|---|
Chained transformers that type-hint Generator as input will break — they now receive individual items instead |
|
$event->transformResult now contains a single item instead of the full generator output |
|
skip() now only skips the individual transformed item, not all items from the generator |
|
Property type changed from TransformerInterface to TransformerInterface|BatchTransformerInterface |