PHP
Challenge - Generators and closures
Build an ETL that processes a large data stream with generators, yield from and closures.
Lazy ETL pipeline with generators
In this five-lesson expert final you'll build a mini application that combines every expert topic. The first lesson sets up an ETL pipeline: readRows yields rows one by one, transform applies a closure to each, pipeline delegates to the inner generator with yield from, and loadAll collects results into an array.
Generators process large data without loading it all into memory; each row is produced on demand. With closures and arrow functions (fn) the transform step is a single line. This foundation will host reflection and performance layers in later lessons.
Exercise Write the ETL pipeline
Write the ETL pipeline. readRows(array $rows): Generator yields each row. transform(iterable $rows, callable $fn): Generator applies $fn to each row. pipeline(array $rows, callable $fn): Generator uses yield from to delegate to readRows + transform. loadAll(iterable $rows): array collects rows into an array.
Demo: $rows = [["name" => "kale", "price" => 5], ["name" => "defter", "price" => 15]]. Pass an arrow function to pipeline: fn($r) => ["name" => strtoupper($r["name"]), "price" => $r["price"] * 2]. Collect with loadAll and print each row as "{name}: {price}\n".
Show example solution
yield each row. transform: apply callable fn to each row. pipeline: use yield from to delegate to the readRows + transform chain. loadAll: collect results into an array. Write the demo and produce the output.
Example solution:
<?php
// ETL pipeline: readRows + transform + pipeline + loadAll
function readRows(array $rows): Generator {
foreach ($rows as $row) {
yield $row;
}
}
function transform(iterable $rows, callable $fn): Generator {
foreach ($rows as $row) {
yield $fn($row);
}
}
function pipeline(array $rows, callable $fn): Generator {
yield from transform(readRows($rows), $fn);
}
function loadAll(iterable $rows): array {
$out = [];
foreach ($rows as $row) {
$out[] = $row;
}
return $out;
}
// Demo:
$rows = [
["name" => "kale", "price" => 5],
["name" => "defter", "price" => 15],
];
$result = loadAll(pipeline($rows, fn($r) => [
"name" => strtoupper($r["name"]),
"price" => $r["price"] * 2,
]));
foreach ($result as $row) {
echo $row["name"] . ": " . $row["price"] . "\n";
}
The power of closure pipelines
A closure pipeline is an elegant way to process data step by step. Each step takes a callable and passes the result to the next. Combined with generators it becomes memory friendly: you can stream thousands of rows without holding them all at once.
The arrow function fn($r) => expr keeps one-line transforms short and captures outer variables automatically. yield from merges nested generators into a single stream. This pattern carries into the reflection and performance layers of later lessons; each lesson adds one piece.
Test yourself
-
What does the yield keyword do?
- Produces a value, returns it to the caller, then pauses
- Terminates the function completely
- Loads the whole array into memory
- Throws an exception
Show answer
Produces a value, returns it to the caller, then pauses — Correct. yield produces a value, hands it to the caller and resumes from that point on the next call.
-
Complete the yield from expression
function pipeline(array $rows, callable $fn): Generator {
____ transform(readRows($rows), $fn);
}
Show answer
function pipeline(array $rows, callable $fn): Generator {
yield from transform(readRows($rows), $fn);
}
-
Match ETL and generator concepts
- yield
- Produces one value
- yield from
- Delegates to inner generator
- Generator
- Lazy iterator interface
- callable
- Function reference
Next lesson: Challenge - Reflection app
