PHP-ETL - Cook Books
With Context - Import External File
With Context - Import External File
The ETL can be used to fetch files from another filesystem and process them using the ETL. Files will be moved into
processing and processed directories as the ETL runs.
We will use the ExternalFileFinderOperation to find the files. And use the ExternalFileProcessorOperation to copy
the files to the context of the ETL execution.
The file finder will use a directory that we will add to the context at the beginning of the execution.
We will also need to provide the finder with a regex as chain input so that it can find all the files.
find-file1:
operation: external-file-finder-local
options:
directory: "@context['dir']"
process-new-file1:
operation: external-file-processor
options: []
š Standalone
$options = [
// ...
'dir' => $dir,
];
$chainProcessor->process(
new ArrayIterator(['/^file[0-9]\.csv$/']),
$options
);
šµ Symfony
./bin/console etl:execute myetl "['/^file[0-9]\.csv$/']" "{'dir': '/var/import'}"
Complete Code
chain:
find-file1:
operation: external-file-finder-local
options:
directory: "@context['dir']"
process-new-file1:
operation: external-file-processor
options: []
read-file:
operation: csv-read
options: []
write-new-file:
operation: csv-write
options:
file: "output.csv"
process-finalized-file1:
operation: external-file-processor
options: []