PHP-ETL - Understand the ETL

Execution Context - Why to have an execution context & what it does

Execution Context - Why to have an execution context & what it does

In most of our examples our chain had access to the whole file system. This means having multiple chains running together, or having a list of files each execution has generated is impossible.

Both the 🎵 Symfony Bundle(and therefore the 🦢 Sylius integration) and the Magento2 Module will use contextual chains. This means the “main” operations have only access to a particular directory created for the execution of the chain.

Additional operations such as the ExternalFileFinderOperation and ExternalFileProcessor will be use to process files that are either on a remote directory (sftp, bucket s3…) or files that are on the local file system. Because operations such as the CsvLoader will not have access to those files unless they are copied into the contextual directory of the current execution.

Let start by a simple example.

Write the result of an API to a CSV File.

<?php
$workdir = __DIR__ . "/var/";
$dirManager = new ChainWorkDirManager($workdir);
$loggerFactory = new NullLoggerFactory();
$fileFactory = new LocalFileSystemFactory($dirManager);

return new PerExecutionContextFactory(
        $dirManager,
        $fileFactory,
        $loggerFactory
    );

$options = [
    'etl' => [
        'execution' => new PockExecution(new DateTime())
    ]
];

$chainProcessor->process(
    new ArrayIterator([[]]),
    $options
);

Executing this will create a directory in var/ with the output result. Everytime you execute the chain a new directory wil be created.