Process Bundle Laravel Package

Changes

• #365 ArrayElementTransformer (array_element) and ArrayLastTransformer (array_last) return null when the position is outside the array (or the array is empty), without the "Undefined array key" PHP warning (it was an exception in debug mode, null in production); a negative index before the start of the array no longer returns its first element. InputCsvReaderTask (input_csv_reader): an input that is not a path (e.g. null) throws an explicit UnexpectedValueException instead of an "Undefined array key" PHP warning followed by a TypeError. Documentation: rate n/a of advanced_stat_counter, headers: [] of csv_reader / input_csv_reader, optional $accessor of ConditionTrait, ParsedExpression accepted by the condition of rules. Add tests.
• #337 StopTask (stop): a process stopped on purpose is identified as stopped, not as failed. Its history is flagged as stopped (new ProcessHistory::STATE_STOPPED, setStopped(), isStopped()) instead of failed, the Process <code> stopped log is written at the success_level (instead of Process <code> failed at the failed_level), and the cleverage_process.end event tells it with the new ProcessEvent::isProcessStopped(). The process still ends normally (tasks finalized, exit code 0, parent process going on). Update documentation, add tests.
• #339 A shared service referenced by several tasks of the same process is rejected when the container is compiled (InvalidArgumentException naming the service, the tasks and the process): all these tasks ran with the same instance, initialized with the options of each task in turn, without error. Declare the task services with shared: false (applied by the autoconfiguration of TaskInterface since 5.4). Update documentation, add tests.
• #351 ProcessManager and ProcessHelpCommand are typed on TaskRegistryInterface instead of the final TaskRegistry class: the task registry can be decorated or replaced. Add tests.
• #357 Documentation: input of file_splitter, log_empty_lines of the CSV readers, empty lines of the line readers, invalid regular expressions of the conditions, cache pool of the cached transformer, missing serializer / validator, exceptions of the bundle in the reference pages, failure of a sub-process of process_executor, sleep_on_finalize_interval of process_launcher, initialization failures, process_options of process_launcher in UPGRADE.md.

Fixes

• #343 The temporary files of FileSplitterTask (file_splitter), CsvSplitterTask (csv_splitter) and of the JSON buffer of ProcessLauncherTask (json_buffering) are created with a random name, exclusively (an existing path, symbolic links included, is never reused) and readable by their owner only, instead of a name derived from the time (uniqid()), opened in w mode with the permissions of the umask. Update documentation, add tests.
• #331 ProcessManager: the iterable tasks that produced items and the blocking tasks waiting for proceed() are tracked by task configuration instead of task code, and forgotten when their process ends. A task of a sub-process (process_executor) with the same code as a task of the parent no longer makes the parent lose the output of its blocking task (without error) or skip the flush at the end of its iteration (merged batches), and a failed sub-process no longer leaves its tasks to its next execution. Add tests.
• #333 ProcessManager: restore the previous task configuration (getTaskConfiguration(), read by the log processors for task_code and task_service) when a task, an initialization, a finalization or a sub-process ends. The logs of an iterable task no longer lose their task after its first item, and the logs of a parent process are no longer attributed to a task of its sub-process; the failing task is kept for the logs of the process failure. Add tests.
• #335 LineReaderTask (line_reader) and InputLineReaderTask (input_line_reader): the last line of a file that does not end with a line break is no longer lost. Add tests.
• #345 YamlWriterTask (yaml_writer): a write failure throws a RuntimeException instead of a PHP warning (an exception in debug mode only: in production, the path was output as if the file had been written), and the parent directory is created if needed, as for the other writers. Update documentation, add tests.
• #353 cleverage:process:execute: the process codes and the output format are checked before any process is executed (explicit message for dump without symfony/var-dumper); the output is printed to STDOUT whatever the verbosity when --output-format is given (it required -vv); a non-array output that cannot be stored as JSON stream is reported on the error output instead of being silently lost. Update documentation, add tests.
• #355 ProcessConfigurationRegistry::reset(), tagged kernel.reset: a long-running worker (Messenger, UI bundle) releases the resolved process configurations, with the task instances and states (inputs, outputs, aggregated data) of their last execution, between two messages. Add tests.

Changes

• #266 Allow to stop a running process from the outside: services implementing the new ProcessStopCheckerInterface (autoconfigured, tag cleverage_process.stop_checker, new optional $stopCheckers argument of ProcessManager) are asked before each task execution; when one of them returns true, the task is not executed and the process fails with a ProcessFailedException whose previous exception is the new ProcessStoppedException. Add documentation and tests.

Changes

• #197 Add missing tests: every Task and Transformer is now covered by unit tests. Remove the obsolete tests.old directory.
• #143 Improve PHPStan configuration: remove all ignoreErrors and [@phpstan-ignore](https://github.com/phpstan-ignore) comments (report unmatched ignored errors again), add missing iterable value types and generic types in PHPDoc, remove unreachable code in AbstractIterableOutputTask and InputAggregatorTask.
• #143 Improve PHPStan level from 6 to 7, and fix the level 8 errors that do not require a signature change (the remaining ones, due to nullable return types such as AbstractConfigurableTask::getOptions(): ?array, will be fixed in v6.0). Default to a ContextualOptionResolver in ProcessState and to a property accessor in ConditionTrait when none is set.
• #94 TransformerTrait: the transformers option (TransformerTask, MappingTransformer, ArrayMapTransformer, CachedTransformer, RulesTransformer, generic transformers) also accepts a list, whose items are a transformer code without options (- trim) or a single code: options map (- callback: {...}), to chain the same transformer without # suffix. The map syntax is still supported. Update documentation, add tests.
• #117 MappingTransformer: writing a target property that is a nested path (e.g. address.city) on an array destination, which silently created an address.city literal key, is deprecated and will throw an \UnexpectedValueException in v6.0. Use [address.city] for a literal key or [address][city] for a nested array; a simple property name (e.g. name) is still added as a key. Update documentation, add tests.
• #258 Add XmlStreamReaderTask: iterable task reading an XML file element by element with \XMLReader (constant memory), outputting each element named node_name as a \DOMElement in its own \DOMDocument, ready for XpathEvaluatorTransformer. The file path is set in the file_path option or given as input. Add documentation and tests.
• #259 JsonStreamWriterTask: add a mode option (default wb, e.g. ab to append to an existing JSON Lines file), and release the file on finalize(). Update documentation, add tests.
• #157 Configure the level of the logs written at the end of each process with the new logs option, globally (clever_age_process.logs) or per process (configurations.<code>.logs): success_level (default info) for Process <code> succeed, and failed_level (default debug) for the new Process <code> failed log, written when a process fails, with or without exception. Update documentation, add tests.

Fixes

• #143 Fix InputIteratorTask: an \IteratorAggregate input whose getIterator() does not return an \Iterator (e.g. another \IteratorAggregate) is iterated instead of failing with a TypeError. Update documentation, add tests.
• #143 Fix level 8 edge cases failing with a TypeError or an Error: CsvResource rejects a blank header line and checks ftell(), XpathEvaluatorTransformer throws an explicit \UnexpectedValueException on an invalid query (with the libxml messages), a namespace node to unwrap or a node without document, SlugifyTransformer on a value that cannot be transliterated, ProcessState::getContext() returns [] until the context is defined, GenericTransformer and TransformerTrait throw a \LogicException when not initialized, IterableBatchTask and JsonStreamReaderTask can be flushed / iterated before their first execution, RegistryCompilerPass checks its configuration, SubprocessInstance checks the PHP executable. Update documentation, add tests.
• #239 Fix PropertyGetterTask: catch any \Throwable, so that the TypeError of the property accessor on a scalar input is set on the state (with the property error context) and handled according to the task error_strategy instead of escaping it. Update documentation, add tests.
• #241 Fix CsvWriterTask: a non-array input throws the documented \UnexpectedValueException instead of a TypeError when headers is null, and the CSV tasks release their file on finalize() so that the task can be executed again. Update documentation, add tests.
• #240 Fix FileMoverTask: with autoincrement, the suffix was computed on the whole path (a dot in a directory name was taken as the extension, making the move fail for a file without extension) and an existing numeric suffix of the file name was replaced (report-2024.csv became report-1.csv). The suffix is now appended to the file name only (report-2024-1.csv, file-1, .env-1). Update documentation, add tests.
• #243 Fix RecursivePropertySetterTransformer: a \stdClass item without the property was replaced by a copy in the output, so the input object was not modified; the property is now added to the item itself. Update documentation, add tests.
• #242 Fix InputFileReaderTask: an input that is not a non-empty string (e.g. null) throws an explicit \UnexpectedValueException (No file path given as input) instead of a PHP warning followed by a TypeError. Update documentation, add tests.
• #244 Fix MappingTransformer: a missing target property of a \stdClass destination (initial_value or keep_input) threw Property '...' is not writable, it is now added when the target is a simple property name (nested paths still throw). Update documentation, add tests.
• #143 Type the remaining untyped task options: split_character of CsvWriterTask and SplitJoinLineTask must be a string (a wrong type used to fail later with a TypeError), write_headers of CsvWriterTask and log_empty_lines of CsvReaderTask are cast to bool (any value used to be evaluated as a boolean, so it is still accepted). Update documentation, add tests.

Changes

• #190 Update quality stack: use Rector withComposerBased() sets (removed SYMFONY_64 / PHPUNIT_100 sets), declare used Symfony packages and PHPUnit range in composer.json, apply quality tools fixes
• #145 Add missing documentations: reference pages for every Task & Transformer, ConditionTrait & GenericTransformer, complete guides and cookbooks. Harmonize and fix existing documentation.
• #229 StatCounterTask and AdvancedStatCounterTask pass their input to their output on every execution (they used to output null, and AdvancedStatCounterTask was skipped when not logging), so they can be placed anywhere in a branch. Update documentation, add tests.
• #236 TransformerTrait: document that any non-empty #suffix is accepted on transformer codes (callback#1, callback#reverse), and anchor the matching regex (an empty suffix is no longer accepted). Update documentation, add tests.

Fixes

• #192 Fix CommandRunnerTask: only pass the options option to Process::setOptions(), support string commandline through Process::fromShellCommandline(), validate option types. Update documentation, add tests.
• #194 Fix ProcessLauncherTask: the process_options normalizer returned an array despite its scalar return type, so the task always failed with a TypeError. Update documentation, add tests.
• #189 Fix EventDispatcherTask: dispatch the event under the event_name option (regression since v4.0). event_name is now optional: when null (default), the event is dispatched under its class name. Update documentation, add tests.
• #198 Fix DateParserTransformer: a \DateTimeImmutable input is converted to a \DateTime instead of throwing a TypeError. Update documentation, add tests.
• #199 Fix PregFilterTransformer: an array replacement is no longer cast to the string "Array", and requires an array pattern. Update documentation, add tests.
• #200 Fix GenericTransformer: a contextual option declared with required: false and no default can now be used; its placeholders are replaced by null when omitted. Update documentation, add tests.
• #202 Fix XmlReaderTask: throw an explicit \UnexpectedValueException on an empty file or invalid XML (with the libxml error messages) instead of a ValueError or a silent empty \DOMDocument. Update documentation, add tests.
• #203 Fix ProcessState::addErrorContextValue(): accept any value type, so that iterable tasks on an empty iterable are skipped and PropertySetterTask reports the original exception instead of a TypeError. Update documentation, add tests.
• #204 Fix CachedTransformer, SlugifyTransformer and TypeSetterTransformer edge cases: accept a non-string input in cached (key built after key_transformers), reject an invalid transliterator in slugify with an InvalidOptionsException, remove the unreachable error branch of type_setter. Update documentation, add tests.
• #205 Fix FileSplitterTask: produced files lost the first line, doubled line breaks and ended with an empty file; each file now contains exactly max_lines lines of the source. Update documentation, add tests.
• #206 Fix AdvancedStatCounterTask, IterableBatchTask, ConditionTrait and ColumnAggregatorTask edge cases: log the first counted execution with correct counts, accept a null batch_count (only flush at the end), allow scalar inputs in conditions ('' path on the whole value), aggregate null column values. Update documentation, add tests.
• #207 Fix JsonStreamReaderTask / JsonStreamWriterTask: throw an explicit \UnexpectedValueException when a line decodes to a scalar, create the missing parent directory when writing. Update documentation, add tests.
• #208 Fix FolderBrowserTask, InputFolderBrowserTask, CsvReaderTask and LineReaderTask (and their Input* variants): reset the state at the end of the iteration, so that a following input is read from its beginning. Update documentation, add tests.
• #201 Fix minor defects: error messages of TransformerTrait, RulesTransformer and ExpressionLanguageMapTransformer, useless setRequired() in ImplodeTransformer and SprintfTransformer, stray namespace in TrimTransformer, wrong or missing docblocks. Add tests.
• #220 Fix the stop error strategy: throw a ProcessFailedException (with the original exception as previous) instead of a FatalError, so that the command exits with a non-zero code when a process fails and ProcessLauncherTask detects failed sub-processes. Update documentation, add tests.
• #221 Fix CsvSplitterTask: each produced file contains max_lines data lines (instead of max_lines - 2), no infinite loop with max_lines <= 2 (max_lines must now be an integer greater than 0), no header-only file at the end. Update documentation, add tests.
• #223 Fix CounterTask: the final count is outputted once, as flush() may be called several times. Document that flush() implementations must be idempotent. Update documentation, add tests.
• #228 Add the missing setAllowedTypes() on HashTransformer (raw_output), SimpleBatchTask (batch_count), ConditionTrait (empty, not_empty), NormalizerTask / SerializerTask / DeserializerTask (context) and ObjectUpdaterTask (property_path): a wrong type is now reported when the options are resolved. Add tests.
• #232 Fix edge cases looping forever or failing with a TypeError: validate FileSplitterTask max_lines (greater than 0, also when given as input), catch any \Throwable in PropertySetterTask, type SlugifyTransformer options, explicit exceptions in InputFolderBrowserTask (no folder path) and XmlFile::write() (saveXML() failure). Update documentation, add tests.

Deprecated

• #189 EventDispatcherTask: when event_name is set, listening to CleverAge\ProcessBundle\Event\EventDispatcherTaskEvent is deprecated (the event is still dispatched under its class name, with an E_USER_DEPRECATED error, if it has listeners). Listen to the configured event_name instead: the BC layer will be removed in v6.0.
• #222 ProcessManager: going on with the process after an exception thrown by a task initialize() is deprecated (an E_USER_DEPRECATED error is triggered). In v6.0, the process will fail before executing any task.

Add

• #179 Support PHP 8.5 / Symfony 8
• #145 Documentation : Add array_map_transformer, multi_replace_transformer and sprintf_transformer documentations. Update implode_transformer and slugify_transformer documentations

Breaking Changes

• #179 Drop PHP 8.1 / Symfony 7.3

Changes

• #176 Upgrade to Symfony 7.3 & PHP 8.4

Add

• #168 Add JsonStreamWriterTask with related doc
• #172 Add spl_file_object_flags and json_flags options on JsonStream*Tasks. Update docs.
• #174 Add FileSplitterTask using Filesystem/SplFile. Add doc.

Fixes

• #169 Fix JsonStreamFile empty line at the end issue, even if SKIP_EMPTY is set. Add JsonStreamReaderTask doc.

Add

• #166 Add PregMatchTransformer

Add

• #cleverage/archive-process-bundle#1 Add new bridge ArchiveProcessBundle
• #162 Add timestamp placeholder on file_path parameter of CsvWriterTask. Improve documentation.
• #164 Add cleverage/cache-process-bundle dependency

Fixes

• #158 Add dependency to symfony/service-contract

Add

• https://github.com/cleverage/process-bundle/issues/155 Add InputFileReaderTask (Reads the whole input file and outputs its content), InputLineReaderTask (Reads an input file line by line and outputs each line.) and LineReaderTask (Reads a file line by line and outputs each line.)

BC breaks

• #142 Remove FileFetchTask, use cleverage/flysystem-process-bundle instead.
• #142 YamlReaderTask & YamlWriterTask namespaces changed to CleverAge\ProcessBundle\Task\File\Yaml
• #142 Array***Transformers namespaces changed to CleverAge\ProcessBundle\Transformer\Array
• #142 NormalizeTransformer & DenormalizeTransformer namespaces changed to CleverAge\ProcessBundle\Transformer\Serialization
• #142 DateFormatTransformer & DateParserTransformer namespaces changed to CleverAge\ProcessBundle\Transformer\Date
• #142 ExplodeTransformer, HashTransformer, ImplodeTransformer, SlugifyTransformer, SprintfTransformer & TrimTransformer namespaces changed to CleverAge\ProcessBundle\Transformer\String
• #142 InstantiateTransformer, PropertyAccessorTransformer RecursivePropertySetterTransformer namespaces changed to CleverAge\ProcessBundle\Transformer\Object
• #147 Replace Symfony\Component\Form\Exception\InvalidConfigurationException by Symfony\Component\Config\Definition\Exception\InvalidConfigurationException
• #148 Update services (step 1) according to Symfony best practices. Services should not use autowiring or autoconfiguration. Instead, all services should be defined explicitly. Services must be prefixed with the bundle alias instead of using fully qualified class names => cleverage_process
• #150 The class \CleverAge\ProcessBundle\Task\Debug\MemInfoDumpTask has been deleted without suggested replacement
• #115 New mandatory configuration default_error_strategy on clever_age_process level. See Quick Start/Global configuration

Changes

• #139 Update Makefile & .docker for local standalone usage
• #139 Update rector, phpstan & php-cs-fixer configurations & apply it
• #141 league/flysystem-bundle is not required anymore
• #130 EventDispatcherInterface service declaration breaks dependency injection
• #147 Add missing dependencies on symfony/dotenv and symfony/runtime
• #147 Remove dependencies on symfony/form, symfony/messenger & symfony/scheduler
• #146 eav-process-bundle, enqueue-process-bundle, cache-process-bundle and process-soap-bundle were deprecated / archived.
• #141 Add a default value to the node "default_error_strategy"

Fixes

• #129 Remove wrong replace configuration on composer.json. Add missing suggest
• Miscellaneous fixes, show full diff : https://github.com/cleverage/process-bundle/compare/v4.0.0-rc2...v4.0.0

Full Changelog: https://github.com/cleverage/process-bundle/compare/v3.2.8...v3.2.9

Upgrade psr/cache