Console Parallelization Laravel Package

Features

• Drop support for PHP 8.1 and 8.2 (#309)
• Make the PHP executable into an array which allows to configure the PHP executable for child processes (#314, #329)

Fixes

• Fix the forwarding of array options (#326)
• Fix the double escaping done to options passed to the child process (#325)
• Harden the handling of non-rewindable generators and non-array keys (#330)

Misc

• Bump dependency constraints (#313)

Features

• Allow thecodingmachine/safe 3 (#306)

Features

• Use the available CPUs (#299)
• Add support for Symfony 7 (#267)

Fixes

• Handle the case where SERVER['PWD'] is not set (#219)
• Do not prepend the working directory for the script path when the script path is an absolute path (#268)
• Remove hooks API deprecations (#275)

Misc

• Bump minimum PHP version supported to 8.1 (#207)
• Bump minimum Symfony version supported to 5.4 (#210)

Features

• 3428044 Add ::configureParallelExecutorFactory (#201)
• 7c29eb3 Add a decorator logger (#202)

Bugfixes

• 578c646 Update CPUCoreCounter (#198)
• 9bd951d Fix exit code can exceed 255 (#203)

Misc

• bf173c8 Add PHP 8.2 to the CI (#199)
• 8814d54 Fix the PHPStan build name (#200)

Bugfixes

• aa52e0c Improve CPU cores count detection (#189)

Misc

• 3a23cbf Add test for CpuCoreCounter (#192)
• 7c28119 Move CpuCoreCounter to a sub-class (#193)
• 744c0b0 Add composer_normalize Makefile command (#194)
• 4a8a47c Update PHPStan (#196)
• 5f9e090 Tweak the Makefile commands (#197)
• 008284f Add MakefileTest (#195)

Bugfixes

• 436871f Allow PSR log 3 (#184)

Misc

• 7f8d421 Fix flaky test (#183)
• 07f7bbc Fix readme example (#182)

Changelog since 1.2.4:

Highlights

• A user-friendly BC layer
• A new ParallelMap command as an alternative to the Parallelization trait
• A lot more extension points, more comprehensive extension points, more user-friendly validation
• Remove the master terminology usage
• Allow the processing of an unknown number of items
• Guesses the maximum number of parallel processes it can use (no longer need to pass processes=X)
• Allow the library to be used in a non-Symfony app context

This release contains a number of BC breaks, they should all be explained in UPGRADE.md.

Features

• Remove support for PHP 7.3 (#57)
• Remove support for Symfony 3, 4.x except 4.4, 5.x except 5.4 (#57, #60, #61)
• Rework in depth the codebase to provide more re-usable and testable pieces
  • Modernize the codebase via PHPStan/PHPStorm (#70, #71, #72)
  • ItemBatchIterator / ChunkedItemsIterator:
    • Rename ItemBatchIterator to ChunkedItemsIterator (#69)
    • Remove ItemBatchIterator#batchSize (#68)
    • Introduce a ::fromStream() static factory which makes it more convenient to create the iterator for STDIN whilst allowing testing easily without relying on STDIN (#75)
    • Rename ::create() to ::fromItemOrCallable() (#75)
    • Move the normalization of items from __construct() to the static factories, this makes testing a lot easier (#75)
    • Simplify some tests: when testing the static factories we are only interested in the items themselves since the batchSize is just forwarded. The chunk logic is tested when testing the constructor directly (#75)
  • ParallelizationInput:
    • Introduce ParallelizationInput::fromInput() (#74)
    • Rename ParallelizationInput::configureParallelization() to ::configureCommand() (#143)
  • InputOptionsSerializer:
    • Introduce InputOptionsSerializer (#77, #78, #79)
  • Logger:
    • Introduce a custom logger (#80)
    • Allow the logger to be overwritten (#82)
    • Make the logger as a runtime parameter (#90)
    • Add NullLogger (#116)
    • Adjust the configuration logging (#130)
  • ProcessLauncher:
    • Remove string command from ProcessLauncher (#83)
  • ParallelExecutor / ParallelExecutorFactory
    • Introduce the ParallelExecutor (#86, #103, #104)
    • Introduce the ParallelExecutorFactory (#96)
    • Minor ParallelExecutor refactoring (#98)
  • ErrorHandler
    • Add an error handler (#88)
    • Rename ItemProcessingErrorHandler into ErrorHandler (#114)
    • Add null error handler (#115)
    • Make ResetContainerErrorHandler handle any resettable service (#140)
    • Make ResetServiceErrorHandler a decorator (#153)
  • ProcessLauncher / ProcessLauncherFactory
    • Introduce a ProcessLauncherFactory (#99, #100)
    • Make the process factory and process tick extensible (#149)
  • Parallelization:
    • Remove logError property (#107)
  • Configuration:
    • Move configuration calculation to a factory (#134)
    • Refactor Configuration (#146)
  • Add a ParallelizationCommand console command (#121)
  • ChildCommandFactory:
    • Extract child command factory into a class (#157)
• Add a dot at the end of each exception message sentence (#73)
• Remove dependencies to the Container (#88, #91, #92, #95, #109, #111)
• Allow a unknown count of items (#128)
• Automatically set the maximum number of processes to use (#137)
• Always do processing in child processes unless --main-process is passed (#138)
• Add a more friendly BC layer (#139)
• Limit the number of parallel processes (#147)
• Ensures the item is processed in the main process when given (#145)
• Keep track of the exit codes (#154)
• Pass output to the logger factory (#156)
• Always pass the full IO (#167)
• Improve Logger docs and re-order the methods (#169)
• Rename logger methods (#170)
• Handle negative process exit code (#171)
• Improve the Logger API (#168)
• Forward the child process output type to the logger (#174)
• Rework the output (#166)
• Update parallelization input arguments and options description (#175)
• Add batch and segment size options (#163)

Bugfixes

• Fix the handling of negatable options when forwarding the options to the child process(es) (#77, #123)
• Fix the options quoting detection when forwarding the options to the child process(es) (#77)
• Fix the forwarding of options: do not forward non given options (#81)
• Fix the support of resettable containers and more generally resettable services (#88)
• Prevent an item to be passed to a child process via the argument (#126)
• Fix the calculation of the segment size (#132)
• Fix the calculation of the batch size (#132)
• Use the validated/calculated segment size instead of the user configured one (#133)
• Ensure the items do not contain new lines (#148)
• Fix child command generation if the command or item argument are not passed as the first argument (#158)
• Do not forward default arguments to the child process(es) (#158)
• Properly escape the PHP executable (#162)
• Handle the case where no items has been returned (#178)

Misc

• Migrate from Travis CI to GitHub Actions (#56)
• Rework the initial CI to an acceptable state (#62, #13, #64, #65)
• Update CS (#58, #59, #63, #66, #108, #125)
• Improve the developer development experience with the Makefile (#84, #101, #159, #161)
• Upgrade to PHPUnit 9 (#85)
• Add infection (#89)
• Simplify the passed environment variables (#91)
• Rename PHP executable finder (#93)
• Pass progress symbol as a parameter instead of part of the constructor (#97)
• Add a variety of tests and make the tests more robust (#102, #124, #141)
• Add missing tests to reach 100% code coverage (#105)
• Fix issues pointed by Infection (#106)
• Move Inputs related classes under an Input namespace (#110)
• Remove references to "master" (#118)
• Add code of conduct (#119)
• Checks the Symfony different version compatibilies in the CI (#122, #123)
• Rename "number of batches" to "total number of batches" (#131)
• Split the PHPStan configuration (#135)
• Go to a higher level for PHPStan (#127)
• Add a test script to check how it works with no framework (#142)
• Remove legacy container tests (#144)
• Add @internal tag (#150)
• Restore default batch size configuration (#151)
• Remove obsolete TODOs (#152, #155)
• Add a method to get the input raw arguments (#160)
• Add missing @internal tags (#165)
• Fix PHPUnit coverage (#172)
• Fix flaky tests (#173)
• Introduce a more convenient exception for unexpected calls (#177)

Bugfixes

• Handle the case where no items has been returned (#178)

Misc

• Introduce a more convenient exception for unexpected calls (#177)

[BC Break]: BC break when migrating from 1.x. [Beta BC break]: BC break when migrating from 2.0.0-beta.0.

Features

• [BC Break] Always pass the full IO (#167)
• [Beta BC break] Improve Logger docs and re-order the methods (#169)
• [Beta BC break] Rename logger methods (#170)
• Handle negative process exit code (#171)
• [Beta BC break] Improve the Logger API (#168)
• [Beta BC break] Forward the child process output type to the logger (#174)
• Rework the output (#166)
• Update parallelization input arguments and options description (#175)
• Add batch and segment size options (#163)

Misc

• Add missing @internal tags (#165)
• Fix PHPUnit coverage (#172)
• Fix flaky tests (#173)

Highlights

• A user-friendly BC layer
• A new ParallelMap command as an alternative to the Parallelization trait
• A lot more extension points, more comprehensive extension points, more user-friendly validation
• Remove the master terminology usage
• Allow the processing of an unknown number of items
• Guesses the maximum number of parallel processes it can use (no longer need to pass processes=X)
• Allow the library to be used in a non-Symfony app context

This release contains a number of BC breaks, they should all be explained in UPGRADE.md.

Features

• Remove support for PHP 7.3 (#57)
• Remove support for Symfony 3, 4.x except 4.4, 5.x except 5.4 (#57, #60, #61)
• Rework in depth the codebase to provide more re-usable and testable pieces
  • Modernize the codebase via PHPStan/PHPStorm (#70, #71, #72)
  • ItemBatchIterator / ChunkedItemsIterator:
    • Rename ItemBatchIterator to ChunkedItemsIterator (#69)
    • Remove ItemBatchIterator#batchSize (#68)
    • Introduce a ::fromStream() static factory which makes it more convenient to create the iterator for STDIN whilst allowing testing easily without relying on STDIN (#75)
    • Rename ::create() to ::fromItemOrCallable() (#75)
    • Move the normalization of items from __construct() to the static factories, this makes testing a lot easier (#75)
    • Simplify some tests: when testing the static factories we are only interested in the items themselves since the batchSize is just forwarded. The chunk logic is tested when testing the constructor directly (#75)
  • ParallelizationInput:
    • Introduce ParallelizationInput::fromInput() (#74)
    • Rename ParallelizationInput::configureParallelization() to ::configureCommand() (#143)
  • InputOptionsSerializer:
    • Introduce InputOptionsSerializer (#77, #78, #79)
  • Logger:
    • Introduce a custom logger (#80)
    • Allow the logger to be overwritten (#82)
    • Make the logger as a runtime parameter (#90)
    • Add NullLogger (#116)
    • Adjust the configuration logging (#130)
  • ProcessLauncher:
    • Remove string command from ProcessLauncher (#83)
  • ParallelExecutor / ParallelExecutorFactory
    • Introduce the ParallelExecutor (#86, #103, #104)
    • Introduce the ParallelExecutorFactory (#96)
    • Minor ParallelExecutor refactoring (#98)
  • ErrorHandler
    • Add an error handler (#88)
    • Rename ItemProcessingErrorHandler into ErrorHandler (#114)
    • Add null error handler (#115)
    • Make ResetContainerErrorHandler handle any resettable service (#140)
    • Make ResetServiceErrorHandler a decorator (#153)
  • ProcessLauncher / ProcessLauncherFactory
    • Introduce a ProcessLauncherFactory (#99, #100)
    • Make the process factory and process tick extensible (#149)
  • Parallelization:
    • Remove logError property (#107)
  • Configuration:
    • Move configuration calculation to a factory (#134)
    • Refactor Configuration (#146)
  • Add a ParallelizationCommand console command (#121)
  • ChildCommandFactory:
    • Extract child command factory into a class (#157)
• Add a dot at the end of each exception message sentence (#73)
• Remove dependencies to the Container (#88, #91, #92, #95, #109, #111)
• Allow a unknown count of items (#128)
• Automatically set the maximum number of processes to use (#137)
• Always do processing in child processes unless --main-process is passed (#138)
• Add a more friendly BC layer (#139)
• Limit the number of parallel processes (#147)
• Ensures the item is processed in the main process when given (#145)
• Keep track of the exit codes (#154)
• Pass output to the logger factory (#156)

Bugfixes

• Fix the handling of negatable options when forwarding the options to the child process(es) (#77, #123)
• Fix the options quoting detection when forwarding the options to the child process(es) (#77)
• Fix the forwarding of options: do not forward non given options (#81)
• Fix the support of resettable containers and more generally resettable services (#88)
• Prevent an item to be passed to a child process via the argument (#126)
• Fix the calculation of the segment size (#132)
• Fix the calculation of the batch size (#132)
• Use the validated/calculated segment size instead of the user configured one (#133)
• Ensure the items do not contain new lines (#148)
• Fix child command generation if the command or item argument are not passed as the first argument (#158)
• Do not forward default arguments to the child process(es) (#158)
• Properly escape the PHP executable (#162)

Misc

• Migrate from Travis CI to GitHub Actions (#56)
• Rework the initial CI to an acceptable state (#62, #13, #64, #65)
• Update CS (#58, #59, #63, #66, #108, #125)
• Improve the developer development experience with the Makefile (#84, #101, #159, #161)
• Upgrade to PHPUnit 9 (#85)
• Add infection (#89)
• Simplify the passed environment variables (#91)
• Rename PHP executable finder (#93)
• Pass progress symbol as a parameter instead of part of the constructor (#97)
• Add a variety of tests and make the tests more robust (#102, #124, #141)
• Add missing tests to reach 100% code coverage (#105)
• Fix issues pointed by Infection (#106)
• Move Inputs related classes under an Input namespace (#110)
• Remove references to "master" (#118)
• Add code of conduct (#119)
• Checks the Symfony different version compatibilies in the CI (#122, #123)
• Rename "number of batches" to "total number of batches" (#131)
• Split the PHPStan configuration (#135)
• Go to a higher level for PHPStan (#127)
• Add a test script to check how it works with no framework (#142)
• Remove legacy container tests (#144)
• Add @internal tag (#150)
• Restore default batch size configuration (#151)
• Remove obsolete TODOs (#152, #155)
• Add a method to get the input raw arguments (#160)

Bugfixes

• Option value could be of type ' int' in case it's defined as a default (#51)

Fix release process

Bugfixes

• Fix Symfony 5 compatibility (#48)

Bugfixes

• Fix Symfony 5 compatibility (#48)

Features

• Add PHP8 support (#46, #47)
• Make getEnvironmentVariables protected

Bugfixes

• Use the Kernel project dir instead of root dir (#45)

Misc

• Introduce ParallelizationInput (#30)
• Introduce a Configuration class (#29)
• Introduce the ItemBatchIterator VO (#33)

Features

• Add protected method to get the console path (#43)

Bugfixes

• Add quotation to children's input options if required (#42)

Bugfixes

• Pass input options to child process (#28)

Bugfixes

• Fix ProcessLauncher on Windows (#25)

Features

• Allow Symfony 5 (#23)

Bugfixes

• Do not spawn sub-processes if the --processes option is not used whilst still allowing to spawn only one sub-process (with --processes=1) (#22)

Misc

• Add PHP 7.4 to the test build
• Ensures all major Symfony versions supported (3.x, 4.x, 5.x)

Misc

• Added support for Symfony 3 (#18)

Bugfixes

• Fixes child processes console path (#3)
• Fixed CVE-2019-10910 (#9)
• Fixed performance problems when using multiprocessing (#11)
• Use child processes even if number of processes is only one (#15)

Misc

• Add Travis configuration (#4)
• Tweak the composer.json file (#5)
• Add Makefile (#6)
• Fix CS (#7)

• renamed fetchTasks() to fetchItems()
• renamed getTaskName() to getItemName()