Flusher

By endroid

When you import or modify large amounts of data it is often necessary to define the optimal batch size before flushing: small batch sizes perform bad because of the overhead in each flush. And batch sizes that are too large perform bad because of the high memory usage and the need to calculate a large change set. Also the batch size you choose can give different results on different types of hardware.

This library helps you write entities to the database without worrying about the batch size. It incrementally tries new batch sizes (given a step size), sticks with the one that gives the highest performance or switches to a better batch size if the circumstances have changed.

Installation

Use Composer to install the library.

$ composer require endroid/flusher

Usage

In order to enable auto flushing you first need to create a Flusher for the entity manager you are currently using.

$flusher = new Flusher($manager);

Then when you performed operations on your entity manager you can call the flush() method on the flusher any time to notify there are changes.

for ($n = 1; $n <= 50000; $n++) {
    $task = new Task();
    $task->setName('Task '.$n);
    $manager->persist($task);
    $flusher->flush();
}

Because there is no way of knowing if there are pending flushes at the end you need to call finish() to make sure all data is flushed.

$flusher->finish();

Versioning

Version numbers follow the MAJOR.MINOR.PATCH scheme. Backwards compatibility breaking changes will be kept to a minimum but be aware that these can occur. Lock your dependencies for production and test your code when upgrading.

License

This bundle is under the MIT license. For the full copyright and license information please view the LICENSE file that was distributed with this source code.