Weave Code
Doctrine Orm Batcher Laravel Package
setono/doctrine-orm-batcher
Opinionated batch processing for Doctrine ORM using seek pagination (no slow LIMIT/OFFSET) to handle large datasets efficiently. Create ID ranges or collections of IDs/entities for synchronous or message-driven async processing.
At a glance
Frequently asked questions about Doctrine Orm Batcher
- How does setono/doctrine-orm-batcher improve performance compared to Laravel’s chunk() method?
- This package uses Doctrine’s native batching with the `seek` method for pagination, which avoids MySQL’s inefficient `LIMIT-OFFSET` for large datasets. It’s optimized for async processing and reduces memory spikes by handling entities in configurable chunks, whereas `chunk()` relies on PHP-level iteration and lacks built-in transaction control.
- Can I use this package with Laravel’s Eloquent models directly?
- Yes, since Eloquent uses Doctrine ORM under the hood, you can use Eloquent models with this package. However, ensure your models are properly mapped to Doctrine entities (via annotations, attributes, or XML/YAML). For hybrid Laravel/Symfony apps, it works seamlessly if Doctrine is already configured.
- What’s the recommended batch size for MySQL/PostgreSQL to avoid timeouts or deadlocks?
- Start with 500–2,000 records per batch for MySQL and adjust based on your schema complexity. PostgreSQL can often handle larger batches (e.g., 5,000+) due to better concurrency. Test with `memory_get_usage()` and monitor for lock contention or transaction timeouts. Smaller batches improve reliability but increase overhead.
- Will batch operations trigger Laravel model events (e.g., saved, deleted) for every entity?
- Yes, by default, events fire for each entity in a batch. To disable this, wrap the batcher in a transaction and use `EventManager::clear()` or override the batcher’s event handling. Alternatively, isolate critical events by using a custom event dispatcher for batch operations only.
- Does this package support soft deletes (e.g., Gedmo’s SoftDeleteable) in batch operations?
- Yes, it works with soft deletes if your Doctrine entities include the `SoftDeleteable` behavior. However, ensure your query builder filters out soft-deleted entities explicitly (e.g., `WHERE deleted_at IS NULL`) to avoid processing them unintentionally. Test edge cases like partial batch failures during soft deletes.
- How do I handle failed batches in production (e.g., retries, rollbacks)?
- Use Doctrine transactions to wrap batches: failed batches will auto-rollback. For retries, implement a queue system (e.g., Laravel Queues) with exponential backoff. Log batch IDs and individual failures using `IdRangeBatcher::setLogger()` to debug issues. Consider compensating transactions for critical operations (e.g., refunds after failed order updates).
- Is this package compatible with Laravel 10 and PHP 8.2+?
- The package officially supports PHP 8.1+, but Laravel 10 (PHP 8.2+) compatibility is likely fine since it relies on Doctrine ORM’s stable APIs. Test with your specific Laravel version by checking for deprecation warnings or type errors. If issues arise, check the GitHub issues for patches or update the `doctrine/orm` dependency to a compatible version.
- Can I use this for real-time API bulk updates (e.g., PUT /users/batch) instead of async jobs?
- Yes, but monitor memory and response times closely. For real-time APIs, use smaller batches (e.g., 100–500 records) and stream responses incrementally. Avoid long-running transactions in web requests, as they can block HTTP workers. For high-throughput APIs, offload to queues or use Laravel’s `sync` queue driver for immediate feedback.
- How do I integrate this with Laravel’s caching (e.g., Redis) if batch operations modify data?
- Batch operations may invalidate cached data unpredictably. Use cache tags or versioned keys (e.g., `users:v2`) to scope invalidations. For critical caches, implement a post-batch invalidation hook (e.g., `Cache::forget('users')` after successful batches). Alternatively, bypass caching for bulk operations or use a read-through cache with short TTLs.
- What’s the difference between `IdRangeBatcher` and `ObjectCollectionBatcher`?
- `IdRangeBatcher` fetches entities by ID ranges (e.g., `WHERE id BETWEEN 100 AND 200`), ideal for async processing where you dispatch jobs by ID ranges. `ObjectCollectionBatcher` loads entities directly from a Doctrine query, useful for in-memory collections or smaller datasets. Choose `IdRangeBatcher` for scalability and `ObjectCollectionBatcher` for simplicity in synchronous workflows.
Weaver
How can I help you explore Laravel packages today?