Laravel Bulk Upsert Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require lapaliv/laravel-bulk-upsert
   

2. Add Trait to Model:

   use Lapaliv\BulkUpsert\Bulkable;
   
   class User extends Model {
       use Bulkable;
   }
   

3. First Use Case:

   $data = [
       ['email' => 'john@example.com', 'name' => 'John'],
       ['email' => 'david@example.com', 'name' => 'David'],
   ];
   
   User::query()->bulk()->uniqueBy('email')->create($data);
   

Where to Look First

• README.md: Focus on the "Usage" section for quick examples.
• Events: Check the "Listeners" section for event-based workflows.
• API Reference: Use the provided API table for method signatures.

Implementation Patterns

Core Workflows

1. Bulk Creation with Events:

   $bulk = User::bulk()->onCreating(fn(User $user) => $user->setAttribute('active', true));
   $bulk->uniqueBy('email')->createAndReturn($data);
   

2. Chunked Processing:

   $bulk->chunk(100);
   foreach ($largeDataset as $item) {
       $bulk->createOrAccumulate($item);
   }
   $bulk->createAccumulated();
   

3. Upsert with Related Data:

   User::bulk()
       ->uniqueBy('email')
       ->upsert($usersWithComments)
       ->onSavedMany(fn($users, $bulkRows) => $this->handleComments($bulkRows));
   

Integration Tips

• Observers: Use savedMany to trigger cascading operations (e.g., creating related models).
• Validation: Attach validation logic in onCreating/onUpdating callbacks.
• Transactions: Wrap bulk operations in transactions for data consistency:

  DB::transaction(fn() => $bulk->upsert($data));
  

Advanced Patterns

• Dynamic Unique Keys:

  $bulk->uniqueBy(fn(array $item) => $item['email'] ?? $item['username']);
  

• Conditional Processing:

  $bulk->onCreating(fn(User $user) => $user->email === 'admin@example.com' ? false : null);
  

Gotchas and Tips

Pitfalls

1. Event Order:

   • onSaving runs before onCreating/onUpdating. Return false in onSaving to skip all subsequent events.
   • onCreatingMany/onUpdatingMany receive a BulkRows object; ensure you handle its structure.

2. Field Alignment:

   • The package auto-aligns fields, but explicit fillable attributes in your model override this. Ensure fillable matches your data structure.

3. Soft Deletes:

   • Soft deletes (delete) respect SoftDeletes trait, but forceDelete bypasses it. Verify your model’s deleted_at column.

4. Performance:

   • Large chunks (>1000 rows) may hit database limits. Monitor query performance with DB::enableQueryLog().

Debugging Tips

• Log Events:

  $bulk->onCreating(fn(User $user) => logger()->debug('Creating:', $user->toArray()));
  

• Inspect BulkRows:

  $bulk->onSavedMany(fn($users, $bulkRows) => logger()->debug($bulkRows->first()->original));
  

• Validate Data: Use onCreating to log or validate data before bulk operations:

  $bulk->onCreating(fn(User $user) => $user->email ? null : logger()->error('Missing email'));
  

Extension Points

1. Custom BulkRow: Extend Lapaliv\BulkUpsert\Entities\BulkRow to add metadata:

   class CustomBulkRow extends BulkRow {
       public function __construct(array $original, Model $model, array $unique) {
           parent::__construct($original, $model, $unique);
           $this->original['custom_field'] = 'value';
       }
   }
   

2. Override Query Builder: Replace the default query builder by binding a custom instance:

   $bulk = new Bulk(User::class, app()->make(CustomQueryBuilder::class));
   

3. Batch Processing: Use chunk() with a callback for async processing:

   $bulk->chunk(50, fn($chunk) => dispatch(new ProcessChunk($chunk)));
   

Configuration Quirks

• Chunk Size: Default is 100. Adjust via config(['bulk-upsert.chunk_size' => 200]).
• Database Drivers: SQLite may require explicit id handling due to auto-increment behavior.