Laravel Bulk Upsert Laravel Package

Getting Started

Minimal Setup

1. Install the package:

   composer require lapaliv/laravel-bulk-upsert
   

2. Add the Bulkable trait to your Eloquent model:

   use Lapaliv\BulkUpsert\Bulkable;
   
   class User extends Model {
       use Bulkable;
   }
   

First Use Case: Bulk Create

$data = [
    ['email' => '[email protected]', 'name' => 'John'],
    ['email' => '[email protected]', 'name' => 'David'],
];

User::query()->bulk()->uniqueBy('email')->create($data);

First Use Case: Bulk Upsert with Return

$data = [
    ['email' => '[email protected]', 'name' => 'Jacob'],
    ['id' => 1, 'email' => '[email protected]', 'name' => 'Oscar'],
];

$users = User::query()->bulk()->uniqueBy(['email'])->upsertAndReturn($data);

Implementation Patterns

Common Workflows

1. Bulk Create with Chunking

$bulk = User::query()->bulk()->uniqueBy('email')->chunk(100);

foreach ($data as $item) {
    $bulk->createOrAccumulate($item);
}

$bulk->createAccumulated();

2. Bulk Update with Query Filtering

User::query()
    ->whereIn('id', [1, 2, 3, 4])
    ->selectAndUpdateMany(['role' => null]);

3. Event-Driven Processing

User::query()->bulk()
    ->onCreating(fn(User $user) => {
        // Pre-create logic
    })
    ->onCreated(fn(User $user) => {
        // Post-create logic
    })
    ->upsert($data);

4. Accumulating and Flushing

$bulk = User::query()->bulk()->chunk(50);

foreach ($data as $item) {
    $bulk->upsertOrAccumulate($item);
}

// Flush remaining items
$bulk->upsertAccumulated();

Integration Tips

With Observers

// app/Observers/UserObserver.php
public function creatingMany(Collection $users, BulkRows $bulkRows) {
    $bulkRows->each(fn(BulkRow $row) => {
        // Custom logic per row
    });
}

With API Requests

// Handle bulk payloads from API
$bulk = User::query()->bulk()->uniqueBy('email');
$bulk->onCreatingMany(fn(Collection $users) => {
    $this->validateBulkData($users);
});
$bulk->upsert($request->input('users'));

With Transactions

DB::transaction(function () use ($bulk, $data) {
    $bulk->upsert($data);
    // Additional transactional operations
});

Gotchas and Tips

Pitfalls

1. Unique Key Mismatch

   • Ensure uniqueBy() matches the actual unique constraint in your database.
   • Example: uniqueBy(['email', 'status']) must align with a composite unique index.

2. Event Order Confusion

   • onSaving fires before onCreating/onUpdating. Return false to skip operations.
   • Example:

     ->onSaving(fn(User $user) => {
         if ($user->email === '[email protected]') {
             return false; // Skip this record
         }
     })
     

3. Chunk Size Too Small

   • Small chunks (e.g., chunk(1)) defeat the purpose of bulk operations.
   • Tip: Use chunk(100) or higher for optimal performance.

4. Missing Fillable Fields

   • Ensure all fields in $data are in $fillable or use massAssign:

     protected $fillable = ['email', 'name', 'custom_field'];
     

5. Observer Conflicts

   • Avoid duplicate observers for the same event (e.g., creating and onCreating).

Debugging Tips

1. Log BulkRows

   ->onCreatingMany(fn(Collection $users, BulkRows $bulkRows) => {
       \Log::debug($bulkRows->toArray());
   })
   

2. Check SQL Queries

   • Enable Laravel query logging:

     DB::enableQueryLog();
     $bulk->upsert($data);
     \Log::debug(DB::getQueryLog());
     

3. Validate Data Before Bulk Operations

   ->onSavingMany(fn(Collection $users, BulkRows $bulkRows) => {
       $bulkRows->reject(fn(BulkRow $row) => !valid($row->original));
   })
   

Extension Points

1. Custom BulkRow Processing

   • Extend Lapaliv\BulkUpsert\Entities\BulkRow for additional metadata:

     class CustomBulkRow extends BulkRow {
         public function getExtraData() { ... }
     }
     

2. Dynamic Unique Keys

   • Use closures for dynamic uniqueBy:

     ->uniqueBy(fn(BulkRow $row) => [$row->original['email'], $row->original['tenant_id']])
     

3. Bulk Soft Deletes with Conditions

   ->onDeletingMany(fn(Collection $users, BulkRows $bulkRows) => {
       $bulkRows->filter(fn(BulkRow $row) => $row->model->isActive);
   })
   

4. Post-Upsert Hooks

   • Use onSavedMany to trigger side effects (e.g., webhooks, notifications):

     ->onSavedMany(fn(Collection $users) => {
         Notification::send($users, new UserCreated());
     })
     

Configuration Quirks

• Default Chunk Size: No default; always specify chunk(N) explicitly.
• Event Priority: Library events (*Many) fire after single-model events.
• Soft Deletes: Requires Illuminate\Database\Eloquent\SoftDeletes trait on the model.