Laravel Chunky Laravel Package

Getting Started

Install the backend package via Composer:

composer require netipar/laravel-chunky
php artisan vendor:publish --tag=chunky-config
php artisan migrate

For frontend integration, install the framework-specific package (e.g., Vue 3):

npm install @netipar/chunky-vue3

First use case: Upload a single file with progress tracking in Vue 3:

<script setup>
import { useChunkUpload } from '@netipar/chunky-vue3';

const { upload, progress, isUploading } = useChunkUpload();

function handleFileChange(event) {
    upload(event.target.files[0]);
}
</script>

<template>
    <input type="file" @change="handleFileChange" />
    <progress v-if="isUploading" :value="progress" max="100" />
</template>

Key next steps:

1. Configure middleware in chunky.routes.middleware (e.g., ['auth:sanctum'])
2. Set up a queue worker for AssembleFileJob
3. Listen to events like UploadCompleted in EventServiceProvider

Implementation Patterns

Backend Workflow

1. Initiate Upload: Frontend sends metadata → backend generates upload_id and returns chunking instructions.
2. Chunk Upload: Frontend splits file into chunks (default: 5MB) and uploads in parallel.
3. Progress Tracking: Backend validates chunks via SHA-256, updates progress in DB.
4. Assembly: On completion, AssembleFileJob merges chunks to storage/app/chunky/uploads/{uploadId}.
5. Events: Trigger custom logic via UploadCompleted, UploadFailed, etc.

Example Event Listener:

// app/Listeners/ProcessUploadedFile.php
public function handle(UploadCompleted $event) {
    $filePath = $event->getFinalPath();
    // Process file (e.g., generate thumbnail, store metadata)
}

Frontend Integration Patterns

Single File Upload (Vue 3)

<script setup>
const { upload, progress, isUploading, error } = useChunkUpload({
    maxConcurrent: 4, // Parallel chunks
    autoRetry: true,
    maxRetries: 3,
});

function handleUpload(file) {
    upload(file, { folder: 'user_uploads' }); // Custom folder
}
</script>

Batch Upload (React)

const {
    upload,
    progress,
    isUploading,
    completedFiles,
    totalFiles,
} = useBatchUpload({ maxConcurrentFiles: 2 });

function handleFiles(files) {
    upload(files, { context: 'documents' }); // Group files
}

Livewire Integration

<livewire:chunky-upload context="profile_avatar" />

• Listen to events in parent Livewire component:

#[On('chunky-upload-completed')]
public function handleUpload(array $data) {
    $this->emit('upload-success', $data['fileName']);
}

Queue and Job Patterns

• Run AssembleFileJob asynchronously (avoid sync driver):

  php artisan queue:work
  

• Customize job behavior by extending AssembleFileJob:

  namespace App\Jobs;
  
  use NETipar\Chunky\Jobs\AssembleFileJob as BaseJob;
  
  class CustomAssembleJob extends BaseJob {
      protected function assembleChunks() {
          // Override logic (e.g., validate file type before assembly)
      }
  }
  

Configuration Patterns

Dynamic Contexts

Use context to isolate uploads (e.g., by user or feature):

// Frontend
upload(file, { context: 'user_123_avatars' });

// Backend (config/chunky.php)
'contexts' => [
    'user_*_avatars' => [
        'disk' => 's3',
        'folder' => 'avatars',
    ],
],

Custom Storage Disks

Configure per-context disks in chunky.php:

'disks' => [
    's3' => 's3',
    'local' => 'local',
],

Gotchas and Tips

Pitfalls

1. Queue Stuck Jobs:

   • If AssembleFileJob fails silently, check failed_jobs table.
   • Fix: Retry with php artisan queue:retry.

2. Lock Contention:

   • 503 Service Unavailable during high traffic? Use Redis for chunky.lock_driver.
   • Fix: Set chunky.lock_driver = 'redis' in config.

3. Frontend Timeouts:

   • Large files may time out if chunk size is too big.
   • Fix: Reduce chunk_size in chunky.php (default: 5MB).

4. CSRF Issues:

   • If using custom auth, ensure X-XSRF-TOKEN is sent.
   • Fix: Override defaults:

     setDefaults({ headers: { 'X-CSRF-TOKEN': 'custom-token' } });
     

5. Memory Limits:

   • Assembling large files may hit PHP memory limits.
   • Fix: Increase memory_limit or use chunky.staging_directory on a high-memory server.

Debugging Tips

• Log Events:

  // config/chunky.php
  'logging' => [
      'enabled' => true,
      'channel' => 'single',
  ],
  

• Check Upload Status:

  php artisan chunky:status {uploadId}
  

• Clean Up Stale Uploads:

  php artisan chunky:cleanup --force
  

Extension Points

1. Custom Authorizer: Override ownership checks (e.g., for shared folders):

   // app/Providers/ChunkyServiceProvider.php
   $this->app->bind(
       \NETipar\Chunky\Contracts\Authorizer::class,
       \App\Services\CustomChunkyAuthorizer::class
   );
   

2. Pre-Assembly Validation: Extend AssembleFileJob to validate files before merging:

   protected function assembleChunks() {
       if (!in_array($this->fileName, ['jpg', 'png'])) {
           throw new \Exception('Invalid file type');
       }
       parent::assembleChunks();
   }
   

3. Broadcast Custom Data: Expose internal paths in broadcasts:

   // config/chunky.php
   'broadcasting' => [
       'expose_internal_paths' => true,
   ],
   

Performance Tips

• Batch Uploads: Limit max_files_per_batch to avoid DB bloat:

  // config/chunky.php
  'metadata' => [
      'max_keys' => 100,
  ],
  

• Chunk Size: Tune chunk_size based on network stability (smaller = more resilient):

  'chunk_size' => 2 * 1024 * 1024, // 2MB
  

• Parallelism: Adjust max_concurrent_chunks (default: 4) for faster uploads:

  useChunkUpload({ maxConcurrent: 6 });
  

Security Quirks

• File Name Sanitization: Frontend sends raw fileName; sanitize in UploadInitiated listener:

  public function handle(UploadInitiated $event) {
      $event->setFileName(strtolower($event->fileName));
  }
  

• Batch Limits: Prevent abuse by capping batch size:

  // config/chunky.php
  'batch' => [
      'max_files' => 10,
  ],
  

• Lock Expiry: Short-lived locks may cause race conditions. Adjust lock_ttl:

  'lock_ttl' => 300, // 5 minutes