Flysystem Cached Extra Laravel Package

Technical Evaluation

Architecture Fit

• Performance Optimization: The package extends Laravel’s Flysystem caching layer with traits like TemporaryFile, FileLocking, and Streaming, enabling micro-optimizations for:
  • Temporary file handling: Ideal for job payloads, uploads, or ephemeral storage (e.g., Storage::put('temp/job-123.json', $data)).
  • Concurrent writes: FileLockingTrait prevents race conditions in multi-process environments (e.g., Horizon workers).
  • Streaming: StreamingTrait reduces memory usage for large files (e.g., video processing pipelines).
• Laravel Synergy:
  • Integrates seamlessly with Laravel’s Storage facade and CachedAdapter, requiring minimal refactoring.
  • Supports Laravel’s event system (e.g., storage:linked) and queue workers (e.g., delayed jobs with locked files).
• Anti-Patterns:
  • Not a replacement for core Flysystem or Laravel’s Filesystem. Best used as a layered extension.
  • Memory constraints: Cached adapters hold files in memory; avoid for large binaries (>100MB) without TTL tuning.
  • Overhead for static files: Unnecessary for CDN-cached or immutable assets (e.g., public/ assets).

Integration Feasibility

• Dependencies:
  • Requires league/flysystem (≥3.0) and league/flysystem-cached-adapter (≥1.1).
  • No Laravel-specific dependencies (pure PHP), but assumes Laravel’s Storage facade or Flysystem-compatible setup.
• Configuration:
  • Minimal setup: Extend CachedAdapter with traits via DI or facade overrides.
  • Example:

    Storage::extend('cached_temp', function ($app) {
        $adapter = new CachedAdapter(
            Storage::disk('s3')->getAdapter(),
            new TemporaryFileTrait($app['cache.store'])
        );
        return new Filesystem($adapter);
    });
    

• Testing:
  • Unit-testable with Flysystem’s mock adapters (e.g., InMemoryAdapter).
  • Edge cases: Test file locks under concurrency (e.g., php-cs-fixer parallel runs) and cache eviction policies.

Technical Risk

Key Questions

1. Performance Justification:
   • What % of files are read/written repeatedly? (Cache hit ratio target: >70% for ROI.)
   • Are there hot/cold file patterns (e.g., logs vs. media)?
2. Concurrency Model:
   • How many processes/jobs will access locked files simultaneously?
   • Is FileLockingTrait’s flock-based locking sufficient, or needed Redis/Database-backed locks?
3. Persistence:
   • How are cached files evicted? (TTL vs. manual clear() calls.)
   • What’s the fallback if the cache layer fails (e.g., direct Storage::disk('s3'))?
4. Observability:
   • Are there metrics for cache hit/miss rates or lock contention?
   • How will errors (e.g., CacheException) be surfaced (Sentry, logs, Telescope)?
5. Scaling:
   • Will the cache layer (e.g., file driver) become a bottleneck under load?
   • How will temporary files scale with job queue depth?

Integration Approach

Stack Fit

• Laravel Ecosystem:
  • Filesystem: Replace local/s3 disks with cached_temp for ephemeral or high-frequency files.
  • Queues: Use TemporaryFileTrait for job payloads (e.g., Storage::put('temp/job-123.json', $data)).
  • Horizon: Monitor FileLockingTrait usage in delayed jobs.
• Non-Laravel PHP:
  • Works with any flysystem-compatible app (e.g., Symfony, Lumen) via direct adapter injection.
• Anti-Fit:
  • Avoid for read-heavy, immutable files (e.g., static assets). Overhead outweighs benefits.
  • Not suitable for filesystem-agnostic apps (e.g., CLI tools) without Laravel’s Storage facade.

Migration Path

1. Phase 1: Proof of Concept (1–2 weeks)
   • Scope: Replace a single disk (e.g., local) with cached_temp for a non-critical endpoint (e.g., user uploads).
   • Validation:
     • Measure cache hit/miss rates (add Storage::disk()->getAdapter()->getCache()->stats()).
     • Verify no regressions in file operations (e.g., copy(), delete()).
   • Tools: Use telescope to log storage events and laravel-debugbar to inspect adapters.
2. Phase 2: Incremental Rollout (2–4 weeks)
   • Temporary Files: Migrate job payloads or uploads to TemporaryFileTrait.
     • Example: Replace Storage::put('uploads/temp-'.$jobId.'.json', $data) with cached_temp disk.
   • Locking: Enable FileLockingTrait for critical sections (e.g., invoice generation).
     • Test with php-cs-fixer --parallel to simulate concurrency.
   • Streaming: Use StreamingTrait for large file uploads (e.g., video processing).
3. Phase 3: Full Adoption (1–2 weeks)
   • Replace all CachedAdapter usages with flysystem-cached-extra traits.
   • Deprecate custom caching logic (e.g., Redis-backed file locks).
   • Update documentation and runbooks for support teams.

Compatibility

• Backward Compatibility:
  • Drop-in replacement for league/flysystem-cached-adapter (same method signatures).
  • Traits are opt-in; existing code remains unchanged.
• Forward Compatibility:
  • Follows flysystem’s semantic versioning. Test against flysystem 4.0+ if upgrading.
• Breaking Changes:
  • None in v1.x. Future versions may add StreamingTrait config options or deprecate flock-based locking in favor of distributed locks.

Sequencing

1. Dependency Updates:
   • Pin league/flysystem and league/flysystem-cached-adapter to compatible versions.
   • Example composer.json:

     "require": {
         "league/flysystem": "^3.0",
         "league/flysystem-cached-adapter": "^1.1",
         "nao-pon/flysystem-cached-extra": "^1.0",
         "symfony/cache": "^5.0" // For Psr6CacheStorage
     }
     

2. Configuration:
   • Extend Laravel’s filesystems.php:

     'disks' => [
         'cached_temp' => [
             'driver' => 'custom',
             'adapter' => NaoPon\CachedExtra\CachedAdapter::class,
             'source' => 's3',
             'cache' => env('CACHE_DRIVER', 'file'), // 'redis', 'array', or 'file'
             'traits' => [
                 NaoPon\CachedExtra\Traits\TemporaryFileTrait::class,
                 NaoPon\CachedExtra\Traits\FileLockingTrait::class,
             ],
             'options' => [
                 'cacheTTL' => 3600, // 1 hour
                 'lockTimeout' => 30, // seconds
             ],
         ],
     ],
     

3. Service Provider:
   • Bind the custom adapter in AppServiceProvider:

     public function register()
     {
         Storage::extend('cached_temp', function ($app) {
             $adapter = new CachedAdapter(
                 Storage::disk('s3')->getAdapter(),
                 new Psr6CacheStorage($app['cache.store'])
             );
             $adapter->useTrait(new TemporaryFileTrait());
             $adapter->useTrait(new FileLockingTrait());
             return new Filesystem($adapter);
         });