Parallel Functions Laravel Package

Getting Started

Install the package via Composer:

composer require amphp/parallel-functions  

Start with parallelMap()—the most common entry point—to run I/O-bound or CPU-heavy operations in parallel. For example, concurrently fetching multiple URLs:

use function Amp\ParallelFunctions\parallelMap;  

$results = parallelMap($urls, fn($url) => file_get_contents($url));  

Verify upfront:

• PHP ≥ 8.1 (required for v2+ with fibers)
• All closures and captured variables must be serializable (no resources, generators, or non-serializable objects in use)
• Test worker spawning in your environment (e.g., CLI works; shared hosting often blocks proc_open)

Implementation Patterns

Batch Processing with parallelMap

Ideal for homogeneous tasks over collections:

$images = glob('uploads/*.jpg');  
$thumbnails = parallelMap($images, fn($file) => resizeImage($file, 200, 200));  

Custom Worker Pools

For long-running processes (e.g., queue workers), reuse a pool to avoid process thrashing:

use Amp\Parallel\Worker\DefaultPool;  

$pool = new DefaultPool(4);  
$results = parallelMap($tasks, $callable, $pool);  

Top-Level Functions for Clarity

Keep logic reusable and testable outside closures:

// app/Parallel/ThumbnailGenerator.php  
class ThumbnailGenerator {  
    public static function generate(string $file): string { /* ... */ }  
}  

// usage  
$results = parallelMap($files, [ThumbnailGenerator::class, 'generate']);  

Laravel-Specific Use Cases

• Use inside a single queued job to parallelize subtasks (e.g., process 100 notifications at once)
• Avoid spawning workers from queued jobs—workers are heavy; keep them at the entry point
• Pass config/data explicitly; don’t rely on Laravel’s service container (it doesn’t carry over to workers)

Gotchas and Tips

Serialization Landmines

• Closures capturing unserializable objects (e.g., PDO, Symfony\Component\HttpFoundation\Request) fail silently unless checked.
• Test with serialize($callable) early. Laravel’s SerializableClosure helps, but still fails for:
  ❌ Closures using yield/generators
  ❌ Closures referencing anonymous classes
  ❌ Objects without __sleep()/__wakeup() or non-serializable properties
• ✅ Safe: Static methods, named classes, plain closures with scalar/array use variables

Process Isolation Gotchas

• Workers are fresh processes—no autoloader cache, no global state, no environment variables (except $_ENV/$_SERVER['argv']).
  → Always require __DIR__ . '/vendor/autoload.php'; inside the parallel callable.
• PHP extensions in the parent process (e.g., imagick via FPM) may be missing in CLI workers.
  → Add startup checks: extension_loaded('imagick') || throw new \RuntimeException('imagick required');

Debugging & Reliability

• Exceptions thrown in workers are rethrown in the main process—wrap worker logic to log + rethrow:

  fn() => try { return heavyTask(); } catch (\Throwable $e) { error_log($e); throw $e; }  
  

• Disable Xdebug in production workers (XDEBUG_MODE=off); it causes >10x slowdown.
• Use pcntl_async_signals(true) + declare(ticks=1) if handling SIGTERM for graceful shutdowns.

Performance Tuning

• Worker spawn overhead is ~20–100ms; only parallelize tasks taking >50ms.
• OOM is common with too many workers—always cap concurrency:

  $pool = new DefaultPool(min(8, count($tasks))); // never let it scale to infinity  
  

• Prefer non-blocking I/O (amphp/http-client) before parallelization—even inside parallelMap, blocking calls bottleneck CPU.