Parallel Downloader Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require amarkhai/parallel-downloader-bundle
   

   Ensure config/bundles.php includes Amarkhai\ParallelDownloaderBundle\ParallelDownloaderBundle::class.

2. First Use Case Inject DownloadManager into a service/controller and call download() with an associative array of filename => url:

   use Amarkhai\ParallelDownloaderBundle\Service\DownloadManager;
   
   public function downloadFiles(DownloadManager $downloadManager)
   {
       $downloadManager->download([
           'report.pdf' => 'https://example.com/report.pdf',
           'image.jpg' => 'https://example.com/image.jpg',
       ]);
   }
   

   Files will save to var/downloads/ by default.

Implementation Patterns

Workflows

1. Batch Processing Use for downloading multiple files (e.g., user uploads, API responses):

   $urls = $this->fetchFileUrlsFromDatabase();
   $this->downloadManager->download($urls);
   

2. Subfolders Organize downloads by category:

   $this->downloadManager->download($urls, 'reports/2023');
   

3. Custom Guzzle Options Apply per-request settings (timeouts, headers, etc.):

   $options = ['timeout' => 30, 'headers' => ['User-Agent' => 'MyApp/1.0']];
   $this->downloadManager->download($urls, null, $options);
   

Integration Tips

• Event Listeners: Extend functionality by listening to download.start/download.end events (if supported).
• Progress Tracking: Log progress via Symfony’s LoggerInterface (inject into DownloadManager if needed).
• Queue Integration: Combine with Symfony Messenger for async downloads:

  $this->messageBus->dispatch(new DownloadFilesMessage($urls));
  

Gotchas and Tips

Pitfalls

1. Concurrency Limits Default download_concurrency: 10 may overwhelm servers. Adjust in config or dynamically:

   # config/packages/amarkhai_parallel_downloader.yaml
   parameters:
     amarkhai_parallel_downloader.download_concurrency: 5
   

2. File Overwrites Duplicate filenames in the input array will overwrite silently. Validate keys first:

   if (count(array_unique(array_keys($urls))) !== count($urls)) {
       throw new \InvalidArgumentException('Duplicate filenames detected.');
   }
   

3. Retry Logic Retries (default: 3) may hide transient issues. Monitor logs for failed URLs.

Debugging

• Verify Paths: Ensure download_files_folder is writable:

  chmod -R 775 var/downloads
  

• Guzzle Errors: Check options for on_error callbacks or enable Guzzle’s debug handler.

Extension Points

1. Custom Storage Override DownloadManager to use S3, database storage, etc.:

   class CustomDownloadManager extends DownloadManager {
       protected function saveFile(string $content, string $path): void {
           // Custom logic (e.g., upload to S3)
       }
   }
   

2. Pre/Post-Processing Hook into download() via dependency injection:

   $this->downloadManager->download($urls, null, $options, $this);
   

   Implement PostDownloadInterface for callbacks.

3. Config Overrides Use environment variables for dynamic settings:

   parameters:
     amarkhai_parallel_downloader.download_concurrency: '%env(int:DOWNLOAD_CONCURRENCY)%'