- How do I install marquine/php-etl in a Laravel application?
- Install it with Composer using composer require marquine/php-etl. Because the package is framework-agnostic, no Laravel service provider registration is typically required; check the README for optional configuration. Then define a pipeline and run it from a console command or queued job.
- Does marquine/php-etl support streaming for large CSV files?
- Yes, the library uses generator-based streaming so rows flow through extract, transform and load stages without loading the whole file into memory. That only holds if every stage stays lazy; transformers that sort, group or validate the full dataset will materialize data and increase memory use. For large files, keep transformations row-level and use chunked or batched loaders.
- Which PHP and Laravel versions are supported by marquine/php-etl?
- The package is Composer-installed and framework-agnostic, so it can be used in plain PHP and Laravel applications. The exact PHP and Laravel compatibility matrix is not stated in the overview, so check the repository's composer.json and README before committing. For long-lived projects, verify support against your target PHP version and plan a fork-and-maintain contingency if needed.
- How do I create a custom extractor, transformer or loader with marquine/php-etl?
- You build pipelines from extractors, transformers and loaders, and the API is designed to be extended. Implement your own classes for custom sources, row transformations or destination writes, then chain them with the fluent builder. Keep custom transformers lazy and avoid side effects so the pipeline remains streamable.
- Can I run marquine/php-etl inside a Laravel queue job or scheduler?
- Yes, the recommended Laravel integration is to invoke pipelines from console commands for scheduled runs and from queued jobs for user-triggered or chunked work. The library itself does not schedule, retry or orchestrate anything, so use Laravel's scheduler, queues, Horizon and WithoutOverlapping primitives around it. Design jobs to be idempotent and resumable if you need safe retries.
- Is marquine/php-etl good for CSV imports and nightly exports in Laravel?
- It fits bounded data movement well: CSV or XLSX imports, partner feed ingestion, nightly exports and one-off data migrations. It is not a scheduling or orchestration platform, so pair it with Laravel commands, queues and monitoring. For very large or wide files, benchmark memory and time on your own corpus first.
- How does marquine/php-etl handle errors and partial loads?
- Error handling and row-level policies need to be designed around the pipeline rather than assumed. The library does not provide production-grade metrics, dead-letter capture or structured logging out of the box, so add those in your command or job. For partial loads, define transaction boundaries, idempotency keys and a resume strategy before going live.
- Is marquine/php-etl actively maintained and safe for production?
- The project is MIT licensed and small in dependency size, which makes it easy to fork and maintain privately. The public repository has a modest community and appears to be single-maintainer, so bus factor and long-term support are the main risks. Treat it as a well-shaped primitive, run a time-boxed spike, and add your own hardening and characterization tests before relying on it in production.
- How does marquine/php-etl compare to hand-rolling ETL or using a platform like Airflow?
- Hand-rolled ETL often starts simple and becomes an unmaintained mess of loops, so a small chainable library can improve readability and reuse. Platforms like Airflow or dbt solve scheduling, orchestration and observability, which this package deliberately does not. Use marquine/php-etl as the in-process transformation engine inside Laravel and keep orchestration in Laravel queues or your platform of choice.
- How do I test an ETL pipeline built with marquine/php-etl?
- Test pipelines by running them against small fixture files and asserting the output of each extractor, transformer and loader. Add characterization tests on your real worst-case files, including malformed CSV quoting, encodings and header drift. For integration tests, use an in-memory or disposable database and verify idempotency and partial-load behavior.