Csv Importer Laravel Package

Getting Started

The first thing to internalize: firefly-iii/csv-importer is a standalone Laravel application, not a Composer library you require into an existing project. You don't add it to a composer.json alongside your app — you run it as its own service that talks to a Firefly III instance over the REST API. Treat it that way from day one and most of the surprises disappear.

Minimal setup:

1. Stand up Firefly III first. The importer is useless without it. Create an OAuth client (Profile → OAuth → Create New Client) and generate a Personal Access Token. Save the token.
2. Run the importer as a container. Typical invocation:

   docker run -d --name csv-importer \
     -p 8081:8080 \
     -e FIREFLY_III_URL=https://firefly.example.com \
     -e FIREFLY_III_ACCESS_TOKEN=<token> \
     -e APP_KEY=base64:... \
     fireflyiii/csv-importer:latest
   

   If you're building from source, composer install && cp .env.example .env && php artisan key:generate && php artisan serve works, but the container is the supported path.
3. Confirm reachability before importing anything. Hit https://firefly.example.com/api/v1/about with the token via curl. If that fails, the importer will fail in opaque ways later.
4. First use case: upload a single small CSV through the web UI at http://localhost:8081. The UI is the fastest way to learn the column-mapping model without writing config by hand.

Where to look first in the repo: routes/, app/Http/Controllers/, and the import command under app/Console/Commands/. The mapping logic lives in app/Services/ — that's where the CSV → Firefly III transaction model happens.

Implementation Patterns

Web-first mapping, CLI-first execution. Design your workflow so the web UI is used once to produce a mapping config, then the CLI is used forever after. This is the intended happy path and keeps you out of trial-and-error territory.

Version-control your mapping configs. The importer persists mapping definitions (typically JSON) under storage/app/. Copy these into your repo and mount or symlink them back into the container. A mapping config is code: it encodes field transformations, date formats, and account resolution rules. When a bank changes its export format, you want to see that as a diff, not discover it in a failed import.

CLI import for automation. The real leverage is scheduled imports:

docker exec csv-importer php artisan importer:import /path/to/file.csv

Wire this into cron/systemd timers. Combine with a step that first pulls the CSV from wherever your bank drops it (email attachment, S3, etc.).

Routine imports. For recurring bank feeds, use the importer's routine concept — define a routine that pairs a file location with a mapping, then trigger it on a schedule. This is the "set and forget" pattern and is what makes the tool genuinely useful day-to-day rather than a one-off tool.

Token rotation as a first-class concern. Personal access tokens in Firefly III can expire. Store the token in your secret manager, not in a compose file that gets committed. When the token rotates, the importer container must be recreated (env vars are not hot-reloaded).

Idempotency via Firefly III's duplicate detection. Rather than deduplicating CSVs yourself, rely on Firefly III's built-in detection keyed on date + amount + description. Re-run the same file twice and verify nothing is duplicated — if it is, your column mapping is probably dropping or mangling the description.

Integration pattern: pre-process, then import. The importer is deliberately not a data-cleaning tool. Do your concatenation, encoding fixes, and column trimming with a small script (Python/awk) upstream, then hand the importer a clean, stable CSV. This keeps the mapping config tiny and the failure surface small.

Gotchas and Tips

• Don't composer require it. This is the single most common misunderstanding. There is no meaningful library API surface — its output is HTTP calls to Firefly III, not PHP methods you call.
• Version-pin the image to your Firefly III version. The API contract is mostly stable but not perfectly. latest on both sides is a recipe for a silent schema mismatch after an update.
• CSV encoding bites first. Byte-order marks, Windows-1252 encodings, and trailing whitespace in headers are the top causes of "column not found." Run file -i your.csv and head -c 3 your.csv | xxd before debugging the importer itself.
• Date format ambiguity is silent. 03/04/2024 will import — as the wrong date — without erroring. Always set the explicit format in the mapping and verify with a small sample file whose true dates you know.
• Debug via the Firefly III side. When an import "succeeds" but nothing appears, check Firefly III's own logs and its webhooks/duplicate rules. The importer's logs will tell you the HTTP call succeeded; they won't tell you Firefly III rejected a transaction due to a rule.
• Amount sign conventions. Different banks flip debit/credit signs. The mapping UI supports negation and multiplier transforms — use them explicitly rather than assuming the importer infers direction from a separate "type" column.
• Account resolution is strict. If an account name in the CSV doesn't match an existing Firefly III asset account (whitespace and case included), the transaction lands in the wrong place or fails. Establish a canonical account list and normalize upstream.
• Extension point: custom field mappings. The mapping config supports custom column expressions. If a bank gives you a combined "Description" with memo appended, this is where you split it rather than post-processing after import.
• Container state is ephemeral by default. Mapping configs, uploaded CSVs, and logs live inside the container unless you mount volumes. Mount storage/ or you'll lose your carefully-built mappings on the next docker rm.
• No built-in retry semantics. Scheduled imports that hit a network blip will simply fail. Wrap your cron invocation in a retry loop and alert on repeated failure — the importer itself won't email you.