Data Importer Laravel Package

Getting Started

The single most important thing to internalise first: firefly-iii/data-importer is a Laravel application, not a Composer library. There is no composer require story, no service provider to register, no facade. It ships as a Docker image (fireflyiii/data-importer) that runs next to a Firefly III instance and talks to it over the REST API. So your "day-to-day" relationship with it is either (a) operating it as a sidecar service, or (b) forking/cloning it to extend the import providers.

Minimal setup

1. Have Firefly III running and reachable at some URL.
2. In Firefly III: Profile → OAuth → Personal Access Tokens → create a token. Note the token and the Firefly III base URL.
3. Generate an APP_KEY (php artisan key:generate --show, or any base64:32 random string) — this app is stateful and will break without a stable key.
4. Run it:

services:
  firefly:
    image: fireflyiii/core:latest
    # ...
  importer:
    image: fireflyiii/data-importer:latest
    environment:
      APP_KEY: "base64:..."
      APP_URL: "http://localhost:8081"
      FIREFLY_III_URL: "http://firefly:8080"     # service name, NOT localhost
      FIREFLY_III_ACCESS_TOKEN: "eyJ0eXAiOiJK..."
    ports: ["8081:8080"]
    volumes: ["./importer-storage:/var/www/html/storage"]

Where to look first

• docker-compose.yml and .env.example in the repo root — authoritative list of env vars; the README's config table is often behind it.
• routes/web.php — the whole app is a handful of wizard-style routes (/import/upload, /import/configure, /import/run, /import/runs). Reading them tells you the flow faster than any doc.
• app/Http/Controllers/ (import controllers) — the orchestration layer: upload → column mapping → "configuration" JSON → run.
• The Import/-style service namespace under app/ — where CSV row parsing, duplicate hashing, and API payload building actually live.

First use case: import one bank CSV. Upload the statement, let the wizard guess the delimiter/date format, map date_description_amount style columns to Firefly III fields, run it, then open Firefly III and verify the transactions landed. Then immediately go back and export/save the mapping as a reusable configuration — that is the feature that turns this from a toy into something scriptable.

Implementation Patterns

Sidecar + shared network. The importer authenticates to Firefly III server-to-server. In Compose, point FIREFLY_III_URL at the service name and never expose Firefly III's port to the importer via the host loopback. Only the importer needs to be reachable by a human browser.

PAT vs OAuth client — pick deliberately. A Personal Access Token is bound to one Firefly III user and one default administration. Everything imported lands there. If you support multiple users/administrations, configure FIREFLY_III_CLIENT_ID and let users go through the OAuth redirect flow instead; the OAuth callback URL is derived from APP_URL, so that must be externally correct.

CSV configuration JSON is your automation surface. The wizard's column mapping is serialisable. Persist that JSON in your own repo (or generate it from a template) and reuse it per-bank. This is how you make imports repeatable instead of re-clicking through a UI every month. Combine it with IMPORT_DIR_ALLOWLIST so the importer can read files from a mounted directory rather than requiring an upload — then a cron job that drops statement_2026_10.csv into that volume plus a scheduled run gets you 90% of "automated banking sync".

Duplicate detection via hashes is the idempotency mechanism. Each row gets a hash over the fields you choose in the configuration. Re-running the same file is safe only if the mapping and hash inputs are unchanged. Design your workflow around "same file, same config" rather than "fix the config and re-run".

Use Firefly III webhooks for the downstream work. The importer is deliberately dumb — it creates transactions and gets out of the way. If you need enrichment (categories, budgets, tags, notifications, syncing to your own Laravel app), attach a Firefly III webhook on transaction creation and handle it in your application. Don't fork the importer for business logic.

Forking is a supported-but-heavy path. Adding a new bank connector or a bespoke CSV dialect means cloning the repo, implementing the provider alongside the existing ones, and building your own image. Budget for keeping it rebased against upstream, because Firefly III's API surface shifts between major versions.

Testing and local dev: it's a normal Laravel app — composer install, set .env, php artisan serve, php artisan test. Use SQLite or MySQL for the importer's own (thin) state: sessions, import runs, cached tokens. The develop image tag is useful for reproducing bugs against main, but pin a release tag in anything real.

Gotchas and Tips

• Don't composer require it into an existing app. It registers its own routes, config, and session handling; you'll get route collisions and confusing auth behaviour. Deployment boundary = process boundary.
• APP_KEY loss is a hard reset. Sessions and encrypted state die; users get logged out mid-import. Store it in your secrets manager, not in the compose file.
• Persist /var/www/html/storage. Uploaded files, logs, and the framework cache live there. Container restarts without a volume produce "my import run disappeared" bugs.
• localhost inside the container is the container. The #1 setup failure is FIREFLY_III_URL=http://localhost:8080. Use the Compose service name or a real hostname.
• Reverse proxies need TRUSTED_PROXIES and a correct APP_URL. Otherwise OAuth redirects go to http:// and cookies get dropped. It also does not enjoy being served from a sub-path; give it its own hostname.
• PHP upload limits bite before Laravel does. A year of transactions in one CSV can exceed upload_max_filesize/post_max_size, or blow memory_limit and the request timeout, because the run happens inside the web request. Split large statements, or raise limits on the importer container specifically.
• Date format and decimal separator are the top mapping failures. A European CSV (31-12-2026, 1.234,56) mapped as US will silently produce garbage amounts or drop rows. Pin both in the saved configuration rather than trusting the auto-guess.
• Changing the mapping invalidates duplicate hashes. Users re-import "the same file with a fix" and get duplicates, then blame the tool. Verify the hash inputs before re-running; check the run's log page for skipped-duplicate counts as your sanity signal.
• Open-banking connectors are rate-limited and consent-expiring. GoCardless/Nordigen free tier allows a small number of fetches per account per day and requires re-consent roughly every 90 days. A "daily sync" cron that works in testing will get 429s in production — schedule conservatively and handle the consent-expired state in your runbook.
• Version coupling with Firefly III. The importer pins expectations about Firefly III's API. Upgrade both together, read both release notes, and don't run importer:latest against firefly:3.x from two years ago.
• Debugging path: docker compose logs -f importer, then storage/logs/laravel.log. APP_DEBUG=true locally only — the stack traces include your Firefly III token.
• AGPL-3.0 matters here. If you fork it and expose the modified version as a network service to others, you must publish your changes. Internal-only use is fine; "white-label importer for our SaaS customers" is not, without releasing source.
• Extension points are the CSV configuration and the provider classes — in that order of cost. Reach for a config preset first; write a provider only when the format genuinely can't be expressed as a CSV mapping, and write a Firefly III webhook consumer before you consider touching this codebase for post-processing.