Codeception Module Ai Reporter Laravel Package

Getting Started

• Install with composer require --dev webproject-xyz/codeception-module-ai-reporter (check the package name in composer.json), then enable the module in your Codeception suite config (typically tests/Unit.suite.yml, tests/Feature.suite.yml, or a shared codeception.dist.yml).
• Configure the AI provider credentials via environment variables rather than committing keys — most reporters of this type read an API key + model from _env placeholders in the suite YAML.
• First use case: run a failing test in a small suite, e.g. vendor/bin/codecept run Feature SomeTest::testSomething, and confirm the module emits an AI-generated explanation/report artefact alongside Codeception's normal output.
• Where to look first: the module's README config block (required vs optional keys), then src/ — as of 2.1.0 the failure-analysis logic lives in a dedicated FailureExtractor module, so that class is the best entry point for understanding what data is actually sent to the model.
• In a Laravel app, run this on the Codeception suites only; it is orthogonal to PHPUnit/Pest, so keep it out of CI jobs that don't use Codeception.

Implementation Patterns

• Config-layer setup, not test-layer: register the module once per suite. Avoid instantiating the reporter manually inside tests — it hooks into Codeception's event lifecycle (_before, _after, _failed) and expects to be driven by the runner.
• Failure-only invocation: the value of the package is post-failure triage. Keep it enabled only for local dev and nightly/PR jobs where you want an explanation, and disable it on fast feedback loops to save cost and latency.
• Wrap it in a CI step, not a blocking gate: emit the AI report as an artifact (log file or PR comment) rather than failing the build on the model's output. Model responses are non-deterministic and should never be the source of truth for pass/fail.
• Use it as a triage triage aid, not a fixer: the intended workflow is fail → read extracted failure context → jump to the relevant app/ or tests/ file yourself. Treat any suggested patch as a hint.
• Architecture-wise (2.1.0): the extraction of FailureExtractor into its own deep module means you can now reason about — and potentially reuse — the "turn a Codeception failure into structured context" step independently of the AI-calling step. If you have custom reporting needs, that boundary is the natural extension point (e.g. feeding extracted failures into your own Slack/Issue tracker pipeline instead of the bundled AI call).
• Pair with Laravel's own debugging: for HTTP/DB failures, Codeception's _failed event fires before Laravel teardown logs are flushed in some setups, so correlate the AI report with storage/logs/laravel.log and the Codeception _output/ directory.

Gotchas and Tips

• Not a Laravel package in the framework sense — there is no service provider, no facade, no config publish. All configuration is Codeception suite YAML (modules: enabled: - ... with config:). Don't go looking in config/.
• Secrets in suite YAML: if you hardcode an API key in *.suite.yml it gets committed. Use %ENV_VAR% substitution or .env loading in codeception.dist.yml.
• Cost and rate limits: every failure triggers a model call. A suite with 200 failing tests can produce 200 requests. Batch/limit where the module allows, or restrict to a tagged subset of tests.
• Network in CI: sandboxed CI runners and PRs from forks won't have API credentials. Guard with a conditional (skip the module or set a no-op provider) so forks don't fail on missing keys.
• Upgrading 2.0.1 → 2.1.0: the FailureExtractor extraction is an internal refactor, so public configuration keys and behaviour should be unchanged. Still, if you had extended or monkey-patched the module's failure-handling internals (subclassing, overriding _failed), re-check against 2.1.0 — internal method/property locations likely moved.
• Debugging empty reports: the most common cause is the extractor finding no useful context (e.g. an assertion failure with no stack frame in your code). Log the extracted structure separately from the AI response to distinguish "extraction produced nothing" from "model returned nothing" — the 2.1.0 refactor makes this easier to isolate.
• Don't trust the diagnosis blindly: with only a failure message and partial trace, the model will confidently misattribute causes in framework-heavy stacks. Always confirm against the actual stack trace before acting.