Weave Code
Evaluation framework for AI agents built on the official Laravel AI SDK.
Frequently asked questions about Evals
- What is vizra/evals used for?
- Vizra Evals is an evaluation framework for AI agents built on the official Laravel AI SDK. It lets you write agent evals as Pest tests and keeps every result, including sampled scores, pass rates, judge reasoning, tool calls, and cost. Use it to hold a baseline, fail CI on regressions, and track quality trends in a dashboard.
- How do I install vizra/evals in a Laravel app?
- Run composer require vizra/evals --dev, then run php artisan migrate. You need PHP 8.4+, Laravel 12+, laravel/ai, and Pest 4 or 5 for the Pest testing surface. The README also mentions a standalone CLI option.
- Do I need Pest to use vizra/evals?
- Pest is the main testing surface: you write agent evals as Pest tests and use expectations such as toPassEval. The package requires Pest 4 on Laravel 12 or Pest 5 on Laravel 13 for that surface. The README also notes a standalone CLI exists, so you are not limited to Pest.
- Which PHP and Laravel versions does vizra/evals support?
- It requires PHP 8.4+ and Laravel 12.0+ according to the README badges. The Pest testing surface needs Pest 4 on Laravel 12 or Pest 5 on Laravel 13. You also need laravel/ai because the package targets agents built on the official Laravel AI SDK.
- Should I install vizra/evals with --dev?
- Use composer require vizra/evals --dev for normal development and CI reporting. Install it as a regular dependency instead if you use Vizra Cloud's Run button, because that relies on evals:runner in your app scheduler and composer install --no-dev would skip a dev requirement. CI reporting is fine either way because CI installs dev dependencies.
- How do I write my first eval with vizra/evals?
- Create a Pest test, for example under tests/Evals, and call toPassEval on your agent class. The README example uses expect(SupportBot::class)->toPassEval with a dataset from base_path('evals/support.jsonl'). You can then extend the eval chain with the scoring and dataset options shown in the docs.
- How does vizra/evals help fail CI on AI agent regressions?
- Vizra Evals records every run with sampled scores, pass rates, judge reasoning, tool calls, and cost. You can hold a baseline and compare new runs, then fail CI when quality regresses. That gives you regression checks beyond a one-off Pest pass or fail.
- Can I view vizra/evals results in a dashboard?
- Yes. Vizra Evals keeps every result and lets you watch quality trend over time in a dashboard; the README links to the separate vizra-evals-ui dashboard. You can also use Vizra Cloud. Run php artisan migrate so the package has storage for recorded runs.
- What are alternatives to vizra/evals?
- If you only need a pass or fail result for today's agent behavior, Pest with the Laravel AI SDK may be sufficient. Vizra Evals is for teams that also want stored run history, baselines, CI regression failures, and dashboard trends. Evaluate it against your existing Pest suite and any hosted eval tooling based on whether you need persistent records and cost tracking.
- Is vizra/evals maintained and what license does it use?
- Vizra Evals is published by Vizra under the MIT license. The README shows Packagist version and download badges and lists PHP 8.4+ and Laravel 12+ requirements. Check Packagist and the vizra-ai/vizra-evals GitHub repository for the latest release and maintenance activity.
Weaver
How can I help you explore Laravel packages today?