Json Schema Test Suite Laravel Package

Getting Started

This is not a PHP library — it's a language-agnostic folder of JSON fixtures published by the JSON Schema org. You don't composer require it; you vendor it.

1. Add it as a git submodule (or a pinned tarball) at tests/fixtures/json-schema-test-suite:

   git submodule add https://github.com/json-schema-org/json-schema-test-suite.git tests/fixtures/json-schema-test-suite
   

2. Open tests/draft7/ (or draft2020-12/) first. Each file (type.json, enum.json, $ref.json, …) is an array of cases, each with description, schema, and a tests array of {description, data, valid}.
3. Look at remotes/ next — it contains the documents your $refs resolve to, keyed by http://localhost:1234/....
4. First use case: a PHPUnit test that asserts your validator agrees with the suite's valid flag.

// tests/Conformance/Draft7Test.php
use PHPUnit\Framework\Attributes\DataProvider;
use Opis\JsonSchema\Validator;

class Draft7Test extends TestCase
{
    public static function cases(): iterable
    {
        $dir = base_path('tests/fixtures/json-schema-test-suite/tests/draft7');
        foreach (glob($dir.'/*.json') as $file) {
            $group = json_decode(file_get_contents($file), true, 512, JSON_THROW_ON_ERROR);
            foreach ($group as $case) {
                foreach ($case['tests'] as $test) {
                    yield basename($file).' › '.$case['description'].' › '.$test['description'] => [
                        $case['schema'], $test['data'], $test['valid'],
                    ];
                }
            }
        }
    }

    #[DataProvider('cases')]
    public function test_conformance(mixed $schema, mixed $data, bool $valid): void
    {
        $result = (new Validator())->validate($data, json_encode($schema));
        $this->assertSame($valid, $result->isValid());
    }
}

Serve the remotes while running locally so $ref cases behave:

php -S 127.0.0.1:1234 -t tests/fixtures/json-schema-test-suite/remotes

Implementation Patterns

• Wrap the validator, not the test. Expose one method assertValid(mixed $schema, mixed $data): bool in a small SchemaConformance adapter. Then swapping justinrainbow/json-schema ↔ opis/json-schema ↔ swaggest/php-json-schema is a one-file change, and pass rates become directly comparable.
• Adopt-before-adopt. Before committing to a validator for an OpenAPI 3.1 spec or a FormRequest rule, run the suite against it; the pass count per draft is the real feature matrix.
• Draft matrix. Put draft-07 and 2020-12 in separate test classes. Keyword semantics changed (id→$id, definitions→$defs, dependencies split, items array form removed). One suite, one draft.
• Filter by keyword in CI. Run the full suite nightly; on PRs run --filter='type.json|enum.json|required.json' against the files your app actually touches. Full suite in ~seconds still beats a broken deploy, but this keeps the signal high.
• Explicit skip list. Keep tests/Conformance/skips.php mapping file › case › test → reason. Assert in a meta-test that every skip key still exists in the suite, so stale skips don't mask regressions after an upgrade.
• Remote resolver. In opis, register a custom SchemaLoader/resolver mapping http://localhost:1234/ to remotes/; in justinrainbow, a RefResolver with a retrieval callback. Don't rely on the dev server in CI.
• Use it as executable documentation. Ambiguous spec questions ("does exclusiveMinimum require minimum in draft-07?") are answered by the case descriptions. Faster than re-reading the spec.
• Reuse the harness for OpenAPI. OpenAPI 3.1's schema dialect is JSON Schema 2020-12, so the same conformance test doubles as a guard for your spec-validation pipeline.
• CI wiring. composer test:conformance script separate from composer test; mark it @group conformance so it can be excluded from the fast local loop.

Gotchas and Tips

• No Packagist package, no semver. The last release tag is 2023-05-12 but master keeps moving; pin a commit in your submodule and bump deliberately. Renamed and added tests will appear as failures that aren't your validator's fault.
• $ref remotes are localhost URLs. Without a resolver you'll get failures that look like $ref bugs. This is the #1 time sink.
• Optional tests need opt-in. Anything under optional/ (format assertion, content encoding, ecmascript-regex, zeroTerminatedFloats) assumes a feature your validator may legitimately not implement. Skip them by name, never by accident.
• Boolean schemas. true/false are valid schemas from draft-06 onward. Several PHP validators reject non-object schemas — that's a real bug, not a test artifact.
• Numeric edge cases. 1.0 vs 1, integers beyond PHP_INT_MAX, multipleOf float precision. A handful of legitimate disagreements here is normal; document them rather than chasing them.
• Unicode in descriptions and test data. Use JSON_THROW_ON_ERROR; emoji in case names break some data-provider generators and terminal output.
• Resource limits. Some cases are deliberately deep or large. PHP's recursion and memory_limit will blow up before the assertion does; cap iterations and fail loudly instead of hanging CI.
• Descriptions are not globally unique. Key skip lists and failure dumps by file › case › test, not by description alone.
• Assert booleans only. The suite specifies validity, not error messages or paths (output-tests/ is for people implementing output formats). Don't couple tests to your validator's error text.
• Debugging a red case. Dump ['schema' => $schema, 'data' => $data, 'expected' => $valid] in the assertion message. PHPUnit's generic diff is unreadable for nested schemas.
• Your harness is the extension point. There's nothing to subclass or configure in the suite itself — keep your runner small and generic so upgrades are a git pull, not a refactor.