Phpstan Sarif Formatter Laravel Package

Technical Evaluation

Architecture Fit

• PHPStan 2.x Compatibility: The package now explicitly supports PHPStan 2.x, aligning with the latest major version of PHPStan (released in 2024). This ensures long-term viability and reduces risk of toolchain drift.
• Backward Compatibility: The changelog suggests the update maintains compatibility with PHPStan 1.x, but teams should verify if they are upgrading PHPStan concurrently.
• SARIF Schema Stability: No changes to SARIF output format are mentioned, preserving integration with existing CI/CD pipelines (GitHub, Azure, etc.).
• Extensibility: Continued support for custom rules and severity mapping remains intact, as no breaking changes to the formatter’s core logic are indicated.

Integration Feasibility

• Seamless Upgrade Path: PHPStan 2.x adoption is straightforward for teams already using the package, with no additional configuration required beyond updating dependencies.
• Dependency Clarity: Explicit PHPStan 2.x support reduces ambiguity in composer.json constraints, simplifying dependency management.
• CI/CD Impact: No changes to SARIF output imply zero disruption to existing SARIF consumers (e.g., GitHub SARIF uploads, SonarQube plugins).

Technical Risk

• PHPStan 2.x Breaking Changes: While the package claims compatibility, PHPStan 2.x introduced breaking changes (e.g., new rule IDs, configuration structure). Teams must:
  • Audit custom PHPStan rules for compatibility (e.g., deprecated rules like Type\ArrayShape).
  • Validate SARIF output for rule ID consistency (e.g., ruleId in SARIF must match PHPStan 2.x’s new identifiers).
• Performance Regression: PHPStan 2.x’s internal changes (e.g., improved type inference) might alter analysis speed, indirectly affecting SARIF generation time.
• Toolchain Lock-in: Teams using PHPStan 1.x exclusively may face forced upgrades, requiring parallel testing of both versions during migration.

Key Questions

1. PHPStan Version Strategy:
   • Is the team planning to upgrade to PHPStan 2.x, or will they maintain PHPStan 1.x for legacy reasons?
   • If upgrading, what is the timeline, and how will SARIF output be validated post-migration?
2. Rule ID Mapping:
   • Have any PHPStan rules been deprecated or renamed in 2.x that require explicit SARIF metadata updates (e.g., helpUri)?
3. Custom Rules Impact:
   • Do custom PHPStan rules (e.g., via extension.phar) need adjustments to work with both PHPStan 1.x and 2.x during transition?
4. CI/CD Validation:
   • Should SARIF output be validated in CI for both PHPStan 1.x and 2.x during the upgrade window?
5. Performance Baseline:
   • Has the team benchmarked SARIF generation time with PHPStan 2.x against their largest codebase?

Integration Approach

Stack Fit

• PHPStan 2.x Alignment: The package now natively supports PHPStan 2.x’s features (e.g., improved type system, new rules like Type\ObjectType), enhancing SARIF’s accuracy.
• CI/CD Pipelines:
  • GitHub Actions: Continue using actions/upload-sarif with no changes.
  • GitLab/Azure: SARIF compatibility remains unchanged; no updates required.
• Local Development: VS Code’s SARIF extension and sarif-webapp will display PHPStan 2.x results without modification.

Migration Path

1. Phase 0: Pre-Upgrade Audit (New Step)
   • Run PHPStan 1.x and 2.x in parallel to identify rule ID discrepancies in SARIF output.
   • Use phpstan diagnose to list all enabled rules and compare with SARIF’s ruleId fields.
2. Phase 1: Dependency Update
   • Update composer.json to require jbelien/phpstan-sarif-formatter:^1.2 and phpstan/phpstan:^2.0.
   • Test SARIF generation locally:

     vendor/bin/phpstan analyse --generate-sarif=report.sarif
     

   • Validate SARIF with SARIF Validator.
3. Phase 2: CI/CD Canary
   • Enable SARIF generation in a non-production branch (e.g., feature/sarif-canary).
   • Monitor CI runtime and SARIF file size for regressions.
4. Phase 3: Full Rollout
   • Deprecate PHPStan 1.x in CI once SARIF output is stable with 2.x.
   • Update documentation to reflect PHPStan 2.x rule examples in SARIF.

Compatibility

• PHPStan Versions:
  • Supported: PHPStan 1.x and 2.x (verify with composer why-not jbelien/phpstan-sarif-formatter:1.2).
  • Unsupported: PHPStan <1.0 (drop support if no longer in use).
• SARIF Version: Unchanged (2.1.0+), but confirm PHPStan 2.x’s new rules (e.g., Type\ObjectType) are properly mapped.
• Rule Coverage: Audit SARIF output for missing or mislabeled rules (e.g., ruleId: "Type.ArrayShape" → ruleId: "Type.ObjectType").

Sequencing

1. Pre-requisite: Resolve any PHPStan 2.x breaking changes unrelated to SARIF (e.g., configuration syntax).
2. Parallel Testing: Run both PHPStan 1.x and 2.x with SARIF in CI to catch discrepancies early.
3. Post-Migration: Remove PHPStan 1.x dependencies once SARIF output is validated for 2.x.

Operational Impact

Maintenance

• Dependency Updates:
  • Monitor PHPStan 2.x’s evolution (e.g., new rules, deprecated features) and update SARIF metadata (e.g., helpUri) as needed.
  • Example: If PHPStan 2.x drops Type\ArrayShape, ensure SARIF’s ruleId reflects the new naming convention.
• Rule Changes: PHPStan 2.x’s improved type system may require adjustments to SARIF’s message or locations fields for accuracy.
• Tooling Drift: SARIF consumers (e.g., GitHub) may introduce new requirements (e.g., mandatory tool.driver field). Test annually.

Support

• Debugging:
  • PHPStan 2.x’s enhanced error messages will improve SARIF’s message.text, reducing false positives.
  • Use SARIF’s relatedLocations to link related code sections (e.g., type hints and usage).
• False Positives: PHPStan 2.x’s stricter type checking may surface new SARIF alerts; document common patterns (e.g., Type\ObjectType mismatches).
• Team Training:
  • Highlight PHPStan 2.x’s new rules in SARIF (e.g., Type\ObjectType) during onboarding.
  • Provide a cheat sheet mapping PHPStan 1.x → 2.x rule IDs for SARIF triage.

Scaling

• Performance:
  • PHPStan 2.x’s optimizations (e.g., faster type inference) may reduce SARIF generation time. Benchmark with hyperfine:

    hyperfine --warmup 3 'vendor/bin/phpstan analyse --generate-sarif=report.sarif'
    

  • Large codebases: Use --memory-limit=1G to avoid timeouts during SARIF generation.
• Storage: SARIF files from PHPStan 2.x may grow due to richer rule sets; archive old reports per GitHub’s 90-day retention limit.
• Distributed Teams: SARIF’s platform-agnostic format remains ideal for unified reporting, but ensure all teams use PHPStan 2.x to avoid rule ID inconsistencies.

Failure Modes

• SARIF Schema Drift:
  • PHPStan 2.x’s new rules may lack SARIF metadata (e.g., helpUri). Mitigate by:
    • Adding a pre-commit hook to validate SARIF against a schema (e.g., sarif-schema-cli).
    • Contributing missing metadata upstream (e.g., PR to phpstan-sarif-formatter).
• Consumer Parsing Errors:
  • GitHub/Azure may reject SARIF if tool.driver or ruleId formats change. Validate with:

    npx @microsoft/sarif-cli validate report.sarif
    

• Rule ID Conflicts:
  • Overlapping rule IDs between PHPStan 1.x and 2.x (e.g., Type.ArrayShape) may cause SARIF ambiguity. Resolve by:
    • Using ruleId prefixes (e.g., phpstan1:Type.ArrayShape, phpstan2:Type.ObjectType).
    • Deprecating PHPStan 1.x rules in SARIF once 2.x is fully adopted.

Ramp-Up

• Developer Onboarding:
  • 1-hour Session: Demo PHPStan 2.x’s new rules in SARIF (e.g., `Type\ObjectType