Phpstan Doc Code Analyzer Laravel Package

Technical Evaluation

Architecture Fit

• Purpose Alignment: The package bridges static analysis (PHPStan) with documentation/code snippets, addressing a niche but critical gap in developer workflows. It enables runtime-like validation of static examples in docs/readmes, reducing misalignment between implementation and documentation.
• Leverage Points:
  • Developer Experience (DX): Integrates seamlessly with existing PHPStan workflows, requiring minimal additional tooling.
  • Quality Assurance (QA): Automates validation of public-facing code snippets, catching inconsistencies early (e.g., deprecated APIs, type mismatches).
  • Onboarding: Reduces cognitive load for new contributors by ensuring docs reflect current code.
• Anti-Patterns:
  • Overhead Risk: If misconfigured, could introduce flakiness (e.g., false positives in dynamic snippets like CLI examples).
  • Scope Creep: May tempt teams to over-rely on it for non-critical docs (e.g., high-level architecture diagrams).

Integration Feasibility

• PHPStan Dependency: Requires PHPStan (≥v1.10.0) as a prerequisite, which is already a standard in many PHP projects (e.g., Laravel, Symfony). Feasibility: High if PHPStan is already in use.
• Toolchain Compatibility:
  • CI/CD: Can be gated in PR pipelines (e.g., GitHub Actions) alongside PHPStan rules.
  • IDE: No direct IDE integration, but can complement tools like PHPStorm’s PHPStan plugin.
  • Monorepos: May struggle with multi-language repos (e.g., PHP + JS) unless scoped to PHP directories.
• Configuration Complexity:
  • Low: Primarily extends PHPStan’s phpstan.neon with a new rule (PrinsFrank\DocCodeAnalyzer\Rule\DocCodeRule).
  • Example Config:

    includes:
      - src/
      - tests/
      - docs/**/*.md  # Target markdown files
    services:
      - PrinsFrank\DocCodeAnalyzer\Extension
    

Technical Risk

• False Positives/Negatives:
  • Risk: Snippets with dynamic values (e.g., $user->id where id is not statically analyzable) may trigger false positives.
  • Mitigation: Use @var annotations or exclude paths for unanalyzable snippets.
• Performance:
  • Risk: Scanning large doc bases (e.g., 100+ markdown files) could slow CI.
  • Mitigation: Cache results or parallelize with PHPStan’s --parallel flag.
• Maintenance:
  • Risk: Package is new (1 star, last release 2025-09-10). Unclear long-term support.
  • Mitigation: Fork or wrap in a custom extension if needed.

Key Questions

1. Scope of Analysis:
   • Should we validate all markdown files or only those in docs/?
   • How to handle snippets with external dependencies (e.g., database queries)?
2. Error Handling:
   • Should failures block PRs or be logged as warnings?
   • Need for custom error messages (e.g., "Snippet in README.md:42 uses deprecated Carbon::now()").
3. Toolchain Integration:
   • Can it integrate with existing linters (e.g., PHP-CS-Fixer) or should it run independently?
4. Dynamic Content:
   • How to handle snippets with runtime-generated values (e.g., $request->input('dynamic_key'))?

Integration Approach

Stack Fit

• PHP Ecosystem: Ideal for PHP-centric stacks (Laravel, Symfony, custom PHP apps) where:
  • PHPStan is already used for static analysis.
  • Documentation is written in markdown (e.g., Laravel’s official docs).
• Non-PHP Stacks: Limited utility (e.g., Node.js, Python) unless wrapped in a custom solution.
• Hybrid Stacks: Can coexist with other tools (e.g., JS linters) if scoped to PHP-only docs.

Migration Path

1. Assessment Phase:
   • Audit existing docs for code snippets (e.g., src/ references in README.md).
   • Identify high-priority files (e.g., CONTRIBUTING.md, API docs).
2. Pilot Integration:
   • Add to composer.json:

     "require-dev": {
       "prinsfrank/phpstan-doc-code-analyzer": "^1.0"
     }
     

   • Configure phpstan.neon to target docs (see Technical Evaluation).
   • Run locally and in CI with a warning-only mode initially.
3. Full Rollout:
   • Gate in CI (e.g., fail PRs on doc snippet errors).
   • Add to onboarding docs for contributors.
4. Optimization:
   • Exclude non-critical paths (e.g., docs/legacy/).
   • Cache results in CI (e.g., GitHub Actions cache).

Compatibility

• PHPStan Versions: Tested with PHPStan ≥1.10.0. Ensure compatibility with your version.
• Markdown Parsing:
  • Uses php-markdown internally. May struggle with custom markdown syntax (e.g., GitHub-flavored extensions).
  • Workaround: Pre-process docs or use a whitelist of supported syntax.
• IDE Support:
  • No direct IDE integration, but PHPStan’s existing IDE plugins (e.g., PHPStorm) can highlight issues in docs if opened as temporary files.

Sequencing

1. Pre-Requisite: Ensure PHPStan is configured and passing for the codebase.
2. Phase 1: Validate core docs (e.g., README.md, docs/installation.md).
3. Phase 2: Expand to tutorials/examples.
4. Phase 3: Integrate with other tools (e.g., auto-fix snippets via custom scripts).

Operational Impact

Maintenance

• Configuration Drift:
  • Risk: As docs evolve, snippet validation rules may need updates (e.g., new @var annotations).
  • Mitigation: Document exclusion patterns and annotation requirements in a DOCS.md file.
• Dependency Updates:
  • Risk: Package updates may break compatibility (e.g., PHPStan major version bumps).
  • Mitigation: Pin versions in composer.json or fork if critical.
• False Positives:
  • Maintenance Cost: Requires periodic review of suppressed warnings/exclusions.

Support

• Developer Onboarding:
  • Pros: Reduces "works on my machine" issues by validating docs.
  • Cons: New contributors may need guidance on fixing doc snippet errors.
  • Solution: Add a docs/ section to the contributing guide with examples.
• Troubleshooting:
  • Debugging: Errors may be cryptic (e.g., "Snippet failed in line 10"). Enhance error messages with:

    // Custom error formatter in phpstan.neon
    errorLevel: Error
    message: "Documentation snippet error in {file}:{line}. Expected type {expected}, got {actual}."
    

• Community Support:
  • Risk: Limited community (1 star). May need internal triage for issues.

Scaling

• Performance:
  • Small Projects: Negligible overhead.
  • Large Codebases: Scanning 100+ markdown files with complex snippets may slow CI.
  • Mitigation:
    • Parallelize with PHPStan’s --parallel.
    • Cache results (e.g., store analyzed files in .phpstan.cache).
• Distributed Teams:
  • Pros: Reduces doc-code misalignment across teams.
  • Cons: May require coordination to update snippets globally.

Failure Modes

Ramp-Up

• Learning Curve:
  • Low: Familiar to teams already using PHPStan.
  • Moderate: Requires understanding of:
    • PHPStan’s rule system.
    • Markdown parsing quirks (e.g., code blocks, syntax highlighting).
• Training:
  • Internal Docs: Add a "Doc Snippet Validation" section to the wiki.
  • Workshops: Demo how to:
    • Fix a failing snippet.
    • Exclude non-critical paths.
    • Add @var annotations for dynamic values.
• Adoption Metrics:
  • Track:
    • % of docs covered by validation.
    • Reduction in "broken doc" issues reported by users.
    • Time saved in onboarding (e.g