Http Status Check Laravel Package

Getting Started

Minimal Steps

1. Installation

   composer global require spatie/http-status-check
   

   Ensure ~/.composer/vendor/bin is in your PATH or use the full path to the binary.

2. First Run

   http-status-check https://example.com
   

   This will crawl example.com, check all links, and output a table with URLs, status codes, and response times.

3. Where to Look First

   • CLI Help: Run http-status-check --help to explore all available flags.
   • Output Formats: Default is a table, but --format=json or --format=csv can be used for programmatic use.
   • Configuration: Check ~/.config/spatie/http-status-check.php (auto-generated) for customizable defaults like:
     • max_depth (default: 1),
     • ignore_status_codes (e.g., [404, 403]),
     • user_agent,
     • timeout.

4. First Use Case

   • Audit a Site: Quickly identify broken links on a staging or production site.

     http-status-check https://your-site.com --max-depth=2 --ignore-status-codes=404
     

   • Generate Reports: Export results to CSV for further analysis:

     http-status-check https://example.com --format=csv > links_report.csv
     

Implementation Patterns

Usage Patterns

1. Integrating into CI/CD

   • Add the command to a phpunit.xml or Makefile to run on deploy:

     <php>
         <env name="PATH" value="~/.composer/vendor/bin:$PATH"/>
     </php>
     <testsuite name="HTTP Status Checks">
         <file>vendor/bin/http-status-check</file>
         <arg value="https://your-site.com"/>
         <arg value="--fail-on-status=500"/>
     </testsuite>
     

   • Fail the build if critical status codes (e.g., 5xx) are found:

     http-status-check https://your-site.com --fail-on-status=500,404
     

2. Customizing Crawl Behavior

   • Exclude Paths: Use --exclude-paths to skip specific routes (e.g., /admin, /api):

     http-status-check https://example.com --exclude-paths=/admin,/api
     

   • Follow Redirects: Disable with --follow-redirects=false to inspect intermediate responses.
   • Rate Limiting: Add delays between requests to avoid overwhelming the server:

     http-status-check https://example.com --delay=1000  # 1-second delay
     

3. Programmatic Use

   • Pipe output to other tools (e.g., jq for JSON parsing):

     http-status-check https://example.com --format=json | jq '.[] | select(.status_code >= 400)'
     

   • Use in Laravel tasks via Artisan::call() (if wrapped in a custom command).

4. Monitoring Workflows

   • Schedule regular checks with cron:

     0 3 * * * /usr/bin/env php ~/.composer/vendor/bin/http-status-check https://your-site.com --format=csv --output=/var/log/link_checks.csv
     

   • Compare results over time to track regressions.

Integration Tips

• Laravel-Specific:
  • Store results in a database by parsing the output and seeding a link_checks table.
  • Use the package alongside spatie/laravel-backup to include link checks in backup validation.
• Headless Browsers:
  • For JavaScript-rendered content, combine with tools like puppeteer or playwright to extract links before passing to http-status-check.
• Authentication:
  • Use --user-agent and --headers to mimic authenticated requests:

    http-status-check https://example.com --headers="Authorization: Bearer token123"
    

Gotchas and Tips

Pitfalls

1. Performance Issues

   • Problem: Crawling large sites with high max_depth can be slow or hit rate limits.
   • Fix: Start with max_depth=1, then incrementally increase. Use --delay to throttle requests.
   • Example:

     http-status-check https://example.com --max-depth=2 --delay=500
     

2. False Positives/Negatives

   • Problem: Dynamic content (e.g., API endpoints, user-specific URLs) may not be caught.
   • Fix: Exclude dynamic paths (e.g., --exclude-paths=/user/*) or mock responses in tests.

3. Redirect Loops

   • Problem: Infinite redirects can crash the tool.
   • Fix: Limit redirects with --max-redirects=5 or exclude problematic domains.

4. SSL/TLS Issues

   • Problem: Self-signed certificates or outdated TLS may fail silently.
   • Fix: Use --insecure to skip SSL verification (not recommended for production) or update your CA bundle.

5. Output Parsing Quirks

   • Problem: CSV/JSON output may include unexpected characters (e.g., commas in URLs).
   • Fix: Use --format=json for reliable parsing or sanitize CSV output with tools like mlr (Miller).

Debugging

1. Verbose Mode Enable debug output to inspect crawled URLs and responses:

   http-status-check https://example.com --verbose
   

   • Look for Crawling, Checking, and Skipping logs to identify why URLs are missed.

2. Dry Runs Use --dry-run to see which URLs will be checked without making requests:

   http-status-check https://example.com --dry-run
   

3. Logging to File Redirect output for later analysis:

   http-status-check https://example.com --verbose > debug.log 2>&1
   

Config Quirks

1. Global Configuration

   • The package generates a config file at ~/.config/spatie/http-status-check.php. Override defaults here for repeated use:

     return [
         'max_depth' => 2,
         'ignore_status_codes' => [404, 410],
         'user_agent' => 'MyAppLinkChecker/1.0',
     ];
     

   • Note: Changes require restarting the CLI tool to take effect.

2. Environment Variables

   • Some settings (e.g., HTTP_STATUS_CHECK_TIMEOUT) can be set via environment variables, but check the source for supported vars.

3. Custom Headers

   • Headers like Authorization or Cookie must be URL-encoded or passed via --headers:

     http-status-check https://example.com --headers="Cookie: session_id=abc123"
     

Extension Points

1. Custom Status Code Handling

   • Extend the tool by forking the repo and modifying src/Checkers/StatusCodeChecker.php to add custom logic (e.g., treat 429 as a warning).

2. Plugin System

   • The package lacks a formal plugin system, but you can:
     • Pre-process URLs with a script (e.g., filter out internal links).
     • Post-process output (e.g., send alerts for 5xx via webhooks).

3. Integration with Laravel

   • Create a custom Artisan command to wrap http-status-check:

     // app/Console/Commands/CheckSiteLinks.php
     use Spatie\HttpStatusCheck\HttpStatusCheck;
     
     class CheckSiteLinks extends Command {
         protected $signature = 'site:check-links {url}';
         public function handle() {
             $checker = new HttpStatusCheck($this->argument('url'));
             $results = $checker->check();
             // Save to DB or log results
         }
     }
     

   • Use the underlying HttpStatusCheck class for programmatic access.

4. Parallel Checks

   • For large sites, consider parallelizing checks using parallel-lint or a custom script to split URLs across processes. The package itself does not support parallelism.