Visit Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require spatie/visit
   

   No additional configuration is required—just autoload the package.

2. First Use Case: Fetch and render a webpage (e.g., for scraping, testing, or previewing):

   use Spatie\Visit\Visit;
   
   $visit = Visit::url('https://example.com')
       ->visit();
   
   echo $visit->html(); // Outputs the raw HTML
   echo $visit->text(); // Outputs the rendered text
   

3. Where to Look First:

   • Documentation (official docs).
   • Spatie\Visit\Visit class (core functionality).
   • Spatie\Visit\Visitable trait (for customizable visitors).

Implementation Patterns

Core Workflows

1. Basic Visits:

   $visit = Visit::url('https://example.com')
       ->withHeaders(['User-Agent' => 'MyBot/1.0'])
       ->visit();
   
   $visit->html(); // Get HTML
   $visit->text(); // Get rendered text (via Guzzle + DOM)
   

2. Chaining with Middleware: Modify requests/responses globally:

   Visit::extend(function ($visit) {
       $visit->onVisit(function ($request) {
           $request->headers['X-Custom-Header'] = 'value';
       });
   });
   

3. Testing & Previews:

   • Unit Testing: Mock Visit for controlled responses:

     $visit = Visit::fake()
         ->withHtml('<h1>Test</h1>')
         ->visit();
     

   • Live Previews: Use in admin panels to render external content dynamically.

4. Integration with Laravel:

   • Service Providers: Bind Visit to the container for dependency injection:

     $this->app->singleton(Visit::class, function () {
         return new Visit();
     });
     

   • Jobs/Queues: Offload heavy visits to queues:

     Visit::url('https://slow-site.com')->visit()->later();
     

5. Custom Visitors: Extend functionality for specific needs (e.g., PDF rendering):

   class PdfVisitor extends Visitor
   {
       public function visit(Url $url)
       {
           // Custom logic (e.g., use a headless browser)
           return file_get_contents($url->withScheme('https')->__toString());
       }
   }
   
   Visit::extend(new PdfVisitor());
   

Gotchas and Tips

Pitfalls

1. Rate Limiting:

   • Some sites block rapid requests. Use delays or caching:

     $visit->visit()->cacheFor(minutes: 5); // Cache response
     

   • Respect robots.txt and terms of service.

2. Dynamic Content:

   • JavaScript-rendered content won’t be captured by default. Use:
     • Headless Browsers: Integrate with Puppeteer or Playwright via custom visitors.
     • API Endpoints: Prefer fetching data via APIs if available.

3. SSL/TLS Issues:

   • Some sites use self-signed certs. Disable verification cautiously:

     Visit::url('https://insecure-site.com')
         ->withOptions(['verify' => false]) // Not recommended for production!
         ->visit();
     

4. Memory Limits:

   • Large pages (e.g., e-commerce sites) may hit memory limits. Stream responses:

     $visit->visit()->streamHtml(); // Process chunks
     

Debugging

1. Inspect Requests/Responses: Enable debug mode to log raw data:

   Visit::enableDebugMode();
   // Check stored logs in `storage/logs/visit.log`
   

2. Common Errors:

   • 403 Forbidden: Add headers (e.g., User-Agent) or use proxies.
   • Timeouts: Increase timeout:

     ->withOptions(['timeout' => 30])
     

   • CORS Issues: Use a proxy or custom visitor.

Extension Points

1. Custom Headers/Proxies:

   Visit::extend(function ($visit) {
       $visit->onVisit(function ($request) {
           $request->setProxy('http://proxy.example.com:8080');
       });
   });
   

2. Response Processing: Override text() or html() methods in a custom visitor:

   class CleanVisitor extends Visitor
   {
       public function text()
       {
           return parent::text()->replace(['<script>...</script>', '<style>...</style>'], '');
       }
   }
   

3. Event Listeners: Hook into visit events for analytics or logging:

   Visit::onVisit(function ($url, $response) {
       Log::info("Visited {$url}", ['status' => $response->getStatusCode()]);
   });
   

4. Testing Utilities:

   • Fake Responses: Use Visit::fake() to avoid real requests in tests.
   • Assertions: Combine with Laravel’s testing helpers:

     $visit = Visit::fake()->withHtml('<h1>Hello</h1>');
     $this->assertStringContainsString('Hello', $visit->visit()->text());