Laravel Twitter Streaming Api Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require spatie/laravel-twitter-streaming-api
   php artisan vendor:publish --provider="Spatie\TwitterStreamingApi\TwitterStreamingApiServiceProvider"
   

   • Publishes config file (config/twitter-streaming-api.php) with default credentials (update with your Twitter API keys).

2. First Use Case: Listen for tweets containing a specific hashtag and log them:

   use Spatie\TwitterStreamingApi\Facades\TwitterStreamingApi;
   
   TwitterStreamingApi::publicStream()
       ->whenHears('#laravel', function (array $tweet) {
           \Log::info("New tweet: " . $tweet['text']);
       })
       ->startListening();
   

   • Run via an Artisan command (recommended for production) or directly in a route/controller (for testing).

3. Key Config:

   • Set api_key, api_secret_key, access_token, and access_token_secret in .env or config file.
   • Enable/disable SSL verification in config if needed (e.g., for local testing).

Implementation Patterns

Core Workflows

1. Stream Types:

   • Public Stream: Filter tweets in real-time (e.g., hashtags, keywords).

     TwitterStreamingApi::publicStream()
         ->whenHears(['#php', '#laravel'], function ($tweet) { ... })
         ->startListening();
     

   • User Stream: Listen to a specific user’s activity (requires auth).

     TwitterStreamingApi::userStream('target_user_screen_name')
         ->whenLikes(function ($tweet) { ... })
         ->startListening();
     

   • Filtered Stream: Combine multiple rules (e.g., track hashtags + exclude spam).

     TwitterStreamingApi::filteredStream()
         ->track(['#laravel', '#coding'])
         ->whenHears('#spam', function () { return false; }) // Ignore spam
         ->startListening();
     

2. Event Callbacks:

   • Use closures for real-time reactions:

     ->whenHears('#job', function ($tweet) {
         // Process tweet (e.g., save to DB, trigger webhook)
         $this->dispatch(new ProcessTweet($tweet));
     })
     ->whenFollows(function ($user) {
         // Handle follow events
     });
     

   • Batch Processing: For high-volume streams, buffer tweets and process in chunks:

     $buffer = [];
     TwitterStreamingApi::publicStream()
         ->whenHears('#news', function ($tweet) use (&$buffer) {
             $buffer[] = $tweet;
             if (count($buffer) >= 100) {
                 $this->processBuffer($buffer);
                 $buffer = [];
             }
         });
     

3. Artisan Commands:

   • Encapsulate streaming logic in a command for production use:

     php artisan twitter:stream --hashtag="#laravel" --action="log"
     

   • Example command:

     namespace App\Console\Commands;
     use Illuminate\Console\Command;
     use Spatie\TwitterStreamingApi\Facades\TwitterStreamingApi;
     
     class StreamHashtags extends Command
     {
         protected $signature = 'twitter:stream {--hashtag=*}';
         public function handle()
         {
             TwitterStreamingApi::publicStream()
                 ->whenHears($this->option('hashtag'), function ($tweet) {
                     \Log::info($tweet['text']);
                 })
                 ->startListening();
         }
     }
     

4. Integration with Queues:

   • Offload tweet processing to queues to avoid blocking the stream:

     ->whenHears('#alert', function ($tweet) {
         dispatch(new ProcessTweetJob($tweet));
     });
     

5. Database Storage:

   • Store tweets in a table (e.g., tweets) with a model:

     ->whenHears('#save', function ($tweet) {
         Tweet::create([
             'user_id' => $tweet['user']['id'],
             'text' => $tweet['text'],
             'created_at' => now(),
         ]);
     });
     

Gotchas and Tips

Pitfalls

1. Rate Limits:

   • Twitter’s streaming API has rate limits. Exceeding limits may temporarily disconnect the stream.
   • Solution: Implement reconnection logic or exponential backoff in your command.

2. Connection Drops:

   • Streams may drop due to network issues or Twitter’s side. Use ->reconnect() or wrap in a retry loop:

     while (true) {
         try {
             TwitterStreamingApi::publicStream()->whenHears(...)->startListening();
         } catch (\Exception $e) {
             \Log::error("Stream failed: " . $e->getMessage());
             sleep(10); // Wait before reconnecting
         }
     }
     

3. Memory Leaks:

   • Long-running streams can consume memory if callbacks aren’t optimized.
   • Solution: Use generators or batch processing for heavy workloads.

4. Authentication Issues:

   • Invalid API keys/secrets will fail silently. Always verify credentials:

     php artisan twitter:auth:validate
     

   • Tip: Use .env variables and never hardcode keys.

5. Timeouts:

   • PHP’s default max_execution_time may kill long-running streams.
   • Solution: Run the stream in a separate process (e.g., Laravel Horizon, Supervisor) or increase timeout:

     set_time_limit(0); // Infinite timeout (use cautiously)
     

Debugging

1. Logging:

   • Enable debug mode in config:

     'debug' => env('TWITTER_STREAMING_DEBUG', false),
     

   • Log raw tweets for inspection:

     ->whenHears('#debug', function ($tweet) {
         \Log::debug('Raw tweet:', $tweet);
     });
     

2. Testing Locally:

   • Use Twitter’s sandbox environment for testing.
   • Mock the API in unit tests:

     $this->mock(TwitterStreamingApi::class)->shouldReceive('publicStream')
         ->andReturnSelf()
         ->shouldReceive('whenHears')
         ->andReturnSelf()
         ->shouldReceive('startListening')
         ->once();
     

3. Common Errors:

   • 420 Error: Too many requests. Implement backoff.
   • 401 Unauthorized: Invalid credentials. Double-check .env.
   • 403 Forbidden: IP/access restrictions. Use Twitter’s API status page.

Tips

1. Filter Efficiency:

   • Combine rules to reduce noise:

     ->track(['#laravel', '#php'])
     ->whenHears('#spam', function () { return false; }) // Exclude spam
     ->whenFollows(function ($user) {
         if ($user['followers_count'] > 1000) { ... } // Ignore low-engagement users
     });
     

2. Webhooks:

   • Forward tweets to an external service via HTTP:

     ->whenHears('#webhook', function ($tweet) {
         Http::post('https://your-service.com/webhook', ['tweet' => $tweet]);
     });
     

3. Environment-Specific Config:

   • Use Laravel’s config/caching to load different keys per environment:

     'keys' => [
         'local' => ['api_key' => 'local_key'],
         'production' => ['api_key' => env('TWITTER_API_KEY')],
     ],
     

4. Performance:

   • For high-volume streams, use database bulk inserts or message queues (e.g., Redis, RabbitMQ) to avoid I/O bottlenecks.

5. Extending the Package:

   • Add custom stream types by extending Spatie\TwitterStreamingApi\StreamBuilder:

     class CustomStream extends StreamBuilder
     {
         public function customAction($callback)
         {
             $this->actions[] = ['custom', $callback];
             return $this;
         }
     }
     

   • Register the new stream in the service provider:

     $this->app->bind('custom.stream', function () {
         return new CustomStream();
     });