Laravel Url Ai Transformer Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require spatie/laravel-url-ai-transformer
   php artisan vendor:publish --provider="Spatie\UrlAiTransformer\UrlAiTransformerServiceProvider"
   

   Publish the config file to adjust AI service configurations (e.g., OpenAI API key).

2. First Use Case: Transform a URL's content into a structured summary or extract key points:

   use Spatie\UrlAiTransformer\Facades\UrlAiTransformer;
   
   $url = 'https://example.com/article';
   $summary = UrlAiTransformer::transform($url, 'summarize');
   

3. Key Configurations:

   • Set your AI service (e.g., OpenAI, HuggingFace) in config/url-ai-transformer.php.
   • Define default transformation types (e.g., summarize, extract_key_points) in the config.

Implementation Patterns

Core Workflows

1. Basic Transformations: Use the facade or service container to transform URLs:

   // Summarize a URL
   $summary = UrlAiTransformer::transform($url, 'summarize');
   
   // Extract key points
   $keyPoints = UrlAiTransformer::transform($url, 'extract_key_points');
   

2. Custom Transformations: Extend the package by creating custom transformers:

   namespace App\Services;
   
   use Spatie\UrlAiTransformer\Transformers\Transformer;
   
   class CustomTransformer extends Transformer
   {
       public function transform(string $content): string
       {
           return "Custom logic: " . $content;
       }
   }
   

   Register the transformer in config/url-ai-transformer.php under transformers.

3. Queueing Long-Running Tasks: Dispatch transformations to a queue for async processing:

   use Spatie\UrlAiTransformer\Jobs\TransformUrl;
   
   TransformUrl::dispatch($url, 'summarize')
       ->onQueue('ai_transformations');
   

4. Caching Responses: Cache AI-generated responses to avoid redundant API calls:

   UrlAiTransformer::transform($url, 'summarize', cacheFor: now()->addHours(1));
   

Integration Tips

• Laravel Scout: Use transformed content for search relevance.
• API Endpoints: Expose transformations via API routes:

  Route::post('/transform', function (Request $request) {
      return UrlAiTransformer::transform($request->url, $request->type);
  });
  

• Event Listeners: Trigger actions post-transformation (e.g., save to database):

  UrlAiTransformer::transform($url, 'summarize')->then(function ($summary) {
      // Save to DB or notify users
  });
  

Gotchas and Tips

Pitfalls

1. API Rate Limits:

   • AI services (e.g., OpenAI) have rate limits. Implement retries with exponential backoff:

     UrlAiTransformer::setRetryPolicy(3, 1000); // Retry 3 times with 1s delay
     

   • Monitor usage via config/url-ai-transformer.php logging.

2. Content Length Limits:

   • Long URLs or pages may exceed AI token limits. Pre-process content (e.g., truncate or split) before transformation.

3. Dynamic URL Fetching:

   • The package relies on spatie/url-fetcher to fetch URL content. Ensure URLs are accessible and not blocked by robots.txt.

4. Transformer Registration:

   • Custom transformers must implement the transform() method. Forgetting this will throw runtime errors.

Debugging

• Enable Debug Mode:

  UrlAiTransformer::enableDebugMode(); // Logs raw AI responses and errors
  

• Check Logs: Inspect storage/logs/laravel.log for failed transformations or API errors.

Extension Points

1. Custom AI Services: Extend the Spatie\UrlAiTransformer\Services\AiService class to support new providers (e.g., Anthropic, Cohere):

   class CustomAiService extends AiService
   {
       public function generate(string $prompt): string
       {
           // Custom logic for your AI provider
       }
   }
   

   Register the service in the config.

2. Pre/Post-Processing: Use middleware to modify content before/after transformation:

   UrlAiTransformer::addTransformerMiddleware(function ($content) {
       return str_replace(['[REDACTED]', '(CONFIDENTIAL)'], '', $content);
   });
   

3. Fallback Mechanisms: Implement fallback transformers for when the primary AI service fails:

   UrlAiTransformer::setFallbackTransformer('fallback_summarize');
   

Config Quirks

• Default Transformation Type: Set a default type in config/url-ai-transformer.php to avoid specifying it repeatedly:

  'default_transformer' => 'summarize',
  

• Environment-Specific Configs: Use .env variables to switch AI services between environments (e.g., AI_SERVICE=openai for local, AI_SERVICE=huggingface for production).