Getting Started

Minimal Setup

Installation: Add via Composer (Laravel 12+ required):

composer require spatie/laravel-url-ai-transformer

Publish config:

php artisan vendor:publish --provider="Spatie\LaravelUrlAiTransformer\UrlAiTransformerServiceProvider"

Configure AI Provider: Set your preferred AI service in .env (now using Laravel AI package):
```
URL_AI_TRANSFORMER_AI_PROVIDER=openai
URL_AI_TRANSFORMER_AI_MODEL=gpt-4
```

First Transformation: Register URLs and a transformer in a service provider:

Transform::urls('https://example.com/blog')
    ->usingTransformers(new LdJsonTransformer);

Run Command: Execute transformations:
```
php artisan transform-urls
```

First Use Case: Structured Data Extraction with Laravel AI

Use the built-in LdJsonTransformer to extract structured data from a blog post:

Transform::urls('https://example.com/blog/my-post')
    ->usingTransformers(new LdJsonTransformer);

// Later, retrieve results:
$structuredData = TransformationResult::forUrl('https://example.com/blog/my-post', 'ldJson');

Implementation Patterns

Core Workflow (Updated for Laravel AI)

Register URLs: Use Transform::urls() to queue URLs for transformation:

Transform::urls(
    'https://example.com/page1',
    'https://example.com/page2'
)->usingTransformers(new LdJsonTransformer);

Define Transformers: Extend Transformer for custom logic using Laravel AI:

use Laravel\AI\Services\Prism;

class SummaryTransformer extends Transformer {
    public function transform(): void {
        $response = Prism::text()
            ->using(Config::aiProvider(), Config::aiModel())
            ->withPrompt($this->getPrompt())
            ->asText();
        $this->transformationResult->result = $response->text;
    }
}

Execute: Run the transform-urls Artisan command (queued by default).

Retrieve Results: Fetch transformed data via the TransformationResult model:

$result = TransformationResult::forUrl('https://example.com', 'ldJson');

Integration Tips (Laravel AI Compatibility)

Dynamic URL Sources: Use closures to fetch URLs dynamically:

Transform::urls(fn() => Article::published()->pluck('url')->toArray())
    ->usingTransformers(new LdJsonTransformer);

Multiple Transformers: Apply multiple transformers to a single URL:

Transform::urls('https://example.com/article')
    ->usingTransformers(
        new LdJsonTransformer,
        new ImageTransformer,
        new CustomSummaryTransformer
    );

Scheduled Transformations: Schedule the command in app/Console/Kernel.php:
```
$schedule->command('transform-urls')->dailyAt('02:00');
```
Event Listeners: Listen for transformation events (e.g., TransformationStarted, TransformationCompleted):
```
event(new TransformationStarted($url, $transformer));
```

Gotchas and Tips

Pitfalls (Updated for Laravel AI)

Laravel 12+ Requirement: Ensure your project is upgraded to Laravel 12+ and laravel/ai package is installed:
```
composer require laravel/ai
```
Rate Limits: AI providers (e.g., OpenAI) have rate limits. Use exponential backoff in custom jobs:
```
public $backoff = [60, 120, 300]; // Exponential backoff in seconds
```

URL Fetching Failures: Handle HTTP errors gracefully. Override the fetchUrlContent method:

public function fetchUrlContent(): string {
    try {
        return parent::fetchUrlContent();
    } catch (\Exception $e) {
        $this->transformationResult->latest_exception_message = $e->getMessage();
        throw $e;
    }
}

Prompt Design: Poorly designed prompts lead to irrelevant AI responses. Test prompts iteratively:

public function getPrompt(): string {
    return "Extract the following structured data from the webpage:\n"
        . "- Title\n"
        . "- Author\n"
        . "- Publication Date\n"
        . "- Main Content\n\n"
        . $this->urlContent;
}

Database Bloat: Transformations accumulate in the transformation_results table. Clean up old records:

TransformationResult::where('successfully_completed_at', '<', now()->subDays(30))
    ->delete();

Debugging Tips

Log Transformations: Add logging in your transformer’s transform method:

\Log::info("Transforming URL: {$this->url}", ['content' => substr($this->urlContent, 0, 200)]);

Inspect Failures: Check the latest_exception_message field in the database for errors:

$failedResult = TransformationResult::whereNotNull('latest_exception_seen_at')->first();

Test Locally: Use the --now flag to run transformations synchronously:
```
php artisan transform-urls --now
```

Extension Points

Custom Job Handling: Replace the default job for advanced queue logic:

// config/url-ai-transformer.php
'process_transformer_job' => App\Jobs\CustomProcessTransformerJob::class,

Override Actions: Customize URL fetching or AI response handling:

class CustomTransformer extends Transformer {
    public function fetchUrlContent(): string {
        // Custom logic (e.g., proxy requests)
        return file_get_contents($this->url);
    }
}

Conditional Transformations: Skip transformations based on URL patterns or content:

public function shouldRun(): bool {
    return !Str::contains($this->url, 'admin');
}

Custom Transformer Types: Override the default type derivation:

class MyTransformer extends Transformer {
    public function type(): string {
        return 'my_custom_type';
    }
}

Configuration Quirks

AI Provider Dependencies: Ensure your .env matches the installed AI provider package (e.g., spatie/laravel-openai for OpenAI).
Laravel AI Configuration: Verify config/ai.php is properly configured for your AI provider.

Content Length Limits: AI models have token limits. Trim long content in getPrompt():

public function getPrompt(): string {
    $content = Str::limit($this->urlContent, 2000);
    return "Summarize: {$content}";
}

Performance Tips

Batch Processing: Process URLs in batches to avoid queue overload:
```
php artisan transform-urls --batch=50
```

Cache Results: Cache transformation results for URLs that rarely change:

$result = Cache::remember("transform_{$url}_{$type}", now()->addHours(1), function() use ($url, $type) {
    return TransformationResult::forUrl($url, $type);
});

Laravel Url Ai Transformer Laravel Package