Llm Sdk Laravel Package

Technical Evaluation

Architecture Fit

• Strengths:

  • Unified Abstraction: Aligns perfectly with a strategy pattern for AI providers, enabling seamless provider swapping (e.g., OpenAI → Anthropic) via dependency injection. This fits Laravel’s service container and facade patterns natively.
  • Modular Design: Separation of clients, resources (embeddings, files, outputs), and queries allows granular adoption (e.g., start with GenerateOutput before enabling search stores).
  • Query Compilation: Pre-request payload logging and batching support observability and cost optimization, critical for production-grade AI systems.
  • Framework-Agnostic: Works alongside Laravel’s service providers, events, or queues (e.g., dispatch AI tasks to a queue for async processing).
  • Symfony Bundle: Leverages Laravel’s Symfony components (e.g., HttpClient, Cache) for compatibility.

• Gaps:

  • Limited Provider Support: Only 4 providers (Anthropic, Gemini, Mock, OpenAI). Risk: If your roadmap includes Mistral, Cohere, or custom models, you’ll need to extend the SDK or build parallel integrations.
  • Feature Parity: Inconsistent support (e.g., Anthropic lacks batching/search stores). Mitigation: Prioritize providers/features aligned with your MVP.
  • No Native Laravel Integrations: While the Symfony bundle exists, Laravel-specific features (e.g., Eloquent models for AI responses, Horizon queues) require custom glue code.

Integration Feasibility

• Laravel Compatibility:
  • High: Uses PSR-11 containers, PSR-7 HTTP messages, and PHP 8.1+ (Laravel 9+). Integrates with:
    • Service Container: Bind clients/factories in AppServiceProvider.
    • HTTP Client: Works with Laravel’s Http facade or Guzzle under the hood.
    • Events: Extend with Laravel events (e.g., AiRequestCompiled, AiResponseReceived).
    • Queues: Wrap SDK calls in jobs for async processing.
  • Caching: Leverage Laravel’s Cache facade to store compiled queries or responses.
• Database: No ORM assumptions, but you can map responses to Eloquent models (e.g., Embedding, GeneratedOutput) for persistence.
• Testing: Mockable via the Mock provider or PHPUnit’s createMock.

Technical Risk

Key Questions

1. Provider Strategy:
   • Which 2–3 providers will you prioritize initially? (e.g., OpenAI + Gemini for redundancy).
   • Do you need support for custom/self-hosted models (e.g., vLLM, Ollama)? If so, how will you extend the SDK?
2. Feature Scope:
   • Will you start with generative outputs (e.g., chatbots) or embeddings (e.g., RAG)? This dictates which clients/resources to enable first.
   • Do you need search stores (Gemini-only) or batching (OpenAI/Gemini)? Align with your roadmap.
3. Operational Requirements:
   • How will you log AI requests/responses for auditing? (e.g., Laravel’s Log facade or a dedicated table).
   • What’s your SLA for AI responses? If <500ms, test the SDK’s overhead.
4. Extensibility:
   • Will you need to add custom providers? If so, document the adapter pattern (e.g., new CustomClient()).
   • How will you handle provider-specific features not covered by the SDK? (e.g., OpenAI’s fine-tuning).
5. Cost Controls:
   • How will you monitor API usage/costs? (e.g., integrate with a tool like Pulumi or OpenAI’s cost analyzer).
   • Do you need fallback providers if a primary one fails? (e.g., OpenAI → Mock for testing).

Integration Approach

Stack Fit

• Laravel Ecosystem:

  • Service Container: Register clients/factories in AppServiceProvider:

    $this->app->bind(ClientFactory::class, function ($app) {
        return new ClientFactory([
            'openai' => new OpenAIClient(config('services.openai.key')),
            'gemini' => new GeminiClient(config('services.gemini.key')),
        ]);
    });
    

  • Config: Store API keys in config/services.php:

    'llm-sdk' => [
        'providers' => ['openai', 'gemini'],
        'default' => env('LLM_PROVIDER', 'openai'),
    ],
    

  • Events: Dispatch custom events for AI lifecycle hooks:

    event(new AiRequestCompiled($query, $provider));
    

  • Queues: Offload AI tasks to a queue (e.g., GenerateOutputJob):

    GenerateOutputJob::dispatch($query)->onQueue('ai');
    

  • Caching: Cache compiled queries or responses:

    Cache::remember("ai_query_{$hash}", now()->addHours(1), fn() => $client->processQuery($query));
    

• Database:

  • Option 1: Store AI responses in Eloquent models (e.g., GeneratedOutput):

    class GeneratedOutput extends Model {
        protected $casts = ['response' => 'array'];
    }
    

  • Option 2: Use a NoSQL database (e.g., MongoDB) for unstructured responses.

• Testing:

  • Unit Tests: Mock the MockClient or use PHPUnit’s createMock:

    $mockClient = $this->createMock(ClientInterface::class);
    $mockClient->method('generateOutput')->willReturn(new GenerateOutputResponse(...));
    

  • Feature Tests: Use Laravel’s Http tests to verify API endpoints calling the SDK.

Migration Path

1. Phase 1: Proof of Concept (2–4 weeks)

   • Goal: Validate the SDK meets core needs (e.g., generative outputs).
   • Steps:
     • Install the SDK: composer require 1tomany/llm-sdk.
     • Implement a single provider (e.g., OpenAI) in a non-critical feature (e.g., a "draft response" tool).
     • Test with mock data (use the MockClient).
   • Deliverable: A working example with logging and error handling.

2. Phase 2: Core Integration (4–6 weeks)

   • Goal: Integrate with Laravel’s stack and add a second provider.
   • Steps:
     • Register the ClientFactory in the service container.
     • Create a config file for provider keys and defaults.
     • Build a service class to wrap SDK calls (e.g., AiService):

       class AiService {
           public function __construct(private ClientFactory $factory) {}
       
           public function generate(string $prompt, string $provider = null) {
               $client = $this->factory->get($provider ?? config('llm-sdk.default'));
               return $client->generateOutput(new GenerateOutputRequest($prompt));
           }
       }
       

     • Add events for observability (e.g., AiRequestSent, AiResponseReceived).
     • Implement caching for frequent queries.
   • Deliverable: A reusable AiService with 2 providers and basic logging.

3. Phase 3: Advanced Features (4–8 weeks)

   • Goal: Add provider-specific features (e.g., search stores, batching).