Llm Sdk Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require 1tomany/llm-sdk
   

   For Laravel, prefer the Symfony bundle for autowiring and config support.

2. First Use Case: Generate a simple LLM response using OpenAI:

   use OneToMany\LlmSdk\Clients\OpenAI\Client;
   use OneToMany\LlmSdk\Requests\GenerateOutputRequest;
   
   $client = new Client('your-api-key');
   $request = new GenerateOutputRequest(
       model: 'gpt-4',
       prompt: 'Hello, world!',
       maxTokens: 100
   );
   $response = $client->generateOutput($request);
   echo $response->getResponse();
   

3. Where to Look First:

   • Examples: Start with examples/outputs/generate.php for basic usage.
   • Feature Matrix: Check the supported platforms table to verify coverage for your use case.
   • API Docs: Review the Requests and Responses classes (e.g., GenerateOutputRequest, CreateEmbeddingResponse) for payload structure.

Implementation Patterns

Core Workflows

1. Direct Client Usage (Simple Integration)

• When: For one-off scripts or lightweight features.
• Pattern:

  $client = ClientFactory::create('openai', ['api_key' => env('OPENAI_KEY')]);
  $request = new GenerateOutputRequest(/* ... */);
  $response = $client->generateOutput($request);
  

• Pros: Minimal boilerplate, no dependency injection.
• Cons: Harder to test/mock; less flexible for complex apps.

2. Action-Based Pattern (Recommended for Laravel)

• When: Building features with reusable AI logic (e.g., chatbots, content generation).
• Pattern:

  // Register clients in a service provider
  $factory = new ClientFactory();
  $factory->register('openai', new OpenAI\Client(env('OPENAI_KEY')));
  
  // Define an action (e.g., SummarizeArticleAction)
  $action = new SummarizeArticleAction($factory);
  $result = $action->execute($articleText);
  

• Key Classes:
  • ClientFactory: Central registry for all LLM clients.
  • BaseAction: Abstract class for reusable AI workflows (extend for custom logic).
• Pros: Testable, mockable, and framework-agnostic.

3. Query Compilation (Advanced)

• When: Batch processing, logging, or complex payloads (e.g., RAG with files + prompts).
• Pattern:

  $query = new ProcessQueryRequest();
  $query->addPrompt('Explain Laravel middleware');
  $query->addFile($filePath);
  $query->setModel('gpt-4');
  
  $compiled = $client->compileQuery($query);
  $hash = $compiled->getHash(); // For caching/deduplication
  $response = $client->processQuery($compiled);
  

• Use Cases:
  • Caching: Use getHash() to cache responses (e.g., Redis).
  • Batching: Compile multiple queries for Gemini/OpenAI batch endpoints.
  • Audit Logging: Log $compiled->toArray() before execution.

4. File Handling

• Upload/Delete:

  $fileRequest = new FileRequest(
      file: fopen('document.pdf', 'r'),
      purpose: 'assistants'
  );
  $client->uploadFile($fileRequest);
  $client->deleteFile($fileId);
  

• Search Stores (Gemini Only):

  $store = $client->createSearchStore('my-store', 'gemini-1.5-pro');
  $client->importFileToSearchStore($storeId, $fileId);
  $results = $client->searchStore($storeId, 'query text');
  

Integration Tips

Laravel-Specific

• Service Providers: Bind the ClientFactory and actions in AppServiceProvider:

  public function register()
  {
      $this->app->singleton(ClientFactory::class, function () {
          $factory = new ClientFactory();
          $factory->register('openai', new OpenAI\Client(env('OPENAI_KEY')));
          return $factory;
      });
  }
  

• Dependency Injection: Inject ClientFactory or actions into controllers/services:

  public function __construct(private ClientFactory $factory) {}
  

• Configuration: Use the Symfony bundle for .env support:

  # config/packages/1tomany_llm.yaml
  one_to_many_llm:
      clients:
          openai:
              api_key: '%env(OPENAI_KEY)%'
              model: 'gpt-4'
  

Testing

• Mock Clients: Use the Mock client for unit tests:

  $factory->register('mock', new Mock\Client());
  $mockClient = $factory->get('mock');
  $mockClient->setResponse(['choices' => [['text' => 'Mocked reply']]]);
  

• Test Actions: Mock the ClientFactory to return test clients:

  $factory = Mockery::mock(ClientFactory::class);
  $factory->shouldReceive('get')->andReturn(new Mock\Client());
  $action = new MyAction($factory);
  

Performance

• Batching: Gemini/OpenAI support batch requests:

  $batch = $client->createBatch();
  $batch->addQuery($query1);
  $batch->addQuery($query2);
  $results = $client->readBatch($batchId);
  

• Caching: Cache compiled queries by hash:

  $hash = $compiled->getHash();
  $cached = cache()->get("llm_response_{$hash}");
  if (!$cached) {
      $response = $client->processQuery($compiled);
      cache()->put("llm_response_{$hash}", $response, now()->addHours(1));
  }
  

Gotchas and Tips

Pitfalls

1. Platform Limitations:

   • Anthropic: Lacks batching, embeddings, and search stores. Use OpenAI/Gemini for these features.
   • File Operations: Only OpenAI/Gemini support uploads/deletes. Mock client is best for testing file workflows.
   • Search Stores: Only Gemini supports this feature. Avoid in cross-platform code.

2. Model-Specific Quirks:

   • Gemini: Uses Content objects for prompts/files. OpenAI uses arrays.

     // Gemini
     $query->addContent(new Content\Text('Prompt text'));
     // OpenAI
     $query->addPrompt('Prompt text');
     

   • Mock Client: Requires explicit response setup for testing:

     $mockClient->setResponse(['choices' => [...]]);
     

3. Schema Validation:

   • JSON schemas must include a title field. The SDK extracts this for request IDs:

     $schema = ['title' => 'MySchema', 'properties' => [...]];
     

4. Rate Limiting:

   • The SDK doesn’t enforce rate limits. Implement middleware or use Laravel’s throttle:

     Route::middleware(['throttle:100,1'])->group(...);
     

5. File Handling:

   • Files are not automatically included in embeddings for multimodal models. Explicitly opt-in:

     $embeddingRequest->setIncludeFiles(true);
     

Debugging

1. Request Logging: Compile queries before sending to inspect payloads:

   $compiled = $client->compileQuery($query);
   \Log::info('LLM Request:', $compiled->toArray());
   

2. Error Messages: Normalized exceptions include:

   • InvalidRequestException: Malformed payloads.
   • ApiException: HTTP errors (e.g., 429 rate limits).
   • PlatformException: Provider-specific issues. Use try/catch to handle gracefully:

   try {
       $response = $client->generateOutput($request);
   } catch (ApiException $e) {
       \Log::error('LLM API Error:', ['status' => $e->getStatusCode()]);
       return response()->json(['error' => 'Service unavailable'], 503);
   }
   

3. Hash Collisions: CompileQueryResponse::getHash() uses SHA-256 of the request payload. Rare but possible for identical payloads with different metadata (e.g., timestamps