Clickhouse Laravel Laravel Package

Getting Started

1. Install and publish config
   Require the package via Composer, then publish its configuration (typically config/clickhouse.php). The buffer settings live there:

   'max_buffered_rows' => env('CLICKHOUSE_MAX_BUFFERED_ROWS'),
   'max_buffered_bytes' => env('CLICKHOUSE_MAX_BUFFERED_BYTES'),
   

2. Look first at the buffer limits
   In v1.3.1 these are ordinary nullable env entries. Leave them unset to inherit PDO’s defaults: 1,000,000 rows and 64 MiB. Set explicit positive integers in .env to override:

   # Omit entirely to use PDO defaults
   CLICKHOUSE_MAX_BUFFERED_ROWS=500000
   CLICKHOUSE_MAX_BUFFERED_BYTES=33554432
   

3. First use case: buffered inserts
   The most common day-to-day task is inserting rows into ClickHouse without unbounded memory growth. Use the package’s insert/buffer API (e.g. ClickHouse::insert('events', $rows)) and tune the two limits if your workload is memory-sensitive.

Implementation Patterns

• Environment-specific tuning
  Keep local/staging limits lower to surface memory issues early. In production, either omit the variables (PDO defaults) or set values based on worker memory and batch size.

• Queue + chunk pattern
  Dispatch inserts from queued jobs and chunk large collections:

  collect($rows)->chunk(5000)->each(fn ($chunk) => InsertToClickHouse::dispatch($chunk));
  

  Use max_buffered_rows to align with your chunk size and max_buffered_bytes to cap payload memory.

• Runtime overrides
  If the package reads config at insert time, you can override per job:

  config(['clickhouse.max_buffered_rows' => 10000]);
  

  Useful for one-off imports that need tighter limits.

• Config caching
  Set env values before php artisan config:cache in deployment. After changing .env, run php artisan config:clear (or re-cache) so the new nullable values take effect.

• Encapsulate inserts
  Wrap buffer/insert logic in a repository or service class so limits, retries, and error handling stay consistent across the app.

Gotchas and Tips

• Null vs empty string
  env('CLICKHOUSE_MAX_BUFFERED_ROWS') returns null only when the variable is absent or literally null. CLICKHOUSE_MAX_BUFFERED_ROWS= yields an empty string, which fails the “positive integer” requirement. To use PDO defaults, omit the line or set =null.

• Positive integers only
  Explicit values must be integers > 0. 0, negatives, or strings like "64M" will cause validation/config errors.

• PDO defaults are large
  1,000,000 rows and 64 MiB may be too high for memory-constrained queues. If you see worker OOMs, set explicit lower limits and monitor buffer flush behavior.

• Debugging buffered inserts
  Temporarily set a very low limit (e.g. CLICKHOUSE_MAX_BUFFERED_ROWS=100) to force frequent flushes and confirm data is written. Check package logs and ClickHouse query logs for errors.

• No env() outside config
  Use config('clickhouse.max_buffered_rows') in application code. env() calls outside config files break when config is cached.

• Upgrade note for v1.3.1
  Existing .env files that relied on empty string values may now fail. Audit for CLICKHOUSE_MAX_BUFFERED_ROWS= / CLICKHOUSE_MAX_BUFFERED_BYTES= and replace with omitted variables, null, or valid positive integers.

• Extension point
  If the package resolves its buffer manager from the container, bind a custom implementation in a service provider to apply dynamic limits per tenant or job.