Etl Adapter Json Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require flow-php/etl-adapter-json
   

   Ensure your project uses Flow Framework (v10+).

2. Basic Usage Import the adapter in your ETL pipeline:

   use Flow\ETL\Adapter\JsonAdapter;
   use Flow\ETL\Pipeline;
   
   $pipeline = new Pipeline();
   $pipeline->addAdapter(new JsonAdapter());
   

3. First Use Case: Parsing JSON Input

   $jsonInput = '{"name": "John", "age": 30}';
   $result = $pipeline->process($jsonInput);
   // Result: Associative array `['name' => 'John', 'age' => 30]`
   

4. Key Files

   • src/Adapter/JsonAdapter.php (Core logic)
   • tests/ (Unit/integration tests for reference)

Implementation Patterns

Workflow: JSON-to-Array Transformation

$pipeline
    ->addAdapter(new JsonAdapter()) // Parse JSON
    ->addAdapter(new \Flow\ETL\Adapter\FilterAdapter()) // Filter data
    ->addAdapter(new \Flow\ETL\Adapter\ArrayAdapter()); // Output as array

Integration with Flow ETL

• Chaining Adapters: Use JsonAdapter as the first step in a pipeline to parse raw JSON input.
• Error Handling: Wrap in a TryCatchAdapter for graceful JSON decode failures:

  $pipeline
      ->addAdapter(new TryCatchAdapter(new JsonAdapter()))
      ->addAdapter(new LogAdapter()); // Log errors
  

Custom JSON Handling

• Custom Decoding: Extend JsonAdapter to handle non-standard JSON:

  class CustomJsonAdapter extends JsonAdapter {
      protected function decode(string $json): array {
          return json_decode($json, true, 512, JSON_BIGINT_AS_STRING);
      }
  }
  

Batch Processing

• Streaming Large JSON: Use JsonAdapter with StreamAdapter for large files:

  $pipeline
      ->addAdapter(new StreamAdapter())
      ->addAdapter(new JsonAdapter());
  

Gotchas and Tips

Pitfalls

1. Malformed JSON

   • JsonAdapter throws \JsonException on invalid JSON. Use TryCatchAdapter or validate input first:

     if (!json_validate($json)) { /* handle error */ }
     

2. Associative Array Assumption

   • json_decode($json, true) always returns an array, even for objects. Use JsonObjectAdapter (if available) for object preservation.

3. Memory Limits

   • Large JSON files may hit PHP’s memory_limit. Use JSON_BIGINT_AS_STRING or chunk processing.

Debugging

• Enable JSON Error Reporting

  JsonAdapter::setThrowExceptions(false); // Default: true
  $pipeline->addAdapter(new LogAdapter()); // Log decode errors
  

Configuration Quirks

• No Built-in Config: JsonAdapter has no config options. Customize via subclassing or wrapper adapters.
• Default Options: Uses PHP’s json_decode() defaults (associative arrays, no bigint handling).

Extension Points

1. Custom Decoders Override decode() to support custom JSON formats (e.g., JSON5, JSON Lines):

   class JsonLinesAdapter extends JsonAdapter {
       protected function decode(string $json): array {
           return array_map('json_decode', explode("\n", $json));
       }
   }
   

2. Pre/Post-Processing Use BeforeProcessAdapter/AfterProcessAdapter to transform data before/after JSON parsing:

   $pipeline
       ->addAdapter(new BeforeProcessAdapter(function($data) {
           return str_replace('"', "'", $data); // Pre-process
       }))
       ->addAdapter(new JsonAdapter());
   

3. Performance

   • Cache parsed JSON in AfterProcessAdapter if reprocessing identical inputs:

     $cache = [];
     $pipeline->addAdapter(new AfterProcessAdapter(function($data) use (&$cache) {
         $cache[md5($data)] = $data;
         return $data;
     }));