Data Generator Laravel Package

Getting Started

Minimal Steps

1. Install the Package:

   composer require rasel9w9/data-generator:^1.2
   

   Ensure PDO_OCI is enabled in your PHP environment if using Oracle 18c.

2. Register the Service Provider (Laravel <5.5): Add to config/app.php:

   'providers' => [
       rasel9w9\DataGenerator\DataGeneratorServiceProvider::class,
   ],
   

3. First Use Case: Export a specific table via Artisan:

   php artisan alauddin:generate-table-data --table=users
   

   Output: PHP array file saved to database/seeders/data/users (no extension).

Implementation Patterns

Core Workflows

1. Artisan Command Usage:

   • Export a single table:

     php artisan alauddin:generate-table-data --table=posts
     

   • Export all tables:

     php artisan alauddin:generate-data
     

2. Controller Integration:

   use rasel9w9\DataGenerator\GenerateData;
   
   class DataController extends Controller {
       public function exportTable($tableName) {
           $generator = new GenerateData();
           $generator->generateTableData($tableName);
           return response()->json(['status' => 'Exported to seeders/data']);
       }
   }
   

3. Multi-Database Handling:

   • MySQL/PostgreSQL: Use default Laravel connections.
   • Oracle 18c: Ensure config/database.php includes an Oracle connection:

     'oracle' => [
         'driver' => 'oracle',
         'host' => env('ORACLE_HOST'),
         'database' => env('ORACLE_DB'),
         'username' => env('ORACLE_USER'),
         'password' => env('ORACLE_PASSWORD'),
         'prefix' => '',
         'charset' => 'utf8',
         'prefix_indexes' => true,
         'strict' => true,
         'engine' => null,
     ],
     

   • Pass connection name explicitly (if supported in future updates):

     $generator->generateTableData('users', 'oracle');
     

4. Automated Scheduling: Add to app/Console/Kernel.php:

   protected function schedule(Schedule $schedule) {
       $schedule->command('alauddin:generate-data')->dailyAt('2:00');
   }
   

Integration Tips

• Validation: Pre-check table existence:

  if (!Schema::hasTable($tableName)) {
      throw new \InvalidArgumentException("Table {$tableName} does not exist.");
  }
  

• Custom Output Paths: Override the default database/seeders/data by extending the GenerateData class:

  class CustomGenerator extends GenerateData {
      protected $outputPath = 'custom/path/';
  }
  

• Data Transformation: Hook into the data array before saving (e.g., sanitize or obfuscate):

  $data = $generator->generateTableData($tableName);
  array_walk($data, function(&$row) {
      $row['password'] = '*****'; // Example: Mask sensitive fields
  });
  file_put_contents("{$this->outputPath}{$tableName}", '<?php return ' . var_export($data, true) . ';');
  

Gotchas and Tips

Pitfalls

1. Oracle-Specific Quirks:

   • Case Sensitivity: Oracle table/column names are case-sensitive unless quoted. Ensure consistency in queries.
   • Data Type Mismatches: Oracle types like CLOB or TIMESTAMP WITH TIME ZONE may not map cleanly to Laravel’s default types. Handle with:

     $generator->setTypeMapper([
         'oracle.clob' => 'string',
         'oracle.timestamp' => 'datetime',
     ]);
     

   • Connection Issues: Oracle exports may fail silently if the PDO_OCI extension is missing. Verify with:

     php -m | grep pdo_oci
     

2. Performance Bottlenecks:

   • Large tables (>100K rows) may cause memory issues. Use chunking:

     $generator->setChunkSize(5000); // Process 5K rows at a time
     

   • Oracle exports may lock tables. Run during low-traffic periods or use read-only connections.

3. File Overwrites: The package overwrites files by default. Mitigate by:

   • Appending timestamps to filenames:

     $filename = "{$tableName}_" . now()->format('YmdHis');
     

   • Using file_exists checks before writing.

4. Laravel Version Conflicts:

   • Tested on Laravel 6–10. If using Laravel 11+, check for breaking changes in the package’s composer.json dependencies.

Debugging Tips

1. Enable Query Logging: Add to config/database.php:

   'log' => true,
   'log_queries' => true,
   

   Check storage/logs/laravel.log for failed queries.

2. Validate Output: Compare row counts between source and exported data:

   $sourceCount = DB::table($tableName)->count();
   $exportedCount = count($generator->generateTableData($tableName));
   if ($sourceCount !== $exportedCount) {
       throw new \RuntimeException("Row count mismatch for {$tableName}!");
   }
   

3. Oracle-Specific Errors:

   • ORA-01031: Insufficient privileges. Grant SELECT on the table:

     GRANT SELECT ON your_table TO your_laravel_user;
     

   • ORA-00942: Table or view does not exist. Verify the table name and schema.

Extension Points

1. Custom Exporters: Extend the GenerateData class to support additional databases or formats (e.g., CSV):

   class CsvGenerator extends GenerateData {
       public function generateCsv($tableName, $filePath) {
           $data = $this->fetchTableData($tableName);
           $csv = fopen($filePath, 'w');
           foreach ($data as $row) {
               fputcsv($csv, $row);
           }
           fclose($csv);
       }
   }
   

2. Pre/Post-Export Hooks: Override methods to add logic:

   class HookedGenerator extends GenerateData {
       protected function beforeExport($tableName) {
           // Example: Log start time
           Log::info("Exporting {$tableName} at " . now());
       }
   
       protected function afterExport($tableName, $data) {
           // Example: Send notification
           Mail::to('admin@example.com')->send(new ExportComplete($tableName));
       }
   }
   

3. Dynamic Table Selection: Filter tables by prefix or regex:

   public function generateFilteredData($prefix = '') {
       $tables = Schema::getAllTables();
       $filtered = array_filter($tables, fn($table) => str_starts_with($table, $prefix));
       foreach ($filtered as $table) {
           $this->generateTableData($table);
       }
   }
   

4. Connection Management: Dynamically switch connections:

   public function generateTableData($tableName, $connection = null) {
       $this->connection = $connection ?? config('database.default');
       // ... rest of the logic
   }
   

Configuration Quirks

1. Default Connection: The package defaults to Laravel’s default database connection. Explicitly pass the connection name if needed (check future updates for support).

2. Output Directory Permissions: Ensure database/seeders/data is writable:

   mkdir -p database/seeders/data
   chmod -R 775 database/seeders/data
   

3. PHP Memory Limits: Increase memory_limit in php.ini for large exports:

   memory_limit = 512M
   

   Or set it dynamically:

   ini_set('memory_limit', '512M');
   

4. Time Zone Handling: Oracle timestamps may differ from Laravel’s default time zone. Normalize with:

   $data = array_map(function($row) {
       return array_map('DateTime::createFromFormat', $row, array_fill(0, count($row), 'Y-m-d H:i:s'));
   }, $data);