Dark Redis List Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require cursedcoder/dark-redis-list-bundle
   

   Add to bundles.php:

   return [
       // ...
       CursedCoder\DarkRedisListBundle\CursedCoderDarkRedisListBundle::class => ['all' => true],
   ];
   

2. Configure Redis: Add to config/packages/cursed_coder_dark_redis_list.yaml:

   cursed_coder_dark_redis_list:
       redis:
           host: '127.0.0.1'
           port: 6379
           db: 0
   

3. First Use Case: Define a list in a Doctrine entity (e.g., Post):

   use CursedCoder\DarkRedisListBundle\Model\RedisListInterface;
   
   class Post implements RedisListInterface
   {
       // ...
       public function getRedisListName(): string
       {
           return 'viewed_posts'; // List name in Redis
       }
   }
   

4. Add Items to Redis: Use the RedisListManager service:

   $redisListManager = $container->get('cursed_coder_dark_redis_list.manager');
   $redisListManager->addToList($post, 'viewed_posts'); // Format: "Bundle:Entity;id"
   

Implementation Patterns

Workflows

1. Hybrid Storage:

   • Store metadata (e.g., created_at, title) in Doctrine.
   • Offload list operations (e.g., sorting, pagination) to Redis.
   • Example: Cache a "trending posts" list updated via Redis pub/sub.

2. Real-Time Updates:

   • Use Redis pub/sub to trigger list updates when entities change:

     $redisListManager->publishUpdate('viewed_posts', $post);
     

   • Subscribe to updates in a worker:

     $redis->subscribe(['viewed_posts'], function ($message) {
         // Rebuild list logic here
     });
     

3. Multi-Entity Lists:

   • Combine unrelated entities (e.g., Post, Article) into one Redis list:

     $redisListManager->addToList($post, 'global_feed');
     $redisListManager->addToList($article, 'global_feed');
     

   • Retrieve with a custom query:

     $items = $redisListManager->getList('global_feed', 0, 10); // Paginated
     

4. Permanent Caching:

   • Cache lists for a duration (e.g., 24h) with TTL:

     # config/packages/cursed_coder_dark_redis_list.yaml
     cursed_coder_dark_redis_list:
         ttl: 86400 # 24 hours
     

   • Manually refresh via:

     $redisListManager->touchList('viewed_posts'); // Reset TTL
     

Integration Tips

• Doctrine Events: Listen to postPersist/postUpdate to auto-populate Redis lists:

  $entityManager->getEventManager()->addEventListener(
      Doctrine\ORM\Events::postPersist,
      function ($event) {
          $entity = $event->getObject();
          if ($entity instanceof RedisListInterface) {
              $this->redisListManager->addToList($entity, $entity->getRedisListName());
          }
      }
  );
  

• Symfony Cache: Use Symfony’s cache system to store serialized Redis lists locally for read-heavy apps:

  $cache = $container->get('cache.app');
  $cache->set('viewed_posts', $redisListManager->getList('viewed_posts'), 3600);
  

• API Layer: Expose Redis lists via API with pagination:

  // src/Controller/FeedController.php
  public function getFeed(Request $request, RedisListManager $manager)
  {
      $page = $request->query->getInt('page', 1);
      $items = $manager->getList('global_feed', ($page - 1) * 10, 10);
      return $this->json($items);
  }
  

Gotchas and Tips

Pitfalls

1. ID Collisions:

   • Redis lists use auto-incrementing IDs (1, 2, 3...). If you delete items, gaps appear but are not reused.
   • Fix: Use getList() with offset/limit to skip gaps, or manually compact lists with:

     $redisListManager->compactList('viewed_posts');
     

2. Data Corruption:

   • If Redis crashes, lists may become inconsistent. Always back up Redis data.
   • Tip: Use Redis persistence (RDB/AOF) and monitor with redis-cli --stat.

3. Memory Bloat:

   • Storing large entities (e.g., full texts) in Redis lists inflates memory usage.
   • Fix: Store only references (e.g., Bundle:Entity;id) and fetch full entities from Doctrine.

4. Race Conditions:

   • Concurrent writes to the same list can cause duplicates or lost updates.
   • Fix: Use Redis transactions or Lua scripts for atomic operations:

     $redis->transaction(function ($tx) use ($listName, $entity) {
         $tx->lPush($listName, $entity->getRedisKey());
         $tx->expire($listName, 86400);
     });
     

5. Symfony Cache Invalidation:

   • If you cache lists in Symfony’s cache, manually invalidate them on entity updates:

     $cache->delete('viewed_posts');
     

Debugging

• Check Redis Data: Inspect lists directly with:

  redis-cli HGETALL viewed_posts
  

  Or use a GUI like RedisInsight.

• Log Redis Operations: Enable debug mode in config:

  cursed_coder_dark_redis_list:
      debug: true
  

  Logs will appear in var/log/dev.log.

• Test Locally: Use Docker for Redis:

  docker run --name redis -p 6379:6379 -d redis
  

Extension Points

1. Custom List Formats: Extend the RedisListInterface to support nested data:

   class Post implements RedisListInterface
   {
       public function getRedisListValue(): array
       {
           return [
               'id' => $this->id,
               'title' => $this->title,
               'score' => $this->getEngagementScore(),
           ];
       }
   }
   

2. Add Indexes: Use Redis sorted sets (ZSET) for indexed lists (e.g., by created_at):

   $redis->zAdd('viewed_posts:sorted', $post->getCreatedAt()->getTimestamp(), $post->getRedisKey());
   

3. Event Dispatching: Trigger Symfony events when lists are updated:

   $dispatcher->dispatch(new ListUpdatedEvent($listName, $entity));
   

4. Fallback to Doctrine: Implement a fallback to Doctrine queries if Redis fails:

   try {
       $items = $redisListManager->getList($listName, $offset, $limit);
   } catch (\Exception $e) {
       $items = $doctrine->getRepository(Post::class)->findBy([], [], $limit, $offset);
   }