Yammi Jobs Monitoring Laravel Laravel Package

Technical Evaluation

Architecture Fit

• Observability Layer: Fits seamlessly into Laravel’s queue ecosystem as a non-intrusive monitoring layer, complementing existing job processing without modifying core logic.
• Multi-Driver Support: Aligns with Laravel’s queue abstraction (Redis, SQS, DB, sync), reducing vendor lock-in and ensuring broad applicability.
• Event-Driven Design: Leverages Laravel’s event system (e.g., JobProcessed, JobFailed) for minimal overhead, avoiding custom hooks or middleware.
• Alerting Integration: Extends observability beyond dashboards to actionable alerts (Slack, PagerDuty), bridging the gap between monitoring and incident response.

Integration Feasibility

• Low Friction: Requires only:
  1. Composer install.
  2. Database migration.
  3. Service provider registration.
  4. Optional: Alert configuration (e.g., Slack webhook).
• Backward Compatibility: No breaking changes to existing job classes or queue configurations; works with vanilla Laravel queues.
• Customization Points:
  • Job metadata enrichment via YammiJob trait.
  • Alert thresholds (e.g., retry limits, duration anomalies).
  • Dead-letter queue (DLQ) handling (e.g., auto-purging, manual review).

Technical Risk

Key Questions

1. Current Observability Gaps:
   • Are jobs currently monitored via logs, third-party tools (e.g., Datadog), or none?
   • What are the top 3 pain points in job reliability (e.g., silent failures, retry storms)?
2. Alerting Requirements:
   • Preferred channels (Slack, PagerDuty, etc.) and escalation policies.
   • Desired alert severity thresholds (e.g., "fail after 3 retries" vs. "fail after 1 retry for critical jobs").
3. DLQ Strategy:
   • Should failed jobs be automatically purged after X days, or manually reviewed?
   • Integration with existing error-tracking systems (e.g., Sentry).
4. Scaling Assumptions:
   • Expected peak queue depth and job volume (e.g., 100K jobs/month).
   • Worker concurrency and distribution (single region vs. multi-region).
5. Compliance/Retention:
   • Data retention policies for job logs (e.g., 30 days vs. indefinite).
   • GDPR/privacy considerations for job payloads (e.g., PII in job data).

Integration Approach

Stack Fit

• Laravel Ecosystem: Native support for Laravel 10.x/11.x; leverages existing:
  • Queue workers (php artisan queue:work).
  • Event system (illuminate/queue events).
  • Database (MySQL/PostgreSQL/SQLite) for metadata storage.
• Queue Drivers:
  • Redis/SQS: Minimal overhead; uses driver-specific listeners.
  • Database/Sync: Requires job payload serialization (handled by Yammi).
• Alerting Stack:
  • Slack: Webhook integration.
  • PagerDuty/Opsgenie: API-based alerts.
  • Custom: Webhook support for internal systems (e.g., Grafana alerts).

Migration Path

1. Discovery Phase (1–2 weeks):
   • Audit existing job classes for observability gaps.
   • Define alerting SLAs and DLQ policies.
2. Pilot Deployment (2–3 weeks):
   • Install Yammi in a staging environment.
   • Tag a subset of non-critical jobs with YammiJob trait.
   • Validate dashboard accuracy and alert relevance.
3. Full Rollout (1 week):
   • Apply YammiJob to all critical jobs.
   • Configure alert channels and thresholds.
   • Set up DLQ review process (manual/automated).
4. Optimization (Ongoing):
   • Tune yammi.job_timeout and anomaly detection.
   • Adjust database indexes if query performance degrades.

Compatibility

• Laravel Versions: Tested with 10.x/11.x; verify compatibility with custom queue drivers or forks.
• PHP Extensions: Requires pdo, mbstring, and driver-specific extensions (e.g., redis for Redis queue).
• Third-Party Conflicts:
  • Check for overlapping event listeners (e.g., other job monitoring tools).
  • Ensure no conflicts with existing DLQ implementations.
• Multi-Environment:
  • Configure .env per environment (e.g., YAMMI_ALERT_SLACK_WEBHOOK).
  • Disable alerts in non-production (e.g., YAMMI_ALERT_ENABLED=false).

Sequencing

1. Prerequisites:
   • Laravel app with queue workers already configured.
   • Database with sufficient storage for job metadata (estimate: ~1KB/job).
2. Core Setup:

   composer require romalytar/yammi-jobs-monitoring-laravel
   php artisan vendor:publish --provider="RomaLytar\Yammi\YammiServiceProvider"
   php artisan migrate
   

3. Configuration:
   • Update config/yammi.php for:
     • Queue drivers to monitor.
     • Alert channels.
     • DLQ settings.
4. Job Instrumentation:
   • Add use RomaLytar\Yammi\Traits\YammiJob; to critical job classes.
   • Example:

     class ProcessOrder implements ShouldQueue
     {
         use YammiJob; // Auto-instrumented
         // ...
     }
     

5. Validation:
   • Trigger test jobs and verify dashboard visibility.
   • Simulate failures to test alerts.
6. Go-Live:
   • Monitor worker logs for Yammi-related errors.
   • Gradually enable alerts in production.

Operational Impact

Maintenance

• Database Schema:
  • Yammi adds ~5 tables (jobs, failures, alerts, etc.). Plan for:
    • Regular backups of the yammi_jobs table.
    • Index optimization if queries slow (e.g., created_at, queue).
  • Migration strategy: Use Laravel’s schema builder for future updates.
• Configuration Drift:
  • Centralize yammi.php in a config management tool (e.g., Ansible, Terraform).
  • Document alert thresholds and DLQ policies in runbooks.
• Dependencies:
  • Monitor for Yammi updates (e.g., Laravel 12 compatibility).
  • Pin Composer version in composer.json if stability is critical.

Support

• Troubleshooting:
  • Common issues:
    • Missing job data (check YammiJob trait usage).
    • Alert storms (adjust thresholds or add rate-limiting).
    • Dashboard lag (optimize database queries or worker heartbeats).
  • Debugging tools:
    • php artisan yammi:clear (reset test data).
    • Yammi::ignore() to exclude specific jobs.
• SLA Impact:
  • Alerts may increase incident response volume; prioritize critical jobs.
  • Define on-call rotation for Yammi-related alerts.
• User Training:
  • Train DevOps/SRE teams on:
    • Dashboard navigation (e.g., filtering by job status).
    • DLQ triage workflows.
    • Alert acknowledgment procedures.

Scaling

• Horizontal Scaling:
  • Yammi metadata is read-heavy; consider:
    • Database read replicas for dashboard queries.
    • Caching frequent queries (e.g., Redis for job stats).
  • Worker scaling: Yammi adds minimal overhead (~5–10ms/job for metadata logging).
• Performance Bottlenecks:
  • Database: Monitor yammi_jobs table growth; archive old data if needed.
  • Alerts: Rate-limit webhook calls to avoid channel throttling.
  • Heartbeats: Adjust yammi.worker_heartbeat if workers are distributed across regions.
• Multi-Region:
  • Replicate Yammi database across regions for global observability.
  • Use a centralized alerting system (e.g., PagerDuty) to avoid regional alert silos.

Failure Modes