Skill Validator Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require stolt/skill-validator
   

2. First Validation:

   use Stolt\Ai\Skill\Validator;
   
   $validator = new Validator();
   $result = $validator->validateFile(base_path('skills/example/SKILL.md'));
   

3. Check Validity:

   if ($result->isInvalid()) {
       foreach ($result->errors() as $error) {
           Log::error($error);
       }
   }
   

First Use Case: CI/CD Validation

Add this to a Laravel app/Console/Commands/ValidateSkills.php:

public function handle() {
    $validator = new Validator();
    $results = $validator->validateFromDirectory(base_path('skills'));

    foreach ($results as $path => $result) {
        if ($result->isInvalid()) {
            $this->error("Invalid skill at {$path}:");
            foreach ($result->errors() as $error) {
                $this->line("  - {$error}");
            }
            return 1;
        }
    }
    $this->info('All skills validated successfully!');
}

Register the command in app/Console/Kernel.php and run with:

php artisan validate:skills

Implementation Patterns

Core Workflows

1. File Validation Pipeline:

   // Single file
   $result = $validator->validateFile($path);
   
   // Directory scan (recursive)
   $results = $validator->validateFromDirectory($directory);
   

2. Content Validation:

   $rawContent = file_get_contents($path);
   $result = $validator->validateContent($rawContent);
   

3. Programmatic Validation:

   $skillMd = SkillMd::create('name', 'desc', '# Instructions', ['tags' => ['php']]);
   $result = $validator->validateSkillMd($skillMd);
   

Laravel Integration Patterns

1. Service Provider Binding:

   // app/Providers/AppServiceProvider.php
   public function register() {
       $this->app->singleton(Validator::class);
   }
   

2. Request Validation Middleware:

   // app/Http/Middleware/ValidateSkillUpload.php
   public function handle(Request $request, Closure $next) {
       $validator = app(Validator::class);
       $result = $validator->validateContent($request->skill_content);
   
       if ($result->isInvalid()) {
           throw new \Exception('Invalid skill: ' . implode(', ', $result->errors()));
       }
       return $next($request);
   }
   

3. Model Events:

   // app/Models/Skill.php
   protected static function booted() {
       static::saved(function ($skill) {
           $validator = new Validator();
           $result = $validator->validateContent($skill->content);
           if ($result->isInvalid()) {
               throw new \Exception('Validation failed: ' . implode(', ', $result->errors()));
           }
       });
   }
   

Advanced Patterns

1. Validation Result Caching:

   $cacheKey = 'skill_validation_' . md5($path);
   $result = Cache::remember($cacheKey, now()->addHours(1), function() use ($validator, $path) {
       return $validator->validateFile($path);
   });
   

2. Skill Metadata Extraction:

   $metadata = $result->metadata();
   $tags = $metadata->get('tags', []);
   $effort = $metadata->get('effort');
   

3. Round-Trip Validation:

   $result = $validator->validateContent($rawContent);
   $skillMd = $result->toSkillMd();
   
   // Modify skillMd...
   $skillMd->setDescription('Updated description');
   
   // Re-validate
   $revalidated = $validator->validateSkillMd($skillMd);
   

Gotchas and Tips

Common Pitfalls

1. File Path Handling:

   • validateFromDirectory() uses absolute paths as keys in the result array. Normalize paths before use:

     $results = $validator->validateFromDirectory($directory);
     $relativePath = str_replace(base_path(), '', $filePath);
     

2. YAML Parsing Quirks:

   • The validator expects strict YAML frontmatter (e.g., tags: must be a list, not a string). Common errors:

     # Invalid (string instead of list)
     tags: php,laravel
     
     # Valid
     tags:
       - php
       - laravel
     

3. Name Validation:

   • Skill names must be lowercase, hyphenated (e.g., code-review). Names like CodeReview or code_review will fail.

4. Empty Markdown Body:

   • The validator requires non-empty Markdown content after the YAML frontmatter. A file with only --- and frontmatter will fail.

5. Boolean Fields:

   • Boolean fields like disable-model-invocation must use YAML boolean syntax:

     # Valid
     disable-model-invocation: true
     
     # Invalid (string)
     disable-model-invocation: "true"
     

Debugging Tips

1. Inspect Raw Metadata:

   $rawMetadata = $result->rawMetadata();
   dump($rawMetadata); // Debug YAML parsing issues
   

2. Validate Incrementally:

   • Use validateSkillMd() to test individual SkillMd objects after modifications:

     $skillMd = $result->toSkillMd();
     $skillMd->setTags(['php', 'testing']);
     $revalidated = $validator->validateSkillMd($skillMd);
     

3. Check for Hidden Characters:

   • Invalid UTF-8 or BOM characters in SKILL.md files can cause parsing failures. Use:

     $content = file_get_contents($path);
     $content = preg_replace('/^\xEF\xBB\xBF/', '', $content); // Remove BOM
     

Extension Points

1. Custom Validation Rules:

   • Extend the validator by creating a custom class:

     use Stolt\Ai\Skill\Validator;
     
     class CustomValidator extends Validator {
         public function validateCustomRule($skillMd) {
             if ($skillMd->name() === 'disallowed-skill') {
                 return ['Custom rule failed: Skill name disallowed'];
             }
             return null;
         }
     }
     

2. Override Default Rules:

   • The validator uses stolt/skill-md for parsing. Override its behavior by replacing the dependency:

     $validator = new Validator();
     $validator->setSkillMdFactory(function() {
         return new CustomSkillMd(); // Your implementation
     });
     

3. Add Pre/Post-Validation Hooks:

   $validator = new Validator();
   $validator->setPreValidationCallback(function($input) {
       if (is_string($input)) {
           return str_replace('{{placeholder}}', 'value', $input);
       }
       return $input;
   });
   

Configuration Quirks

1. Directory Traversal:

   • validateFromDirectory() follows symlinks by default. Disable with:

     $results = $validator->validateFromDirectory($directory, ['follow_symlinks' => false]);
     

2. Case Sensitivity:

   • The validator is case-sensitive for field names (e.g., Name: vs name:). Ensure YAML frontmatter uses lowercase keys.

3. Performance:

   • For large directories, use SplFileInfo filtering to exclude non-SKILL.md files:

     $iterator = new \RecursiveIteratorIterator(
         new \RecursiveDirectoryIterator($directory, \RecursiveDirectoryIterator::SKIP_DOTS),
         \RecursiveIteratorIterator::LEAVES_ONLY
     );
     $files = [];
     foreach ($iterator as $file) {
         if ($file->getExtension() === 'md' && $file->getBasename() === 'SKILL.md') {
             $files[] = $file->getPathname();
         }
     }
     $results = $validator->validateFiles($files);
     

Laravel-Specific Tips

1. Artisan Command Integration:

   • Use --path option for flexibility:

     protected $signature = 'validate:skills {--path=}';
     public function handle() {
         $path = $this->option('path') ?? base_path('skills');
         $validator = new Validator();
         $results = $validator->validateFromDirectory($path);
         // ...
     }
     

2. API Response Formatting:

   return response()->json([
       'valid' => !$result->isInvalid(),
       'errors' => $result->isInvalid() ? $result->errors() : [],
       'metadata' => $result->metadata()?->toArray(),
   ]);
   

3. Storage Integration:

   • Store validated skills in the database:

   $skill = Skill::updateOrCreate