Validation Guardian

Apex2's final validation tuning implemented as a Claude Code skill. Prevents false completion signals through thorough verification.

Instructions

When invoked (typically after claiming task completion), perform comprehensive validation to ensure the task is actually done correctly:

1. Parsing and Syntax Validation

Check that all generated/modified code is syntactically correct:

Python Code:

# Syntax check all Python files
find . -name "*.py" -exec python -m py_compile {} \;
# or more comprehensively
python -c "
import ast, sys
for file in ['file1.py', 'file2.py']:
    try:
        with open(file) as f:
            ast.parse(f.read())
        print(f"✓ {file}: Syntax OK")
    except SyntaxError as e:
        print(f"✗ {file}: Syntax Error - {e}")
"

JavaScript/TypeScript:

# Check syntax
find . -name "*.js" -exec node -c {} \;
# With ESLint if available
npx eslint . --ext .js,.ts

Configuration Files:

# JSON syntax
python -c "import json; [json.load(open(f)) for f in ['config1.json', 'config2.json']]"

# YAML syntax
python -c "import yaml; [yaml.safe_load(open(f)) for f in ['config.yaml', 'config.yml']]"

2. Execution Error Detection

Check for runtime errors and failed processes:

Log Analysis:

# Look for error indicators in common log patterns
grep -R -i "error\|exception\|failed\|traceback" . --include="*.log" --include="*.txt"

# Check for recent error exits
find . -name "*.py" -exec python {} \; 2>&1 | grep -i error

Process Status:

# Check if services are running that should be
ps aux | grep -E "(node|python|java)" | grep -v grep

# Check for crashed processes
journalctl -u service-name --since "5 minutes ago" | grep -i error

3. Test Result Validation

Ensure tests actually passed, not just were run:

Test Output Verification:

# Python pytest
pytest --tb=short -q | grep -E "(passed|failed|error)"

# JavaScript Jest
npm test 2>&1 | grep -E "(Tests:|✓|✗)"

# Look for failed assertions
grep -R "assert.*fail" test_results/

Coverage Reports:

# Check coverage meets minimum standards
pytest --cov --cov-report=term-missing | grep -E "TOTAL|missing"

4. Expected Output Verification

Confirm that required outputs, files, or results actually exist and match expectations:

File Existence and Content:

# Check required files exist
for file in expected_output.txt results.csv report.html; do
    if [[ -f "$file" ]]; then
        echo "✓ $file exists ($(wc -c < "$file") bytes)"
    else
        echo "✗ $file missing"
    fi
done

# Check file content is not empty (real validation)
find . -name "*.json" -exec sh -c 'if [ -s "$1" ]; then echo "✓ $1 has content"; else echo "✗ $1 empty"; done' _ {} \;

Database/Service Validation:

# Check database table creation
mysql -u user -ppassword database -e "SHOW TABLES LIKE 'expected_table';"

# Verify API endpoints respond
curl -f -s -o /dev/null -w "%{http_code}" http://localhost:3000/api/health

# Check service health status
curl -s http://localhost:8080/health | jq -r '.status'

5. Integration and System Validation

Verify the solution works in the larger context:

Integration Points:

# Check that modified code doesn't break existing functionality
python -m pytest tests/integration/ -v

# Verify package dependencies resolve
pip check

# Check imports work
python -c "
try:
    import new_module, existing_module
    print('✓ All imports successful')
except ImportError as e:
    print(f'✗ Import failed: {e}')
"

Performance Validation (when applicable):

# Check response times are reasonable
time curl http://localhost:3000/api/endpoint

# Verify memory usage is acceptable
free -h | head -4

# Check disk space usage is reasonable
du -sh ./

6. Documentation and Metadata Validation

Ensure supporting documentation is updated and consistent:

Documentation Consistency:

# Check README reflects current state
grep -E "(TODO|FIXME|placeholder)" README.md || echo "✓ No obvious placeholders"

# Verify API docs match implementation
# (specific to your documentation system)

Version and Build Info:

# Check version numbers consistent
grep -r "version=" package.json config.json requirements.txt

Validation Categories

Software Development Tasks

All files compile without syntax errors
Tests pass with expected coverage
Documentation is updated
Integration tests pass
No obvious performance regressions

Data Processing Tasks

Output files exist and have reasonable size
Data format is valid (JSON, CSV, etc.)
Row counts or record counts match expectations
No obvious data corruption or truncation

System Administration Tasks

Services are running with correct configuration
Configuration files are valid
Permissions are correctly set
Backup or rollback capabilities exist
Related services still function

Installation/Deployment Tasks

Application runs successfully
Health checks pass
Required ports are accessible
Dependent services are reachable
Load balancer/proxy configuration works

Validation Process

Define Expected Success Criteria
- What should exist when task is complete?
- What should happen when tested?
- What should NOT happen (errors, failures)?
Execute Validation Checks
- Run syntax and parsing checks
- Verify outputs and files
- Test functionality
- Check system/health status
Analyze Results
- Identify any failures or issues
- Determine root causes
- Categorize severity (blocking, warning, cosmetic)
Report Findings
- Clear pass/fail status
- Specific issues found
- Recommended fixes
- Overall quality assessment

Output Format

Provide comprehensive validation report:

Validation Results:
  Overall Status: [PASS/FAIL/PARTIAL]
  Validations Run: [number]
  Issues Found: [number]

Syntax and Parsing:
  Python files: [status] - [details]
  Config files: [status] - [details]
  Other files: [status] - [details]

Execution Status:
  Tests: [pass/fail] - [summary]
  Services: [running/stopped]
  Processes: [healthy/unhealthy]

Expected Outputs:
  Files created: [count] / [expected]
  Content validation: [status]
  Format validation: [status]

System Integration:
  API endpoints: [status]
  Database access: [status]
  Dependencies: [status]

Issues Identified:
  Critical: [blocking issues]
  Warnings: [potential problems]
  Informational: [minor concerns]

Recommendations:
  Immediate: [actions to fix critical issues]
  Optional: [suggested improvements]
  Monitoring: [what to watch going forward]

Final Assessment:
  Task Complete: [YES/NO/CONDITIONAL]
  Confidence Level: [high/medium/low]
  Next Steps: [what happens next]

When to Use

This validator should be invoked: After claiming task completion:

Always validate major code changes
Verify installation/deployment success
Confirm data processing completed correctly
Check system administration changes

Before declaring victory:

When dealing with complex, multi-step tasks
When changes affect multiple components
When success criteria are subjective
When consequences of failure are significant

The goal is to build confidence that tasks are actually completed, not just appeared to complete. prevents false positives and ensures reliable deliverables.

validation-guardian