Agentic Testing Patterns

# Agentic Testing Patterns Skill

You are an expert in agentic testing methodology where AI agents autonomously plan, generate, execute, and maintain test suites. When the user asks you to implement agentic testing workflows, create autonomous test pipelines, or build AI-driven quality assurance systems, follow these detailed instructions.

## Core Principles

1. **Agent autonomy with human oversight** -- Agents should operate independently for routine tasks but escalate to humans for ambiguous requirements, security-sensitive tests, and novel failure patterns.
2. **Intent-driven test specification** -- Tests are specified as high-level intents (what to verify) rather than step-by-step instructions. Agents determine the optimal implementation strategy.
3. **Continuous learning from failures** -- Every test failure feeds back into the agent's knowledge base. Agents improve their test generation and maintenance strategies over time.
4. **Risk-based test prioritization** -- Agents analyze code changes, historical failure data, and business impact to determine which tests to run and in what order.
5. **Multi-agent collaboration** -- Different agents specialize in different tasks: one plans, one generates, one executes, one analyzes results. They communicate through structured protocols.
6. **Observable and auditable** -- Every agent decision must be logged with reasoning. Humans must be able to trace why a test was generated, modified, or skipped.
7. **Graceful degradation** -- When AI services are unavailable, the system falls back to deterministic test execution. Agent-generated tests must be valid standalone tests.

## Project Structure

```
agentic-tests/
  agents/
    coordinator/
      coordinator-agent.ts
      task-queue.ts
      priority-engine.ts
    analyzer/
      code-change-analyzer.ts
      failure-pattern-detector.ts
      coverage-gap-finder.ts
    generator/
      test-generator.ts
      fixture-generator.ts
      mock-generator.ts
    executor/
      test-runner.ts
      parallel-executor.ts
      result-collector.ts
    reporter/
      insight-generator.ts
      trend-analyzer.ts
      alert-system.ts
  knowledge/
    failure-patterns.json
    selector-mappings.json
    test-templates/
      unit-template.ts
      integration-template.ts
      e2e-template.ts
  pipelines/
    ci-pipeline.ts
    pr-review-pipeline.ts
    nightly-pipeline.ts
    deployment-pipeline.ts
  config/
    agent-config.ts
    pipeline-config.ts
    model-config.ts
  tests/
    agent-tests/
      coordinator.test.ts
      analyzer.test.ts
      generator.test.ts
  monitoring/
    agent-metrics.ts
    cost-dashboard.ts
    quality-tracker.ts
```

## Coordinator Agent

```typescript
// agentic-tests/agents/coordinator/coordinator-agent.ts
import { CodeChangeAnalyzer, ChangeAnalysis } from '../analyzer/code-change-analyzer';
import { TestGenerator } from '../generator/test-generator';
import { TestExecutor, ExecutionResult } from '../executor/test-runner';
import { InsightGenerator } from '../reporter/insight-generator';

export interface AgenticTask {
  id: string;
  type: 'generate' | 'execute' | 'heal' | 'analyze' | 'report';
  priority: number;
  payload: Record<string, unknown>;
  status: 'pending' | 'running' | 'completed' | 'failed';
  assignedAgent?: string;
  result?: unknown;
  createdAt: string;
  completedAt?: string;
}

export interface CoordinatorConfig {
  maxConcurrentTasks: number;
  maxBudgetPerPipeline: number;
  escalationThreshold: number;
  autoApproveConfidence: number;
}

export class CoordinatorAgent {
  private taskQueue: AgenticTask[] = [];
  private analyzer: CodeChangeAnalyzer;
  private generator: TestGenerator;
  private executor: TestExecutor;
  private reporter: InsightGenerator;
  private config: CoordinatorConfig;

  constructor(config: Partial<CoordinatorConfig> = {}) {
    this.analyzer = new CodeChangeAnalyzer();
    this.generator = new TestGenerator();
    this.executor = new TestExecutor();
    this.reporter = new InsightGenerator();
    this.config = {
      maxConcurrentTasks: 5,
      maxBudgetPerPipeline: 20.0,
      escalationThreshold: 0.6,
      autoApproveConfidence: 0.85,
      ...config,
    };
  }

  async runPipeline(trigger: PipelineTrigger): Promise<PipelineResult> {
    const startTime = Date.now();
    const tasks: AgenticTask[] = [];

    // Phase 1: Analyze what changed
    const analysis = await this.analyzer.analyzeChanges(trigger.changes);

    // Phase 2: Determine what tests to generate or update
    const testPlan = this.createTestPlan(analysis, trigger);

    // Phase 3: Generate new tests for uncovered changes
    for (const gap of testPlan.coverageGaps) {
      const task = this.createTask('generate', { gap, analysis });
      tasks.push(task);
      const generated = await this.generator.generateForGap(gap);
      task.result = generated;
      task.status = 'completed';
    }

    // Phase 4: Execute all relevant tests
    const testFiles = this.selectTestsToRun(analysis, testPlan);
    const executionResult = await this.executor.runTests(testFiles);

    // Phase 5: Analyze results and generate report
    const insights = await this.reporter.generateInsights(executionResult, analysis);

    // Phase 6: Handle failures
    if (executionResult.failures.length > 0) {
      await this.handleFailures(executionResult.failures);
    }

    return {
      duration: Date.now() - startTime,
      testsRun: executionResult.total,
      testsPassed: executionResult.passed,
      testsFailed: executionResult.failures.length,
      testsGenerated: testPlan.coverageGaps.length,
      insights,
      tasks,
    };
  }

  private createTestPlan(analysis: ChangeAnalysis, trigger: PipelineTrigger): TestPlan {
    const coverageGaps = analysis.uncoveredChanges.map((change) => ({
      file: change.file,
      functions: change.functions,
      suggestedTestType: this.inferTestType(change),
      priority: this.calculatePriority(change),
    }));