Artillery Load Testing

# Artillery Load Testing

This skill makes an AI agent author Artillery 2.x load tests as declarative YAML: realistic traffic phases, multi-step scenarios with captured variables, CSV-driven virtual user data, functional `expect` checks inside load flows, and `ensure` thresholds that turn latency regressions into red CI builds. Trigger it when a project contains `artillery.yml`, `artillery` in package.json, or when the user asks for load, stress, soak, or spike testing of an HTTP API or website in a Node.js stack.

## Core Principles

1. **Model traffic in phases, never one flat rate.** Real load has a warm-up, a ramp, and a sustained plateau. A single `arrivalRate` hides cold-start effects and autoscaling lag. Always define at least warm-up, ramp, and sustain phases.
2. **`arrivalRate` is new virtual users per second, not concurrency.** Each arriving VU runs the whole scenario. If your scenario takes 10 seconds and you arrive 50/sec, you have roughly 500 concurrent users. Calculate this before picking numbers.
3. **Thresholds belong in the test file, not in a wiki.** Use the `ensure` plugin so `artillery run` exits non-zero when p95/p99 or error rate budgets are blown. A load test that cannot fail is a demo, not a test.
4. **Assert correctness under load with `expect`.** A server returning 200 with an empty body at p99 latency is still broken. Check `statusCode`, `contentType`, and `hasProperty` inside the flow.
5. **Parameterize virtual users from CSV payloads.** Hammering one account exercises one cache line and one DB row. Use `payload` with hundreds of distinct credentials and SKUs to defeat caches realistically.
6. **Never load test production without a plan; never load test third parties at all.** Point Artillery at a staging environment sized like production, announce test windows, and rate-limit anything that crosses a vendor boundary.

## Setup

```bash
npm install --save-dev artillery@latest
npx artillery version

# Run a test locally
npx artillery run load/checkout.yml

# Save raw metrics for trend analysis
npx artillery run load/checkout.yml --output artillery-report.json
```

## Patterns

### 1. Phases and a multi-step scenario with capture

```yaml
# load/checkout.yml
config:
  target: https://staging-api.example.com
  http:
    timeout: 10
  phases:
    - duration: 60
      arrivalRate: 2
      name: warm-up
    - duration: 120
      arrivalRate: 5
      rampTo: 40
      name: ramp-to-peak
    - duration: 300
      arrivalRate: 40
      name: sustained-peak
  payload:
    path: users.csv
    fields:
      - email
      - password
    order: random
    skipHeader: true
  plugins:
    expect: {}
    ensure: {}
  ensure:
    thresholds:
      - http.response_time.p95: 250
      - http.response_time.p99: 500
    conditions:
      - expression: http.codes.200 > 0
        strict: true
    maxErrorRate: 1

scenarios:
  - name: login-browse-order
    flow:
      - post:
          url: /auth/login
          json:
            email: '{{ email }}'
            password: '{{ password }}'
          capture:
            - json: $.token
              as: authToken
          expect:
            - statusCode: 200
            - contentType: json
            - hasProperty: token
      - get:
          url: /products?category=audio
          headers:
            Authorization: 'Bearer {{ authToken }}'
          capture:
            - json: $[0].id
              as: productId
          expect:
            - statusCode: 200
      - post:
          url: /orders
          headers:
            Authorization: 'Bearer {{ authToken }}'
          json:
            productId: '{{ productId }}'
            quantity: 1
          expect:
            - statusCode: 201
            - hasProperty: orderId
      - think: 2
```

### 2. CSV payload file

```csv
email,password
loadtest-001@example.com,Str0ngPass!001
loadtest-002@example.com,Str0ngPass!002
loadtest-003@example.com,Str0ngPass!003
loadtest-004@example.com,Str0ngPass!004
```

Generate hundreds of rows with a one-liner instead of writing them by hand:

```bash
seq -w 1 500 | awk -F, 'BEGIN{print "email,password"} {printf "loadtest-%s@example.com,Str0ngPass!%s\n", $1, $1}' > users.csv
```

### 3. Custom logic with a JavaScript processor

```yaml
# In config:
config:
  target: https://staging-api.example.com
  processor: ./processor.js

scenarios:
  - name: create-order-with-dynamic-payload
    flow:
      - function: generateOrderPayload
      - post:
          url: /orders
          json:
            sku: '{{ sku }}'
            quantity: '{{ quantity }}'
          afterResponse: logSlowResponse
```

```js
// processor.js
module.exports = { generateOrderPayload, logSlowResponse };

function generateOrderPayload(context, events, done) {
  // Runs before the request; sets template variables on the VU context
  context.vars.sku = `SKU-${1000 + Math.floor(Math.random() * 9000)}`;
  context.vars.quantity = 1 + Math.floor(Math.random() * 4);
  return done();
}

function logSlowResponse(requestParams, response, context, events, done) {
  const tookMs = response.timings ? response.timings.phases.total : 0;
  if (tookMs > 1000) {
    console.warn(`SLOW ${requestParams.url} -> ${response.statusCode} in ${tookMs}ms`);
    events.emit('counter', 'custom.slow_responses', 1);
  }
  return done();
}
```

### 4. CI gate in GitHub Actions

The `ensure` plugin makes Artillery exit with code 1 on any threshold breach, so the job fails without extra scripting.

```yaml
# .github/workflows/load-test.yml
name: load-test
on:
  workflow_dispatch:
  schedule:
    - cron: '0 2 * * 1'

jobs:
  artillery:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - name: Run load test against staging
        run: npx artillery run load/checkout.yml --output artillery-report.json
        env:
          ARTILLERY_DISABLE_TELEMETRY: 'true'