matlab-write-performance-tests

# Writing MATLAB Performance Tests

Write performance tests for MATLAB code using the `matlab.perftest.TestCase` framework. This framework provides statistically rigorous timing with automatic warmup, multiple samples, and outlier handling.

## When to Use

- User wants to write repeatable performance tests for their MATLAB code
- User needs to benchmark functions with statistical confidence
- User wants to detect performance regressions over time
- User is setting up continuous performance monitoring
- User asks how to use `runperf` or `matlab.perftest.TestCase`

## When NOT to Use

- User wants a quick one-off timing (use `timeit` instead — see `matlab-optimize-performance`)
- User wants to optimize existing code (use `matlab-optimize-performance`)
- User wants to measure memory usage (use `matlab-optimize-memory`)
- User wants to profile code to find bottlenecks (use `matlab-optimize-performance`, Step 2)

## Framework: `matlab.perftest.TestCase`

All performance tests subclass `matlab.perftest.TestCase` and use measurement boundaries to control what gets timed.

### Basic Template

```matlab
classdef MyFeaturePerformanceTest < matlab.perftest.TestCase

    properties (MethodSetupParameter)
        DataSize = struct('Small', 100, 'Medium', 1000, 'Large', 10000)
    end

    properties
        inputData
    end

    methods (TestMethodSetup)
        function setupData(testCase, DataSize)
            % ALL setup outside the measurement boundary
            testCase.inputData = randn(DataSize, 1);
        end
    end

    methods (Test)
        function testMyFunction(testCase)
            data = testCase.inputData;
            while testCase.keepMeasuring
                result = myFunction(data);
            end
            testCase.verifyNotEmpty(result);
        end
    end
end
```

### Running Performance Tests

```matlab
% Run with statistical rigor (automatic sample size)
results = runperf('MyFeaturePerformanceTest');

% View results
disp(results)

% Fixed sample count (faster, less statistical power)
import matlab.perftest.TimeExperiment;
suite = testsuite('MyFeaturePerformanceTest');
experiment = TimeExperiment.withFixedSampleSize(4);
results = run(experiment, suite);
```

## Measurement Boundaries

The framework offers three ways to control what gets measured:

### 1. `keepMeasuring` — Needed when code is fast (<10ms)

Automatically loops the code until enough samples are collected. Required for sub-10ms operations to achieve statistical rigor; works at any speed but adds overhead for slower code where `startMeasuring`/`stopMeasuring` is preferred:

```matlab
function testFastFunction(testCase)
    data = testCase.inputData;
    while testCase.keepMeasuring
        result = fastFunction(data);
    end
    testCase.verifyNotEmpty(result);
end
```

### 2. `startMeasuring`/`stopMeasuring` — For precise control

Use when you need setup between iterations or want to exclude specific code:

```matlab
function testWithBoundaries(testCase)
    data = testCase.inputData;
    % Pre-computation (NOT measured)
    preparedData = preprocess(data);

    testCase.startMeasuring();
    result = functionUnderTest(preparedData);
    testCase.stopMeasuring();

    % Verification (NOT measured)
    testCase.verifyEqual(size(result), [100 1]);
end
```

### 3. No boundary — Entire method is measured

The whole `Test` method body is timed. Use only when the entire method IS the workload:

```matlab
function testSlowFunction(testCase, DataSize) %#ok<INUSD>
    data = testCase.inputData;
    result = slowFunction(data);
    testCase.verifyNotEmpty(result);
end
```

## Parameterization

Parameterize tests to measure across different input sizes or configurations.

### `MethodSetupParameter` — When setup uses the parameter

```matlab
properties (MethodSetupParameter)
    DataSize = struct('Small', 100, 'Medium', 1000, 'Large', 10000)
end

methods (TestMethodSetup)
    function setupData(testCase, DataSize)
        testCase.inputData = randn(DataSize, 1);
    end
end
```

### `TestParameter` — When only test methods use the parameter

```matlab
properties (TestParameter)
    Algorithm = {'chol', 'lu', 'qr'}
end

methods (Test)
    function testSolve(testCase, Algorithm)
        ...
    end
end
```

**Critical gotcha:** Do NOT use `TestParameter` for properties consumed by `TestMethodSetup`. MATLAB will error with "Define 'X' as a MethodSetupParameter." If your setup method needs the parameter, it must be `MethodSetupParameter`.

### Combining Both — Size in setup, algorithm in test

```matlab
properties (MethodSetupParameter)
    DataSize = struct('Small', 100, 'Medium', 1000, 'Large', 10000)
end

properties (TestParameter)
    Algorithm = {'chol', 'lu', 'qr'}
end

methods (TestMethodSetup)
    function setupData(testCase, DataSize)
        testCase.inputData = randn(DataSize);
    end
end

methods (Test)
    function testSolve(testCase, Algorithm)
        A = testCase.inputData' * testCase.inputData; % SPD matrix
        data = A;
        while testCase.keepMeasuring
            result = decomposition(data, Algorithm);
        end
        testCase.verifyNotEmpty(result);
    end
end
```

This produces 9 test points (3 sizes × 3 algorithms).

## Lean Setup — Critical Rule

ALL setup must be outside the measurement boundary:

| Setup Task | Where to Put It |
|-----------|-----------------|
| Data generation | `TestMethodSetup` |
| Loading files | `TestClassSetup` |
| Creating objects | `TestMethodSetup` or `TestClassSetup` |
| Path manipulation | `TestClassSetup` |
| RNG seeding | `TestMethodSetup` |

**Never** include setup/teardown inside the measured region. This inflates timing and adds noise.

**Copy properties to local variables before measuring.** Don't access `testCase.PropertyName` inside the measurement boundary — it measures the `matlab.perftest.TestCase` indexing overhead. Assign to a local variable outside the boundary instead:

```matlab
% Correct: local variable assigned before measurement