correlation-regime

# Correlation-Regime Detection and Crisis Attribution

## Overview

The `correlation-analysis` skill answers *"how correlated are these assets?"* — a snapshot.
This skill answers the temporal questions a snapshot cannot:

1. **When** did the market fuse into one highly-correlated bloc, and when did it release?
   (Mode 1 — regime detection)
2. **What** does a fused regime mean for position sizing? (Mode 2 — risk context)
3. **Who** moved first when a crisis broke — is there a nameable trigger asset?
   (Mode 3 — first-mover attribution)
4. **Who** quietly rewired their relationship to the rest of the market, even without a
   violent move? (Mode 4 — rewiring leaderboard)

The methodology comes from an open-source streaming pipeline (see References)
whose public repository pins the regime machinery's math and an eight-event
historical replay regression (COVID, May-2021, China ban, Nov-2021 top, LUNA,
FTX, SVB, yen-carry — 17 crypto symbols, 1-minute bars) that its CI reproduces
bit-for-bit. The finer-grained numbers quoted in this skill — 13 fused/defused
regime cycles on the continuous 2020–2024 tape at a ~0.008/day calm false-alarm
rate, and zero wrong culprit names across 10 labeled crises (2 held
out-of-sample), including naming FTT roughly two days before the November 2022
collapse — are the **author's unpublished internal replays** on that same
pipeline and are not independently verifiable. All of it is historical replay,
never live results, and the method is market-agnostic even though the
validation tape is crypto.

**What this skill is NOT**: a trade-timing signal. The same validation program tested
regime-based exits head-to-head against a plain price stop and lost — correlation
regimes cannot time tops, and the give-up cost of selling into a crash is a property
of the tape, not of any signal. Use these modes for risk context, monitoring, and
post-hoc attribution; never present them as buy/sell triggers.

---

## Mode 1: Correlation-Regime Detection (Edge Density + Hysteresis)

**Use case**: Maintain a live, causal answer to "is the market currently one bloc?"
Diversification quietly disappears when pairwise correlations fuse; a regime state
machine turns that into an explicit, monitorable state with few false alarms.

### Workflow

```
1. Compute rolling-window pairwise correlations of returns
2. Reduce each correlation matrix to one number: edge density
   = fraction of asset pairs with |ρ| ≥ edge_threshold
3. Smooth the density series with a TRAILING window (causal — see warning below)
4. Run a hysteresis (Schmitt-trigger) state machine over the smoothed series:
   enter FUSED when density ≥ enter_threshold, exit only when ≤ exit_threshold
5. Emit regime state + transition timestamps for monitoring / reporting
```

Two thresholds with a dead band between them are the entire trick: a single
threshold chatters (fires dozens of times as density oscillates around it), while
hysteresis yields a handful of clean regime cycles per market cycle.

```python
import numpy as np
import pandas as pd


def compute_edge_density(
    returns: pd.DataFrame,
    corr_window: int = 60,
    edge_threshold: float = 0.5,
) -> pd.Series:
    """Reduce rolling correlation matrices to an edge-density series.

    Edge density is the fraction of distinct asset pairs whose rolling
    |correlation| clears ``edge_threshold`` — a scalar "how fused is the
    market" gauge in [0, 1].

    Args:
        returns: Multi-asset return matrix, columns are symbols
        corr_window: Rolling window length (bars) for pairwise correlation
        edge_threshold: |ρ| level at which a pair counts as an "edge"

    Returns:
        Edge-density series aligned to ``returns.index`` (NaN during warmup)
    """
    n_assets = returns.shape[1]
    n_pairs = n_assets * (n_assets - 1) // 2
    upper_mask = np.triu(np.ones((n_assets, n_assets), dtype=bool), k=1)

    density = pd.Series(np.nan, index=returns.index)
    for i in range(corr_window, len(returns) + 1):
        corr = returns.iloc[i - corr_window:i].corr().abs().to_numpy()
        density.iloc[i - 1] = float((corr[upper_mask] >= edge_threshold).sum()) / n_pairs
    return density


def detect_regimes(
    density: pd.Series,
    smooth_window: int = 5,
    enter_threshold: float = 0.65,
    exit_threshold: float = 0.45,
) -> pd.DataFrame:
    """Hysteresis (Schmitt-trigger) regime state machine on smoothed density.

    The market is FUSED once smoothed density reaches ``enter_threshold`` and
    stays FUSED until it falls back to ``exit_threshold``. The dead band
    between the two thresholds is what suppresses chatter.

    Args:
        density: Edge-density series from :func:`compute_edge_density`
        smooth_window: Trailing smoothing window (causal; never centered)
        enter_threshold: Density level that opens a FUSED regime
        exit_threshold: Density level that closes it (must be < enter_threshold)

    Returns:
        DataFrame with columns ``density``, ``smoothed``, ``fused`` (0/1)
    """
    if exit_threshold >= enter_threshold:
        raise ValueError("exit_threshold must be below enter_threshold")

    # Trailing mean = causal. A centered window here silently reads the future.
    smoothed = density.rolling(smooth_window, min_periods=1).mean()

    fused = False
    states = np.zeros(len(smoothed), dtype=int)
    for i, value in enumerate(smoothed.to_numpy()):
        if np.isnan(value):
            states[i] = int(fused)
            continue
        if not fused and value >= enter_threshold:
            fused = True
        elif fused and value <= exit_threshold:
            fused = False
        states[i] = int(fused)

    return pd.DataFrame(
        {"density": density, "smoothed": smoothed, "fused": states},
        index=density.index,
    )
```

### Threshold Selection Guide

| Parameter | Guidance |
|------|------|
| `edge_threshold` | 0.5 works for raw daily/intraday return correlations in crypto and equities. It does **not*