matlab-analyze-data

# MATLAB Data Analysis

Generate idiomatic MATLAB code for tabular data analysis tasks using tables and timetables.

## When to Use

- Any task involving tabular data: exploring, cleaning, transforming, or aggregating tables
- Time-series analysis: resampling, synchronizing, trend detection, smoothing
- Answering questions about data in tables (top-N, filtering, group comparisons)
- Data cleaning: missing values, outliers, type conversion, normalization

## When NOT to Use

- The task has no tabular data context (no tables, timetables, or structured datasets)
- The primary goal is visualization or plotting, not data analysis
- The task is purely symbolic math, simulation, or app building

This skill covers core MATLAB functions for tabular and time-series workflows. These functions work natively with `table` and `timetable`, handle missing data correctly, and are performance-optimized. Prefer the modern functions recommended here (e.g., `groupsummary`, `datetime`, `fillmissing`) over legacy alternatives (e.g., `accumarray`, `nanmean`, `datenum`). Override only if the user explicitly requests otherwise.

**Before writing code, read the reference file linked at the end of the relevant section below.** Reference files contain correct syntax, common pitfalls, and "Avoid" patterns that prevent silent bugs. Skipping the reference risks using a deprecated approach or hitting a known pitfall.

### Key Functions — Available From

Most functions in this skill are available in R2023a or earlier. The following require a newer release:

| Function | Available From | Purpose |
|----------|---------------|---------|
| `paddata`, `trimdata`, `resize` | R2023b | Pad, trim, or resize arrays to target length |
| `clip` | R2024a | Clamp values to a range |
| `summary` (enhanced) | R2024b | Supports arrays (numeric, datetime, duration, logical); adds `Statistics`, `DataVariables`, `Detail` name-value args |
| `isapprox` | R2024b | Tolerance-aware floating-point comparison (use instead of `==` for computed values) |
| `isbetween` (numeric) | R2024b | Check elements within a numeric range |
| `numunique` | R2025a | Count distinct values in a variable |
| `allbetween` | R2025a | Validate all values are within a range |
| `allunique` | R2025a | Validate all values are unique |

## Getting Oriented with Data

When data is already in a workspace variable, start by understanding its structure and contents. Use JSON output for reliable parsing — table display is designed for human-readable grids, but as text it is easy to misinterpret which values belong to which variables:

```matlab
jsonencode(summary(T))      % per-variable stats as nested struct
jsonencode(head(T))         % first 8 rows as structured JSON
```

**When getting oriented with unknown data, `summary(T)` already contains per-variable type, size, NumMissing, and — for numeric/datetime/duration — Min, Max, Mean, Median, Std. For categoricals it includes category names and counts.** This is usually sufficient for an initial overview.

If producing a standalone script, leave the semicolon off to invoke the `display` method — it shows dimensions, variable names, and a truncated preview. Avoid `disp` (omits headers and prints every row, flooding output on large tables) and `fprintf` in a loop (verbose, old-style):

```matlab
summary(T)                  % types, ranges, missing counts per variable
size(T)                     % [nRows, nVars]
sum(ismissing(T))           % missing count per variable
T                           % dimensions + header + truncated preview
```

**For Pearson correlation, use `corrcoef` (base MATLAB)** with `Rows="complete"` to handle NaN: `corrcoef(T{:,vartype("numeric")}, Rows="complete")`. For Kendall or Spearman rank correlation, use `corr` (requires Statistics Toolbox): `corr(X, Type="Spearman")`.

> Systematic exploration checklist (distributions, cardinality, duplicates, outlier screening, `groupcounts`, `corrcoef`, time-based checks): [exploration.md](references/exploration.md)

## Data Types

Use modern MATLAB types. These are faster, more readable, and work better with table functions.

| Instead of | Use | Why |
|---|---|---|
| `datenum`, `datestr` | `datetime` | Proper arithmetic, timezone support |
| `char`, `cellstr`, `strcmp` | `string`, `==`/`matches` | `==` for scalar, `matches` for vector comparison |
| Numeric codes or strings with few unique values | `categorical` | Self-documenting, works with grouping functions, memory-efficient |

```matlab
dt = datetime("2024-01-15",TimeZone="America/New_York");
names = ["Alice" "Bob"];           % not {'Alice', 'Bob'}
T.Status = categorical(T.Status);  % not numeric codes
```

Use ordinal categorical for ordered data like rankings or severity levels:
```matlab
T.Priority = categorical(T.Priority, ...
    ["Low" "Medium" "High" "Critical"], Ordinal=true);
urgent = T(T.Priority >= "High",:);
```

Extract datetime components for computed variables, filtering, or display:
```matlab
T.Month = month(T.Date);                   % numeric (1-12)
T.Weekday = weekday(T.Date);
T.MonthStart = dateshift(T.Date,"start","month");
```

String arrays support search, edit, and extraction:
```matlab
T.Domain = extractAfter(T.Email,"@");
T.Status = replace(T.Status,"N/A","Unknown");
T.Name = strip(T.Name);
```

Manage categorical levels with `mergecats`, `renamecats`, `removecats`, `reordercats`:
```matlab
T.Region = mergecats(T.Region,["Northeast" "Southeast"],"East");
T.Size = reordercats(T.Size,["Small" "Medium" "Large"]);
```

> Full examples and "avoid" patterns: [data-types.md](references/data-types.md)

## Tables and Timetables

Tables are the primary container for tabular data. Each table variable can be single-column or multi-column (e.g., a matrix); the only requirement is consistent row count. Use "variable" (not "column") to match MathWorks documentation. Prefer dot notation and named access over numeric indexing.

**Table orientation: each variable (column) holds values of the sam