matlab-engineer-tabular-features

# Engineer Tabular Features

A lean, functional pipeline: **intake → feature pool → select → assess →
deliver → report**. There is no shared context object — each phase is a direct
call to leaf utilities in `scripts/`, and the reference file for each phase
carries the detail. Your value is the **structured, data-driven process** and,
above all, the **consensus selection** at its center — not an ad-hoc answer.

This skill bundles the workflow in `references/` (per-phase detail read on demand)
and `scripts/` (the computation and plotting utilities). Do not invoke files in
`references/` as separate skills — they are loaded only via the `Read` tool when
the phase that needs them runs. The per-phase files call the pipeline's leaf helpers
for you; if you ever need a helper's signature,
[references/internal-helpers.md](references/internal-helpers.md) documents each one's
inputs, outputs, and an example call — so you never open a helper's source.

## When to Use

- Engineering or selecting the best predictors for **single-response supervised
  classification or regression** on a plain in-memory `table`.
- You want a **structured, data-driven selection** — a ranker panel, a consensus
  vote, and an elbow cut — rather than an ad-hoc hand-picked feature set.
- The data is **non-tabular** (signals, images, battery/machinery telemetry): this
  skill routes extraction to the matching domain skill, then engineers, selects,
  assesses, and delivers on the resulting table (see
  [references/domain-routing.md](references/domain-routing.md)).

## When NOT to Use

- **Multi-response problems** — this skill is single-response only.
- **Model training, tuning, or deployment** — it prepares features and stops.
  Hand the delivered table to a model-training/classification workflow to fit and
  compare models.
- **Raw data acquisition.** And for the *extraction* step on non-tabular data, the
  actual feature computation belongs to the matching **domain extraction skill** —
  this skill orchestrates that handoff (see
  [references/domain-routing.md](references/domain-routing.md)), it does not
  re-implement it.

**Requires** the Statistics and Machine Learning Toolbox (SMLT) —
`gencfeatures`/`genrfeatures` build the pool and the ranker/assessment utilities
are SMLT-based. MATLAB Report Generator is optional (enables the PDF report;
markdown is always produced).

## Running MATLAB

Run all MATLAB through the MATLAB MCP server (`mcp__matlab__evaluate_matlab_code`,
or `mcp__matlab__run_matlab_file` for scripts). Set `project_path` to this skill's
`scripts/` directory so the utilities resolve on the current working folder
**without any `addpath` calls**. Every utility is a leaf function called directly —
there is no initialization step and no context object to construct. Validate any
code you author with `mcp__matlab__check_matlab_code` before running it.

**Start each dataset from scratch — but use the live workspace within a run.** The
MCP session is stateful, so a run's intermediates should live in the workspace: set
`RawTbl`, `Splits`, `FullEng`, `SelectedNames`, `Baseline`, etc. once and pass them
phase-to-phase. Do **not** round-trip them through `save`/`load` `.mat` files (noise,
risks stale reads) and do **not** `addpath`. Across *different* datasets/runs, carry
nothing — begin each analysis by setting every variable afresh.

## Communication style while running this skill

Talk to the user about their **data and results**, not the skill's plumbing.
Everything under `references/` and `scripts/` is internal. Rule of thumb: if a
sentence would only make sense to someone who has read this skill's source files,
don't say it.

- **Never name internal files, helpers, or phase/gating mechanics.**
  `runConsensusSelection`, `GenInfo.BinaryReliant`, "the redundancy dimension", etc.
  are internal — give the *outcome* ("these features duplicate each other, so I'm
  keeping the strongest"), not the mechanism. Name an internal only when it *is* a
  problem the user can act on. Read reference files silently.
- **Use plain words for each check.** The three assessment reads: performance →
  whether the new features improve predictions (a held-out estimate, or a
  cross-validated mean ± std); fixed-pool stability → whether the same features get
  picked when rows are resampled; generation stability → whether the same features
  get *built and* picked when the whole pipeline re-runs on resampled rows. Say "the
  ranking step" not "the borda voter"; "reliably re-selected" not "consensus core".
- **Don't narrate uncertainty or mid-flight course-corrections** — settle how a
  function is called silently, then report only the outcome. Surface a difficulty
  only when the user must decide on it.
- **Announce cost before long work**, one sentence — pool size before selection,
  expected time before a K-fold, and **before either stability gate** (both re-run
  selection many times; the generation gate also re-builds the pool each time). And
  **surface user-facing questions verbatim** where a phase specifies one (output
  directory, wide-input, domain routing).
- **Report what was dropped** at every phase (screened predictors, excluded WoE
  columns, skipped rankers) — a silent shrink reads as data loss. But **selection
  evaluates the pool, it doesn't necessarily shrink it** — never call it a reduction.

## Output directory — REQUIRED, HARD HALT

Deliverables are written to disk. **Always confirm the output directory with the
user before writing anything.** Do not assume the working directory, do not
create one silently.

---

## The pipeline

Follow the phases in order. Each links to its reference; read the reference
before executing the phase.

### 1. Intake — [references/intake.md](references/intake.md)

**Ask before running any code.** Intake is a required conversation, not a
default-fill. Confirm every run parameter with the user *before* proceeding past
the screen — data source, response, dataset name, output directory (hard