matlab-evaluate-tutor-quality
Use when reviewing, auditing, scoring, or improving a real or synthetic MATLAB AI tutor transcript, tutoring prompt, generated lesson, exercise, feedback sequence, or skill behavior for MATLAB accuracy, active learning, assignment guardrails, feedback quality, debugging support, transfer prompts, and instructor-facing quality recommendations.
git clone --depth 1 https://github.com/matlab/agent-skills-playground /tmp/matlab-evaluate-tutor-quality && cp -r /tmp/matlab-evaluate-tutor-quality/demos/ai-tutoring/skills/matlab-evaluate-tutor-quality ~/.claude/skills/matlab-evaluate-tutor-qualitySKILL.md
# MATLAB AI Tutor Evaluation ## Purpose Evaluate whether a MATLAB tutor interaction helps a learner think, practice, and transfer understanding. Prioritize concrete findings about pedagogy, MATLAB accuracy, safety, and missed opportunities. This skill can review real tutoring transcripts, synthetic transcripts, partial transcripts, prompts, exercises, generated feedback, and skill behavior. For instructors, this skill is a quality-control tool. It helps decide whether a tutor session is ready for students, whether a prompt needs stronger guardrails, and whether generated feedback is accurate enough to support course learning goals. Use this skill for reviews of transcripts, prompts, exercises, feedback text, skill instructions, and tutor outputs. Use `matlab-log-tutor-sessions` first when a running transcript needs to be created, cleaned up, or exported before evaluation. ## Repeatable Review Workflow 1. Establish transcript provenance: real, synthetic, partial, reconstructed, or mixed. State any limits this creates for the review. 2. Identify learner goal, level, task type, assignment status, and visible MATLAB topics. 3. Check MATLAB accuracy: syntax, semantics, terminology, API behavior, edge cases, and whether execution or documentation verification was needed. 4. Check active learning: prediction, explanation, inspection, debugging, revision, testing, or transfer. 5. Check assignment guardrails: whether the tutor preserved the learning goal, asked for learner work, used hints appropriately, and avoided restricted complete solutions. 6. Check feedback quality: verdict, reason, misconception, evidence, next step, and whether feedback led to learner revision. 7. Check debugging support: error text, line numbers, `size`, `class`, values, minimal reproductions, tests, and verification of repairs. 8. Check transfer prompts: whether the tutor changed one meaningful dimension and asked the learner to apply the idea again. 9. Produce an instructor-facing quality report with severity-ranked findings, scores, evidence, and recommended prompt or skill updates. ## Evaluation Dimensions - **MATLAB correctness**: Syntax, semantics, terminology, and idiomatic usage. - **Learning design**: Learner must predict, inspect, explain, revise, or test. - **Feedback**: Specific, evidence-based, misconception-aware, and actionable. - **Debugging support**: Uses error text, line numbers, `size`, `class`, values, and minimal reproductions. - **Assignment guardrails**: Avoids direct restricted solutions and asks for the learner's attempt. - **Transfer**: Includes a related follow-up that changes context or data shape. - **Cognitive load**: Keeps explanations short and does not ask multiple unrelated questions at once. - **Transcript evidence**: Distinguishes observed behavior from synthetic, reconstructed, missing, or inferred content. ## Output Format For quick reviews, lead with findings. Use this shape: ```text Findings - [Severity] [Dimension]: [Issue and why it matters]. Evidence: [quote or reference]. Strengths - [What the tutor did well, if useful.] Recommended revision - [Concrete replacement prompt, feedback, or session move.] Score - Active learning: [1-4] - MATLAB accuracy: [1-4] - Feedback quality: [1-4] - Guardrails: [1-4 or N/A] - Transfer: [1-4] ``` For instructor-facing quality reports, use this shape: ```text Instructor-Facing Quality Report Review scope - Transcript status: [Real | Synthetic | Partial | Reconstructed | Mixed] - Learner goal: - Assignment status: - MATLAB topics: - Evidence limits: Findings - [Severity] [Dimension]: [Issue and instructional impact]. Evidence: [quote, turn, or line]. Scores - MATLAB accuracy: [1-4] - Active learning: [1-4] - Assignment guardrails: [1-4 or N/A] - Feedback quality: [1-4] - Debugging support: [1-4 or N/A] - Transfer prompts: [1-4] Recommended prompt or skill updates - [Specific update to tutor prompt, guardrail policy, debugging workflow, feedback pattern, or transfer requirement.] Keep, revise, or investigate - Keep: - Revise: - Investigate: ``` Read [references/evaluation-rubric.md](references/evaluation-rubric.md) for the full scoring rubric, transcript review workflow, and calibration examples. Read [references/transcript-review-examples.md](references/transcript-review-examples.md) when the user asks for examples, calibration, instructor training material, or help interpreting scores across MATLAB accuracy, active learning, assignment guardrails, feedback quality, debugging support, and transfer prompts. This demo includes example calibration artifacts, at paths relative to the demo folder that contains `skills/` (not this skill folder): - `assets/examples/transcript-review-calibration.md` - `assets/examples/quality-report-calibration.md` ## Instructor Adoption Notes - Review a small sample of sessions before using the tutor broadly. - Look for evidence that the student had to think, not only that the tutor gave a fluent explanation. - Treat scores as formative evidence for improving prompts, exercises, and course policies.
>
Use this skill when the user wants to author, design, scope, or refine an Agent Skill (a SKILL.md file). Trigger phrases include "build a new skill", "design an agent skill", "scope a SKILL.md", "how should I structure this skill", "write a skill for X", "my skill isn't working well", or any request to improve an existing SKILL.md. Walks the user through an empirical, test-first process — probe the agent for real failures, design only for genuine knowledge gaps, iterate against runnable examples, and verify across models.
Use this skill for any work involving a MATLAB Project (.prj file) — creating a new project, tracking files, managing the project path, configuring Simulink cache and code-generation folders, running project health checks, or writing build scripts that keep the project in sync with the file system. Trigger phrases include "set up a MATLAB project", "create a .prj", "track this file in the project", "project health check", "build script conventions". This skill is the generic foundation; domain-specific skills (e.g. `mbse-workflow`) build on it.
Use this skill for the architecture phases of an MBSE workflow in MATLAB, when writing idempotent buildXxx.m scripts that produce a three-layer RFLPV architecture (Functional, Logical, Physical) with interface dictionaries, stereotype profiles, allocation sets, and requirements Implement links. Trigger for defining stereotype properties, functional-to-logical / logical-to-physical allocation, mapping requirements to components via slreq Implement links, or running quantitative roll-up analysis on the architecture. Do NOT trigger for ad-hoc structural edits to an already-built System Composer model (adding one component, rewiring a port) — use `building-simulink-models` with `model_edit` for that. Works alongside the `system-composer` skill for detailed SC API patterns.
Use this skill for guided MBSE work in MATLAB — starting a new project, resuming work mid-workflow on an existing project, or answering orientation questions about how the MBSE skills fit together. Trigger when the user says they want to create, start, or set up a new MBSE project; work on a model-based systems engineering / RFLPV project; or asks which skill covers which phase. Walks through phases one at a time — propose → approve → generate → run → confirm. Use proactively whenever someone mentions starting or continuing an MBSE project.
Use this skill for all requirements-related work in a MATLAB MBSE project using the Requirements Toolbox (slreq). Covers creating and populating requirement sets, derivation links, test case requirements, verification coverage, reading and tracing links across requirement sets and models, checking link health, allocating requirements to components (Implement links), and building traceability reports. Trigger when the user asks about slreq API, slreqx files, slmx link files, outLinks/inLinks, traceability matrices, coverage analysis, broken links, or mapping requirements to architecture components. Use proactively for any requirements or traceability task.
Use this skill when authoring reusable, idempotent MATLAB scripts that build System Composer architecture models via the architecture-modeling API — `systemcomposer.createModel`, `addComponent`, `addPort`, `setInterface`, `connect(srcPort, dstPort)`, interface dictionaries (.sldd) with `addInterface`/`addElement`, profiles/stereotypes with `Profile.createProfile` and `addStereotype`, or `systemcomposer.allocation.createAllocationSet`. Also trigger when debugging these APIs (connections that don't appear, interfaces that don't resolve, profile save errors, `createAllocationSet` signature-mismatch errors). Do NOT trigger for ad-hoc structural edits to an already-built model (adding one SubSystem, rewiring a port) — use `building-simulink-models` with `model_edit` for that.
Optimize MATLAB code for better performance through vectorization, memory management, and profiling. Use when user requests optimization, mentions slow code, performance issues, speed improvements, or asks to make code faster or more efficient.