A process DAG takes LLM agents from 0% to 100% on CDISC

Eleven attempts, five frontier models, zero valid datasets. That is the starting point of GxP-Agent, a paper published on arXiv on 19 August that measures something very specific: what happens when you ask an LLM to turn a clinical trial protocol into an analysis ready dataset under CDISC standards. In single shot mode, the answer is that none of the five models produced a valid ADSL dataset. Not one.

It is worth understanding why the task is so hostile. Clinical trial programming is the bottleneck of any regulatory submission: raw data has to become normalised tables following a closed standard, with derivation rules that depend on each other and with auditable traceability. The goal is not pretty code, it is code an FDA reviewer can reproduce. A failure in an early derived variable contaminates everything downstream, and the model has no way of knowing that if it emits the whole file in one go.

What the topology changes

The authors do not propose a better model or a longer prompt. They propose a topology. GxP-Agent encodes regulatory process ordering as a directed acyclic graph (DAG) and splits monolithic dataset generation into 15 domain specific nodes. Each node is executed by a worker agent with pharmaverse skill context, passes through a validation gate and, if it fails, enters conditional retry.

Put another way: the system never asks the model to solve the whole problem. It asks it to solve a chunk whose dependencies are already satisfied, and it checks the result before letting it move on. The ordering is not improvised by the LLM, it is imposed by the graph.

The numbers

The authors build CDISC-Bench, an execution based benchmark built from the FDA pilot submission CDISCPilot01: 254 subjects and 49 ground truth ADSL variables. On that testbed, the published results are these:

1. GxP-Agent with Claude Sonnet 4.6: 100% structural match (49/49 variables, 254 correct records) across three independent runs.
2. Best retrieval augmented baseline: 59.2%.
3. All single agent and flat multi agent approaches: 0%.

The figure that interests us most is a different one. Under the same DAG, GPT-4.1 reaches a 59.2% mean structural match, far above what that model scores on its own on the same task. The topology does not just lift the strong model, it rescues the weak one. That suggests much of the failure was not reasoning capacity but the absence of execution structure.

The result should be read carefully. Structural match is not the same as signed off clinical correctness, three runs are three runs, and a single pilot submission does not cover the real variety of protocols. CDISC-Bench is introduced by the authors themselves in the same paper, so there are no third parties replicating the numbers yet.

Who this is useful for

If you work in pharma or at a CRO, the practical reading is direct: the problem is not fixed by waiting for the next model. It is fixed by modelling the process you already have documented, because in a GxP environment that process is documented by obligation, and turning it into an execution graph with validation gates.

And if you do not work in pharma, the pattern transfers anyway. Any flow with hard dependencies and objective validation fits: financial closes, data migrations, regulated reporting, ETL pipelines with business rules. The ingredients are already in Claude Code: subagents for the nodes, skills for each node's domain context, hooks for the validation gates. What the paper adds is the discipline of ordering all of it with an explicit DAG instead of letting an orchestrator decide on the fly.

At ElephantPink we keep seeing the same thing in far less regulated integrations: the multi agent systems that survive in production are the ones whose ordering lives outside the model. This work does not discover that idea, but it measures it with a number that is hard to ignore, from 0% to 100% without changing the model.