project-bourne

# Project Bourne

Project Bourne is open-source execution and provenance infrastructure for
reproducible scientific and engineering workloads.

It plans and executes computational experiments across local machines, GPUs,
Slurm, and PBS while preserving inputs, outputs, execution context, artifact
lineage, telemetry, verification, and the history needed to reproduce a
result.

It is designed for researchers, students from undergraduate through PhD level,
faculty, research engineers, computational scientists, scientific software
users, and scientific-computing teams across academia, public research, and
industry R&D.

## Quick Start

### Human

The human CLI is public today:

~~~bash
python -m pip install bourneprov

bourne run python examples/demo.py
bourne list
bourne show @1

# Or execute an ExecutionRequest v1 document:
bourne execute --request bourne.json
~~~

### Agent / MCP

The v0.6.0 agent and MCP entrypoints are public:

~~~bash
python -m pip install "bourneprov[mcp]==0.6.0"
npx -y @project-bourne/mcp@0.6.0
~~~

For development from a source checkout instead:

~~~bash
python -m pip install -e ".[mcp]"
bourne mcp
~~~

## Why Bourne

Bourne wraps arbitrary executables without requiring changes to the scientific
program. It is local-first and framework-agnostic: Python, compiled solvers,
Julia, MPI programs, and other commands use the same durable experiment model.

~~~bash
bourne run bash -c "echo hello"
bourne run ./solver case.yaml
bourne run julia simulation.jl
bourne run mpirun -np 64 ./solver
~~~

Program stdout and stderr remain visible during execution and are preserved in
the experiment record.

## Architecture

Bourne Core owns deterministic execution, planning, storage, and provenance.
Humans can use it through the CLI or Python services; agents can use the same
services through the optional MCP adapter:

~~~text
             Project Bourne Core
                    │
       ┌────────────┼────────────┐
       │            │            │
      CLI          SDK          MCP
    humans                     agents
~~~

The agent interface is an optional access path, not Bourne's product identity.
MCP works without the portable Skill, and Bourne contains no embedded LLM.

## Agent and MCP Integration

The canonical local stdio server is `bourne mcp`. The stable official MCP
Registry identity is `io.github.KozakHou/project-bourne`, and the portable
Agent Skill is at [`skills/project-bourne`](skills/project-bourne). The v0.6.0
npm package and matching Registry entry are public.

An MCP-compatible agent can translate an explicit request such as “Run this
simulation using four GPUs and preserve provenance” into ExecutionRequest v1,
ask Bourne to plan it, show the deterministic resolution, and execute the
immutable plan after execution intent is established. Bourne itself does not
interpret unconstrained natural language and does not call another model.

The agent path is deliberately two-phase:

~~~text
agent intent → ExecutionRequest v1 → bourne_plan → inspect → bourne_execute_plan
~~~

Planning never runs the workload or silently discovers infrastructure.
Ambiguous targets and unknown facts remain unresolved. MCP annotations are host
UX hints; Bourne Core still enforces immutable plans, exact argv, scheduler job
ownership, artifact semantics, and provenance. See [MCP integration](docs/MCP.md)
and [Agent guidance](docs/AGENTS.md).

## Execution Requests

An execution can now be described once in a bounded, versioned JSON request:

~~~json
{
  "kind": "bourne.execution-request",
  "version": 1,
  "command": ["python", "train.py", "--case", "case1"],
  "artifacts": {
    "inputs": ["config.yaml"],
    "outputs": ["result.h5"]
  },
  "resources": {"cpus": 8, "gpus": 1, "walltime": "2h"},
  "execution": {"backend": "direct"},
  "verification": {
    "checks": [
      {"type": "output_exists", "path": "result.h5"},
      {"type": "output_min_bytes", "path": "result.h5", "min_bytes": 1024}
    ]
  }
}
~~~

Save it as `bourne.json`, then use the same intent for planning or execution:

~~~bash
bourne request validate bourne.json
bourne request show bourne.json

bourne discover
bourne plan --request bourne.json
bourne execute --request bourne.json
~~~

Create a minimal request without executing or discovering anything:

~~~bash
bourne request init --output bourne.json -- python train.py
bourne request schema > execution-request-v1.schema.json
~~~

Existing flag-based commands remain supported. They compile into the same
`ExecutionRequest → WorkloadSpec → ExecutionPlan` pipeline rather than a
parallel implementation:

~~~bash
bourne execute --backend direct --cpus 2 --output result.txt -- python script.py
~~~

For a request file, a relative `working_directory` is resolved from the
request file's directory. Declared artifacts are then resolved from that
scientific working directory. Bourne preserves both the lexical and resolved
working-directory values and does not expand `$HOME`, evaluate shell syntax,
import project code, or execute anything while parsing or planning.

Parent references follow the same intent-preserving rule. A request may use
`latest`, `@N`, a unique prefix, or a full ULID. Bourne retains that requested
value while separately recording the canonical parent ULID used by the compiled
workload.

Summary telemetry is enabled by default and uses already captured facts: wall
time, UTF-8 stdout/stderr byte counts, known artifact byte totals, requested
resources, observed allocation, and scheduler queue timing when timestamps
establish it. `"telemetry": {"mode": "off"}` disables the summary. Missing
metrics remain unavailable, never zero.

The initial deterministic verification checks are `output_exists`,
`output_min_bytes`, and `output_sha256`. They evaluate only captured declared
output `Artifact` records. Verification is persisted separately from process
status: an experiment may be `completed` while verification is `failed` or
`unknown`. These checks establish artifact facts, not general scientific
validity. See [Execution requests, telemetry, and verification](docs/EXECUTION_REQUESTS.md)
for the exact contract and safety limits.

## Planning and Execution

Project Bourne v0.4.0 adds a durable planning layer over v0.3 inventories:

~~~bash
bourne discover

bourne plan --backend direct -- python examples/demo.py
bourne execute --backend direct -- python examples/demo.py

bourne execution list
bourne execution show @1
~~~

`bourne plan` never runs the scientific command and never performs discovery.
It creates a framework-independent `WorkloadSpec`, compares its explicit and
inferred requirements with an existing inventory, explains every candidate,
and persists an immutable `ExecutionPlan` only when selection is unambiguous.
Use explicit resource and placement constraints when needed:

~~~bash
bourne plan \
  --backend slurm \
  --target gpu \
  --cpus 16 \
  --gpus 4 \
  --nodes 1 \
  --memory 64G \
  --walltime 2h \
  -- ./solver case.yaml
~~~

Execute a selected Slurm plan and then inspect or wait for the resulting
execution attempt:

~~~bash
bourne execute --plan @1
bourne execution show @1
bourne execution wait @1
~~~

While a recorded job is still active, `bourne execution cancel @1` requests
cancellation of that Bourne-managed job. The same planning and lifecycle model
supports `--backend pbs`.

Direct execution reuses Bourne's existing live-output, process-group, artifact,
lineage, and experiment-provenance machinery. Slurm and PBS plans use a
self-contained Bourne worker staged with the plan. The worker performs
preflight and records the actual allocated host and scientific experiment;
the access-side controller imports its bounded JSON result transactionally.
No compute-node SSH or preinstalled `bourneprov` package is required, although
the compute allocation must provide Python 3 and visibility of the staging and
working directories.

Submission is not an experiment, scheduler completion is not scientific
success, and requested resources are not allocated resources. Bourne records
these as separate durable facts. Cancellation accepts a Bourne execution
reference—not an arbitrary scheduler job ID—and checks the submitting identity.
See [Workload planning and scheduler execution](docs/WORKLOAD_EXECUTION.md) for
the exact model, safety boundary, and current limitations.

## Compute-site discovery (v0.3.0)

Bourne can take an immutable, local snapshot of the execution surface visible
to your current identity:

~~~bash
bourne discover
bourne inventory
bourne inventory --find python
bourne inventory --json
~~~

Discovery covers the current identity and access target, allow-listed
user-relevant storage paths, direct execution contexts, generic PATH
executables, optional Conda/virtualenv/container/module contexts, safe system
capabilities, Bourne history, and read-only Slurm/PBS target-class summaries
when available. An unknown executable is recorded generically without being
run. Laptops, desktop and GPU workstations, DGX-class personal machines, shared
laboratory systems, and scheduler-backed HPC sites are all valid compute
sites. A scheduler-free machine is complete in its own right.

Discovery is observational: an executable is not verified workload
compatibility, a visible scheduler partition is not proof of submission
authorization, and a storage role hint is not a retention or backup policy.
Inventories remain local. Providers do not traverse other users' homes, crawl
shared storage, inspect SSH credentials or container secrets, dump arbitrary
environment variables, SSH into compute nodes, submit or cancel scheduler
jobs, or modify environments. See [Compute-site discovery](docs/COMPUTE_SITE_DISCOVERY.md)
for the exact topology, evidence, limits, and security semantics.

## Provenance, Artifacts and Lineage

Project Bourne v0.2 adds explicit input/output fingerprints, a minimal
derived_from relationship, safe execution-context observations, and artifact
tracing. Run the deterministic exam