doca-gpunetio-ib-write-bw

# DOCA GPUNetIO ib_write_bw

**Where to start:** This is a tool skill for the GPUNetIO-
flavored `ib_write_bw` benchmark shipped under
`doca/tools/gpunetio_ib_write_bw/` (a client + server pair,
built from source against the installed DOCA via `meson`).
It measures sustained RDMA WRITE bandwidth when the WRs are
posted **from a CUDA kernel through the doca-gpunetio
device-side surface**, with the GPU on the data path. Open
[`TASKS.md`](TASKS.md) and start at
[`## configure`](TASKS.md#configure) for the GPU-NIC
pairing precondition and the build pattern; jump to
[`## run`](TASKS.md#run) for the smoke-before-bulk flow.
Open [`CAPABILITIES.md`](CAPABILITIES.md) when the question
is *what this tool actually measures*, *how the result
decomposes (GPU occupancy vs NIC issue rate vs link
saturation)*, or *how the result reads against the GPI
sister tool and the upstream CPU-initiated `perftest`
`ib_write_bw`*. If DOCA is not installed yet, route to
[`doca-setup`](../../doca-setup/SKILL.md) first; if the
user is still deciding between the GPI and GPUNetIO
programming surfaces, the picture in
[`../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes)
and
[`../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes)
is the first stop.

## Example questions this skill answers well

The CLASSES of `doca-gpunetio-ib-write-bw` questions this
skill is built to answer, each with one worked example. The
class is the load-bearing piece; the worked example is one
instance.

- **"What sustained RDMA-WRITE bandwidth can the GPUNetIO
  path deliver on this GPU-NIC pair?"** — worked example:
  *"measure sustained WRITE BW between two hosts with an
  H100 + ConnectX-7 on each side"*. Answered by the
  GPU-NIC pairing precondition in
  [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)
  + the bring-up flow in
  [`TASKS.md ## configure`](TASKS.md#configure) +
  [`TASKS.md ## run`](TASKS.md#run). The same shape
  answers *"measure GPUNetIO-driven WRITE BW between a
  host GPU and a BlueField DPU"*.
- **"Where is the bottleneck — GPU compute occupancy, NIC
  issue rate, or link saturation?"** — worked example:
  *"I see 120 Gbit/s on a 200 Gbit/s link; is the NIC
  saturated, am I CPU-bound on the client, or is the CUDA
  kernel not driving enough WRs in flight?"*. Answered by
  the throughput-decomposition rules in
  [`CAPABILITIES.md ## Observability`](CAPABILITIES.md#observability)
  + the eval-loop overlay in
  [`TASKS.md ## test`](TASKS.md#test).
- **"How does the result differ from the classic CPU-
  initiated `perftest` `ib_write_bw`?"** — worked example:
  *"my team has a CPU-initiated WRITE BW number on this
  same NIC; should I expect the GPUNetIO number to match
  or be different?"*. Answered by the *"GPU-initiated
  path adds (or removes) overhead vs the CPU-initiated
  path"* rule in
  [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes).
- **"Is the doca-gpunetio path the right surface for my
  sustained-throughput workload class?"** — worked example:
  *"my application streams sensor data from GPU memory at
  line rate to a remote consumer"*. Answered by the
  *"when GPUNetIO is the right surface vs GPI vs CPU-
  initiated"* rule in
  [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)
  + the use-side decision in [`TASKS.md ## use`](TASKS.md#use).
- **"My BW number swings between runs. What do I check
  before quoting it?"** — worked example: *"three runs at
  the same flags gave 145, 187, and 160 Gbit/s; is the
  benchmark noisy or is my platform inconsistent?"*.
  Answered by the measurement-soundness rules in
  [`CAPABILITIES.md ## Error taxonomy`](CAPABILITIES.md#error-taxonomy)
  layer 5 + the steady-state guidance in
  [`TASKS.md ## test`](TASKS.md#test).
- **"What version of DOCA + CUDA Toolkit do I need for this
  binary to build and run?"** — worked example: *"my
  install has DOCA at one semver and CUDA at another; will
  the ToT-shipped `gpunetio_ib_write_bw` even link?"*.
  Answered by the version overlay in
  [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility)
  which cross-links the canonical detection chain in
  [`doca-version`](../../doca-version/SKILL.md).

## Audience

This skill serves **external developers and performance
engineers who need a reproducible measurement of sustained
RDMA WRITE bandwidth when the WRs are posted from a CUDA
kernel through doca-gpunetio**, on the user's actual install
and GPU-NIC pair. Concretely:

- A developer comparing the GPUNetIO path against the GPI
  path or the host-initiated `perftest`-style path before
  committing an application design to one of them.
- A platform operator validating a tuning change (NUMA
  pinning, GPU PCIe placement, IB device choice, GID
  index, NIC firmware burn) by re-running this benchmark
  against the new state.
- An SRE / performance engineer producing a *"this is the
  GPUNetIO-driven WRITE BW on this GPU-NIC pair today"*
  artifact downstream consumers can cite.
- An AI agent answering *"is the doca-gpunetio path a win
  for my sustained-throughput workload class"* honestly —
  with a measured number, the build + invocation that
  produced it, and the GPU + NIC + DOCA version that
  scopes it — rather than guessing from datasheet
  headlines.

It is **not** for users debugging the `doca-gpunetio`
library itself (route to
[`../../libs/doca-gpunetio/SKILL.md`](../../libs/doca-gpunetio/SKILL.md)),
and **not** a substitute for the `perftest` upstream
`ib_write_bw` (which measures CPU-initiated WRITE BW).

## Language scope

The `doca-gpunetio-ib-write-bw` tool is shipped as **C plus
a CUDA `.cu` translation unit** under
`doca/tools/gpunetio_ib_write_bw/`, split into a `client/`
subtree and a `server/` subtree. The verified surface (per
`client/{main.c