debug

# NanoClaw Container Debugging

This guide covers debugging the containerized agent execution system.

## Architecture Overview

The host is a single Node process that orchestrates per-session agent containers. The two session DBs are the **sole** IO surface between host and container — there is no IPC, no file watcher, and no stdin piping.

```
Host (Node)                                Container (Bun, Linux VM)
──────────────────────────────────────────────────────────────────────
src/container-runner.ts                    container/agent-runner/src/
    │                                          │
    │ spawns one container per session          │ polls inbound.db for work,
    │ with the session folder mounted          │ calls the agent provider,
    │ at /workspace                            │ writes replies to outbound.db
    │                                          │
    ├── data/v2-sessions/<group>/<session>/ ──> /workspace
    │     ├── inbound.db   (host writes, container reads RO)
    │     ├── outbound.db  (container writes, host reads)
    │     └── .heartbeat   (container touches → /workspace/.heartbeat)
    ├── groups/<folder> ─────────────────────> /workspace/agent  (cwd)
    ├── <group>/.claude-shared ──────────────> /home/node/.claude
    └── agent-runner src + skills ───────────> /app/src, /app/skills
```

**Message flow:** host writes a row to `inbound.db` (`messages_in`) and wakes the container; the container's poll loop picks it up, runs the agent, and writes the reply to `outbound.db` (`messages_out`); the host's delivery poll reads `messages_out` and sends it through the channel adapter. See [docs/db.md](../../../docs/db.md) and [docs/db-session.md](../../../docs/db-session.md) for the full two-DB model.

**Container identity:** the container runs as user `node` with `HOME=/home/node`. Per-group Claude state (settings, session history) lives in `<group>/.claude-shared` on the host, mounted to `/home/node/.claude`.

## Log Locations

| Log | Location | Content |
|-----|----------|---------|
| **Host errors** | `logs/nanoclaw.error.log` | Delivery failures, crash-loop backoff, warnings — check this first |
| **Host app log** | `logs/nanoclaw.log` | Full routing chain: inbound routing, container spawn/exit, delivery |
| **Setup logs** | `logs/setup.log`, `logs/setup-steps/*.log` | Per-step install output (bootstrap, container, onecli, mounts, service) |
| **Session inbound** | `data/v2-sessions/<group>/<session>/inbound.db` (`messages_in`) | Did the message reach the container? |
| **Session outbound** | `data/v2-sessions/<group>/<session>/outbound.db` (`messages_out`) | Did the agent produce a reply? |

Containers run with `--rm`, so the container's own filesystem is gone after it exits. The host streams container **stderr** into `logs/nanoclaw.log` at debug level, tagged with `container=<group folder>`; raise the log level (below) to see it. If the agent silently failed inside an exited container, there is no persistent in-container log — reconstruct from the session DBs and the host log.

## Enabling Debug Logging

Set `LOG_LEVEL=debug` for verbose output, including streamed container stderr:

```bash
# For development
LOG_LEVEL=debug pnpm run dev

# For launchd service (macOS), add to plist EnvironmentVariables:
<key>LOG_LEVEL</key>
<string>debug</string>
# For systemd service (Linux), add to unit [Service] section:
# Environment=LOG_LEVEL=debug
```

Debug level shows full mount configurations, the container spawn command, and streamed container stderr lines.

## Inspecting Session DBs

The two session DBs are where the message flow lives. Use the in-tree query wrapper (it goes through the `better-sqlite3` dep that setup already installs, avoiding a dependency on the `sqlite3` CLI):

```bash
# List sessions and their agent group / messaging group from the central DB
pnpm exec tsx scripts/q.ts data/v2.db "SELECT id, agent_group_id, messaging_group_id, status, container_status, last_active FROM sessions"

# Or via the admin CLI
ncl sessions list

# Did the message reach the container? (inbound.db, host writes / container reads)
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/inbound.db \
  "SELECT seq, kind, status, timestamp FROM messages_in ORDER BY seq DESC LIMIT 10"

# Did the agent produce a reply? (outbound.db, container writes / host reads)
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/outbound.db \
  "SELECT seq, kind, timestamp FROM messages_out ORDER BY seq DESC LIMIT 10"

# Container-side processing status for each inbound message
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/outbound.db \
  "SELECT message_id, status, status_changed FROM processing_ack ORDER BY status_changed DESC LIMIT 10"
```

Reading the flow:
- `messages_in` has the message but no matching `messages_out` → the container never produced a reply (check `processing_ack`, then `logs/nanoclaw.log` for spawn/exit and container stderr).
- `messages_out` has a reply but the user never received it → a delivery problem (see issue 1 below).
- `messages_in` is empty → routing never reached this session (check the router log lines and the central wiring with `ncl wirings list`).

## Common Issues

### 1. "No adapter for channel type" / Messages silently lost (null platform_message_id)

**Symptom:** The bot stops replying. `logs/nanoclaw.error.log` shows repeated:
```
WARN No adapter for channel type channelType="telegram"
WARN No adapter for channel type channelType="signal"
```
The main log shows "Message delivered" entries with `platformMsgId=undefined` — meaning the delivery poll ran, found no adapter, and marked the message delivered without sending it.

**Root cause: two NanoClaw service instances running simultaneously.**

When a second service instance is active with a stale binary, it has no channel adapters registered. Its delivery poll races the working instance and wins — marking outbound messages delivered without ever sending them.

**D