response-compliance
The response-compliance skill runs the official OpenResponses compliance test suite against a Response API endpoint to validate endpoint behavior, schema compliance, and feature support. Use this when testing Response API implementations, debugging schema issues, validating authentication modes, or running compliance checks in CI/CD pipelines against local or remote endpoints.
git clone --depth 1 https://github.com/lobehub/lobehub /tmp/response-compliance && cp -r /tmp/response-compliance/.agents/skills/response-compliance ~/.claude/skills/response-complianceSKILL.md
# OpenResponses Compliance Test Run the official OpenResponses compliance test suite against the local (or remote) Response API endpoint. ## Quick Start ```bash # From the openapi package directory cd lobehub/packages/openapi # Run all tests (dev mode, localhost:3010) APP_URL=http://localhost:3010 bun run test:response-compliance -- \ --auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 # Run specific tests only APP_URL=http://localhost:3010 bun run test:response-compliance -- \ --auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 \ --filter basic-response,streaming-response # Verbose mode (shows request/response details) APP_URL=http://localhost:3010 bun run test:response-compliance -- \ --auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 -v # JSON output (for CI) APP_URL=http://localhost:3010 bun run test:response-compliance -- \ --auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 --json ``` ## Prerequisites - Dev server running with `ENABLE_MOCK_DEV_USER=true` in `.env` - The `api/v1/responses` route registered (via `src/app/(backend)/api/v1/[[...route]]/route.ts`) ## Auth Modes | Mode | Flags | | --------------- | ------------------------------------------------------------------- | | Dev (mock user) | `--auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1` | | API Key | `--api-key lb-xxxxxxxxxxxxxxxx` | | Custom | `--auth-header <name> --api-key <value>` | ## Test IDs Available `--filter` values: | ID | Description | Related Issue | | -------------------- | -------------------------------------- | ------------- | | `basic-response` | Simple text generation (non-streaming) | LOBE-5858 | | `streaming-response` | SSE streaming lifecycle + events | LOBE-5859 | | `system-prompt` | System role message handling | LOBE-5858 | | `tool-calling` | Function tool definition + call output | LOBE-5860 | | `image-input` | Multimodal image URL content | — | | `multi-turn` | Conversation history via input items | LOBE-5861 | ## Environment Variables | Variable | Default | Description | | --------- | ----------------------- | ----------------------------------------- | | `APP_URL` | `http://localhost:3010` | Server base URL (auto-appends `/api/v1`) | | `API_KEY` | — | API key (alternative to `--api-key` flag) | ## How It Works The script (`lobehub/packages/openapi/scripts/compliance-test.sh`) clones the official [openresponses/openresponses](https://github.com/openresponses/openresponses) repo into `scripts/openresponses-compliance/` (gitignored) and runs its CLI test runner. First run clones; subsequent runs update from upstream. ## Debugging Failures 1. Run with `-v` to see full request/response payloads 2. Common failure patterns: - **"Failed to parse JSON"**: Auth failed, server returned HTML redirect - **"Response has no output items"**: LLM execution not yet implemented - **"Expected number, received null"**: Missing required field in response schema - **"Invalid input"**: Zod validation on response schema — check field format ## Key Files - **Types**: `lobehub/packages/openapi/src/types/responses.type.ts` - **Service**: `lobehub/packages/openapi/src/services/responses.service.ts` - **Controller**: `lobehub/packages/openapi/src/controllers/responses.controller.ts` - **Route**: `lobehub/packages/openapi/src/routes/responses.route.ts` - **Test script**: `lobehub/packages/openapi/scripts/compliance-test.sh` - **Cloud route**: `src/app/(backend)/api/v1/[[...route]]/route.ts`
Add documentation for a new AI provider — usage docs, env vars, Docker config, image resources.
Add server-side environment variables that control default values for user settings.
Agent runtime lifecycle hooks. Use for before/after tool or step hooks, tool mocks, human intervention, sub-agent calls, context compression, evals, callAgent, or lifecycle events.
Build or extend LobeHub Agent Signal pipelines. Use for signal sources, signal/action types, policies, middleware, workflow handoff, dedupe, scope behavior, or observability.
Agent tracing CLI for execution snapshots. Use for agent-tracing, traces, snapshots, LLM call inspection, context engine data, agent step analysis, execution debugging, or pulling remote/production traces ("拉线上 tracing") by operation id. Also the first stop for debugging agent tool calls — wrong or missing tool_calls, unexpected tool arguments or results, which tools were available at a step, or why a tool ran where it did.
Build LobeHub builtin tool packages. Use when adding agent-callable tools, manifests, executors, runtimes, inspectors, renders, placeholders, streaming, interventions, portals, or tool registries.
Build multi-platform chat bots with the chat SDK. Use for Slack, Teams, Google Chat, Discord, GitHub, Linear bots, webhooks, mentions, slash commands, cards, modals, or streaming responses.
>