{"id":"debug","name":"debug","summary":"コンテナエージェントの問題をデバッグしてください。動作が悪い時やコンテナの障害、認証の問題、コンテナシステムの仕組みを理解するために使います。","body":"# NanoClaw Container Debugging\n\nThis guide covers debugging the containerized agent execution system.\n\n## Architecture Overview\n\nThe host is a single Node process that orchestrates per-session agent containers. The two session DBs are the **sole** IO surface between host and container — there is no IPC, no file watcher, and no stdin piping.\n\n```\nHost (Node) Container (Bun, Linux VM)\n──────────────────────────────────────────────────────────────────────\nsrc/container-runner.ts container/agent-runner/src/\n │ │\n │ spawns one container per session │ polls inbound.db for work,\n │ with the session folder mounted │ calls the agent provider,\n │ at /workspace │ writes replies to outbound.db\n │ │\n ├── data/v2-sessions/// ──> /workspace\n │ ├── inbound.db (host writes, container reads RO)\n │ ├── outbound.db (container writes, host reads)\n │ └── .heartbeat (container touches → /workspace/.heartbeat)\n ├── groups/ ─────────────────────> /workspace/agent (cwd)\n ├── /.claude-shared ──────────────> /home/node/.claude\n └── agent-runner src + skills ───────────> /app/src, /app/skills\n```\n\n**Message flow:** host writes a row to `inbound.db` (`messages_in`) and wakes the container; the container's poll loop picks it up, runs the agent, and writes the reply to `outbound.db` (`messages_out`); the host's delivery poll reads `messages_out` and sends it through the channel adapter. See [docs/db.md](../../../docs/db.md) and [docs/db-session.md](../../../docs/db-session.md) for the full two-DB model.\n\n**Container identity:** the container runs as user `node` with `HOME=/home/node`. Per-group Claude state (settings, session history) lives in `/.claude-shared` on the host, mounted to `/home/node/.claude`.\n\n## Log Locations\n\n| Log | Location | Content |\n|-----|----------|---------|\n| **Host errors** | `logs/nanoclaw.error.log` | Delivery failures, crash-loop backoff, warnings — check this first |\n| **Host app log** | `logs/nanoclaw.log` | Full routing chain: inbound routing, container spawn/exit, delivery |\n| **Setup logs** | `logs/setup.log`, `logs/setup-steps/*.log` | Per-step install output (bootstrap, container, onecli, mounts, service) |\n| **Session inbound** | `data/v2-sessions///inbound.db` (`messages_in`) | Did the message reach the container? |\n| **Session outbound** | `data/v2-sessions///outbound.db` (`messages_out`) | Did the agent produce a reply? |\n\nContainers run with `--rm`, so the container's own filesystem is gone after it exits. The host streams container **stderr** into `logs/nanoclaw.log` at debug level, tagged with `container=`; raise the log level (below) to see it. If the agent silently failed inside an exited container, there is no persistent in-container log — reconstruct from the session DBs and the host log.\n\n## Enabling Debug Logging\n\nSet `LOG_LEVEL=debug` for verbose output, including streamed container stderr:\n\n```bash\n# For development\nLOG_LEVEL=debug pnpm run dev\n\n# For launchd service (macOS), add to plist EnvironmentVariables:\nLOG_LEVEL\ndebug\n# For systemd service (Linux), add to unit [Service] section:\n# Environment=LOG_LEVEL=debug\n```\n\nDebug level shows full mount configurations, the container spawn command, and streamed container stderr lines.\n\n## Inspecting Session DBs\n\nThe two session DBs are where the message flow lives. Use the in-tree query wrapper (it goes through the `better-sqlite3` dep that setup already installs, avoiding a dependency on the `sqlite3` CLI):\n\n```bash\n# List sessions and their agent group / messaging group from the central DB\npnpm exec tsx scripts/q.ts data/v2.db \"SELECT id, agent_group_id, messaging_group_id, status, container_status, last_active FROM sessions\"\n\n# Or via the admin CLI\nncl sessions list\n\n# Did the message reach the container? (inbound.db, host writes / container reads)\npnpm exec tsx scripts/q.ts data/v2-sessions///inbound.db \\\n \"SELECT seq, kind, status, timestamp FROM messages_in ORDER BY seq DESC LIMIT 10\"\n\n# Did the agent produce a reply? (outbound.db, container writes / host reads)\npnpm exec tsx scripts/q.ts data/v2-sessions///outbound.db \\\n \"SELECT seq, kind, timestamp FROM messages_out ORDER BY seq DESC LIMIT 10\"\n\n# Container-side processing status for each inbound message\npnpm exec tsx scripts/q.ts data/v2-sessions///outbound.db \\\n \"SELECT message_id, status, status_changed FROM processing_ack ORDER BY status_changed DESC LIMIT 10\"\n```\n\nReading the flow:\n- `messages_in` has the message but no matching `messages_out` → the container never produced a reply (check `processing_ack`, then `logs/nanoclaw.log` for spawn/exit and container stderr).\n- `messages_out` has a reply but the user never received it → a delivery problem (see issue 1 below).\n- `messages_in` is empty → routing never reached this session (check the router log lines and the central wiring with `ncl wirings list`).\n\n## Common Issues\n\n### 1. \"No adapter for channel type\" / Messages silently lost (null platform_message_id)\n\n**Symptom:** The bot stops replying. `logs/nanoclaw.error.log` shows repeated:\n```\nWARN No adapter for channel type channelType=\"telegram\"\nWARN No adapter for channel type channelType=\"signal\"\n```\nThe main log shows \"Message delivered\" entries with `platformMsgId=undefined` — meaning the delivery poll ran, found no adapter, and marked the message delivered without sending it.\n\n**Root cause: two NanoClaw service instances running simultaneously.**\n\nWhen a second service instance is active with a stale binary, it has no channel adapters registered. Its delivery poll races the working instance and wins — marking outbound messages delivered without ever sending them.\n\n**Diagnosis:**\n```bash\n# Check for duplicate running instances\nps aux | grep 'nanoclaw/dist/index.js' | grep -v grep\n\n# Check which services are active (Linux)\nsystemctl --user list-units 'nanoclaw*' --all\n\n# Confirm channel adapters registered by the current process\ngrep \"Channel adapter started\" logs/nanoclaw.log | tail -10\n```\n\n**Fix:**\n1. Identify which service has the correct binary and EnvironmentFile (the one whose log shows the expected channels — e.g. `signal`, `telegram`, `cli` — all started).\n2. Stop and disable the stale duplicate service:\n ```bash\n systemctl --user stop nanoclaw.service # or whichever is the old one\n systemctl --user disable nanoclaw.service\n ```\n3. If the remaining service unit is missing `EnvironmentFile`, add it:\n ```bash\n # Edit the service unit — add this line under [Service]:\n # EnvironmentFile=/home/[user]/nanoclaw/.env\n systemctl --user daemon-reload\n systemctl --user restart nanoclaw-v2-.service\n ```\n4. Verify only one instance runs: `ps aux | grep nanoclaw/dist/index.js | grep -v grep`\n\nMessages marked delivered with a null `platform_message_id` are not automatically retried. Ask the user to resend.\n\n### 2. Container exits immediately / agent produces no reply\n\nA spawned container that exits without writing to `outbound.db` shows up in `logs/nanoclaw.log` as a `Container exited` line with a non-zero `code`, often preceded by streamed `container=` stderr (at debug level).\n\n**Authentication errors:** secrets are injected per request by the OneCLI gateway — none are passed in env vars or chat context. A `401` from an API whose credential is in the vault usually means the agent is in `selective` secret mode and that secret was never assigned:\n```bash\nonecli agents list # check secretMode\nonecli agents set-secret-mode --id --mode all # inject all matching secrets\n```\nIf the gateway itself is unreachable, the container runner refuses to spawn (`OneCLI gateway not applied — refusing to spawn container without credentials` in the host log). Confirm the gateway is up at `http://127.0.0.1:10254`.\n\n**MCP server failures:** a misconfigured MCP server can abort the agent run. Look for MCP initialization errors in the streamed container stderr (`LOG_LEVEL=debug`).\n\n### 3. Mount Issues\n\nSession and group folders are bind-mounted into the container. To see the resolved mounts for a spawn, run with `LOG_LEVEL=debug` and read the spawn command in `logs/nanoclaw.log`, or grep the mount targets directly:\n\n```bash\ngrep -n \"containerPath\" src/container-runner.ts\n```\n\nExpected mount targets inside the container:\n```\n/workspace ← session folder (inbound.db, outbound.db, .heartbeat, inbox/, outbox/)\n/workspace/agent ← agent group folder (cwd; CLAUDE.md, skills, working files)\n/home/node/.claude ← per-group .claude-shared (Claude state, settings, history)\n/app/src ← agent-runner source (read-only)\n/app/skills ← container skills (read-only)\n```\n\nTo inspect what a fresh container sees:\n```bash\ndocker run --rm --entrypoint /bin/bash nanoclaw-agent:latest -c 'whoami; ls -la /workspace/ /app/'\n```\nAll of `/workspace/` and `/app/` should be owned by `node`. Use `:ro` on a `-v` mount for read-only.\n\n### 4. Heartbeat / stale-session detection\n\nLiveness is a file `touch` on `/workspace/.heartbeat` (host path: `data/v2-sessions///.heartbeat`), not a DB write. The host sweep reads its mtime plus the `processing_ack` claim age to decide whether a container is alive or stale. A session stuck \"processing\" with a stale `.heartbeat` mtime means the container died mid-run:\n\n```bash\nstat -f '%Sm' data/v2-sessions///.heartbeat # macOS\nstat -c '%y' data/v2-sessions///.heartbeat # Linux\n```\n\n## Container CLI (`ncl`) inside a session\n\nThe agent reaches the central DB from inside the container via `ncl`, which uses the session DB transport (`container/agent-runner/src/cli/ncl.ts`). On the host, `ncl` connects over a Unix socket (`src/cli/socket-server.ts`). If `ncl` calls fail from inside a container, check the agent group's `cli_scope` in its container config:\n\n```bash\nncl groups config get --id # look at cli_scope: disabled | group | global\n```\n\n`disabled` rejects every `cli_request`; `group` scopes the agent to its own group's `groups`/`sessions`/`destinations`/`members`; `global` is unrestricted.\n\n## Restarting a session's container\n\n```bash\n# Restart all containers for an agent group\nncl groups restart --id \n\n# Restart and rebuild the image first (after package/Dockerfile changes)\nncl groups restart --id --rebuild\n\n# Restart and wake immediately with a message\nncl groups restart --id --message \"on_wake test\"\n```\n\nWithout `--message`, the container comes back on the next user message. From inside a container, `--id` is auto-filled and only the calling session restarts.\n\n## Manual Container Probes\n\nThe container's entry point is `exec bun run /app/src/index.ts`; it talks only to the mounted session DBs, so there is no JSON to pipe in. To probe the image directly:\n\n```bash\n# Interactive shell in the image\ndocker run --rm -it --entrypoint /bin/bash nanoclaw-agent:latest\n\n# Check the image contents\ndocker run --rm --entrypoint /bin/bash nanoclaw-agent:latest -c '\n node --version\n bun --version\n ls /app/src/\n'\n```\n\n## Provider SDK Options\n\nThe default provider wraps the Claude Agent SDK in `container/agent-runner/src/providers/claude.ts`. The query is configured roughly as:\n\n```typescript\nquery({\n prompt: input.prompt,\n options: {\n cwd: input.cwd, // /workspace/agent\n allowedTools: [...TOOL_ALLOWLIST, ...mcpAllowPatterns],\n disallowedTools: SDK_DISALLOWED_TOOLS,\n permissionMode: 'bypassPermissions',\n settingSources: ['project', 'user', 'local'],\n mcpServers: { ... },\n },\n})\n```\n\nEach registered MCP server's allow pattern is derived from the `mcpServers` map, so registering a server already exposes its tools.\n\n## Rebuilding After Changes\n\n```bash\n# Rebuild host TypeScript\npnpm run build\n\n# Rebuild the agent container image\n./container/build.sh\n\n# Force a truly clean rebuild (the buildkit cache retains stale COPY files)\ndocker builder prune -af\n./container/build.sh\n```\n\n## Clearing a Session\n\nConversation continuity lives in the container-owned `session_state` table in `outbound.db` (the provider's session/continuation id). The agent's `/clear` clears it. To reset a session from the host, remove the session folder so a fresh one is provisioned on the next message:\n\n```bash\n# Inspect first\nncl sessions get \n\n# Remove a single session's folder (host re-provisions both DBs on next message)\nrm -rf data/v2-sessions///\n```\n\n## Quick Diagnostic Script\n\n```bash\necho \"=== Checking NanoClaw v2 Setup ===\"\n\necho -e \"\\n1. Container runtime running?\"\ndocker info &>/dev/null && echo \"OK\" || echo \"NOT RUNNING - start Docker Desktop (macOS) or sudo systemctl start docker (Linux)\"\n\necho -e \"\\n2. Agent image exists?\"\ndocker run --rm --entrypoint /bin/echo nanoclaw-agent:latest \"OK\" 2>/dev/null || echo \"MISSING - run ./container/build.sh\"\n\necho -e \"\\n3. OneCLI gateway reachable?\"\ncurl -fsS http://127.0.0.1:10254/ >/dev/null 2>&1 && echo \"OK\" || echo \"CHECK - gateway not responding on 127.0.0.1:10254\"\n\necho -e \"\\n4. Central DB present?\"\n[ -f data/v2.db ] && echo \"OK\" || echo \"MISSING - run setup\"\n\necho -e \"\\n5. Mount targets in container-runner?\"\ngrep -q \"containerPath: '/workspace'\" src/container-runner.ts && echo \"OK\" || echo \"CHECK - session mount target changed\"\n\necho -e \"\\n6. Single host instance running?\"\nN=$(ps aux | grep 'nanoclaw/dist/index.js' | grep -vc grep)\n[ \"$N\" -le 1 ] && echo \"OK ($N)\" || echo \"DUPLICATE - $N instances; stop the stale one (see issue 1)\"\n\necho -e \"\\n7. Recent host errors?\"\ntail -n 5 logs/nanoclaw.error.log 2>/dev/null || echo \"No error log yet\"\n```","author":"@nanocoai","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/nanocoai/nanoclaw/tree/main/.claude/skills/debug","license":"MIT","category":null,"lang":"en","tokens":3439,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"injection.disable-permissions","kind":"injection","where":"SKILL.md:231","excerpt":"bypassPermissions","message":"instructs the agent to disable permission checks","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":[]}}