{"id":"agent-research-aggregator","name":"agent-research-aggregator","summary":"APIの前集約器で、AIエージェントキャッシュディレクトリ(.claude、.cursor、.antigravity、.openclaw)やユーザーが指定した任意のディレクトリをスキャンし、インサイトや数値結果を抽出し、PaperOrchestra対応の入力(idea.md + experimental_log.md…","body":"# agent-research-aggregator\n\n---\n\n## Should I run? (decision gate)\n\nBefore starting Phase 1, check whether aggregation is actually needed:\n\n| Situation | Action |\n|---|---|\n| `workspace/inputs/idea.md` **and** `workspace/inputs/experimental_log.md` both exist and are non-empty | **Skip this skill entirely.** Proceed directly to `paper-orchestra`. |\n| Either file is missing or empty, **and** the user provided a directory path | **Run this skill** with that directory as `--search-roots`. |\n| Either file is missing or empty, **and** no directory was provided | Scan cwd and `~` by default; show the discovery summary to the user before continuing. |\n| The inputs exist but look thin (e.g. idea.md has < 5 lines, no numeric data in experimental_log.md) | **Ask the user** whether to supplement with aggregation or proceed as-is. |\n\nThe skill is intentionally a pre-pass — it is cheap to skip and should only run when the structured inputs don't already exist.\n\n---\n\nA pre-processing skill for PaperOrchestra (arXiv:2604.05018). Reads scattered\nexperimentation artifacts from AI coding-agent cache directories and synthesizes\nthem into the structured `(I, E)` input pair the PaperOrchestra pipeline expects.\n\n```\n[.claude/]  [.cursor/]  [.antigravity/]  [.openclaw/]\n      │            │              │               │\n      └────────────┴──────────────┴───────────────┘\n                          │\n                    Phase 1: Discovery\n                  (discover_logs.py)\n                          │\n                    discovered_logs.json\n                          │\n                    Phase 2: Extraction\n                  (LLM call per log batch)\n                          │\n                    raw_experiments.json\n                          │\n                    Phase 3: Synthesis\n                  (LLM call — consolidate)\n                          │\n                    synthesis.json\n                          │\n                    Phase 4: Formatting\n                  (format_po_inputs.py)\n                          │\n             ┌────────────┴────────────┐\n      workspace/inputs/         workspace/ara/\n        idea.md                   aggregation_report.md\n        experimental_log.md       discovered_logs.json\n                                  raw_experiments.json\n                                  synthesis.json\n```\n\nThe output drops directly into `workspace/inputs/` so the user can immediately\nrun `paper-orchestra` on the same workspace.\n\n---\n\n## Inputs\n\n| Parameter | Required | Default | Description |\n|---|---|---|---|\n| `--search-roots` | no | cwd, `~` | Comma-separated directories to scan for agent caches |\n| `--agents` | no | all | Comma-separated subset: `claude,cursor,antigravity,openclaw` |\n| `--workspace` | no | `./workspace` | PaperOrchestra workspace root |\n| `--depth` | no | 4 | Max directory scan depth (prevents runaway scans on large home dirs) |\n| `--since` | no | none | Only include logs modified after this date (ISO 8601: `2025-01-01`) |\n\nThe user specifies these when invoking the skill, or you may ask them for\n`--search-roots` if the current directory has no detectable agent caches.\n\n---\n\n## Phase 1 — Discovery (deterministic)\n\nRun the discovery script to catalog every relevant log file:\n\n```bash\npython skills/agent-research-aggregator/scripts/discover_logs.py \\\n    --search-roots <roots> \\\n    --agents <agents> \\\n    --depth <depth> \\\n    --since <since> \\\n    --out workspace/ara/discovered_logs.json\n```\n\nThe script exits with code **2** when no `--project` filter is set (this is\nexpected on the first run). It prints a **\"Projects found\"** list to stdout —\nshow it to the user immediately.\n\n**If no logs are found at all:** stop and ask the user to specify\n`--search-roots` or point you at a directory that contains agent cache folders.\n\n---\n\n## Phase 1.5 — Project Selection (mandatory)\n\n**A paper can only be written from a single project. You must ask the user\nwhich project to use before any LLM processing begins.**\n\n1. Display the numbered project list from the discovery summary, e.g.:\n   ```\n   Projects found:\n     [1] /home/alice/projects/my-rl-experiment  (42 files)\n     [2] /home/alice/projects/llm-eval-suite    (17 files)\n     [3] /home/alice/projects/old-demo          (3 files)\n   ```\n2. Ask: *\"Which project should this paper be based on? Please choose a number\n   or paste the project path.\"*\n3. **Do not proceed to Phase 2 until the user has answered.**\n4. Re-run discovery with the chosen project to filter the manifest:\n\n```bash\npython skills/agent-research-aggregator/scripts/discover_logs.py \\\n    --search-roots <roots> \\\n    --agents <agents> \\\n    --depth <depth> \\\n    --since <since> \\\n    --project \"<chosen project path>\" \\\n    --out workspace/ara/discovered_logs.json\n```\n\nThis overwrites `discovered_logs.json` so only the selected project's files\nremain. The script exits 0 on success.\n\n**If the discovery finds only one project:** skip the question and inform the\nuser: *\"Only one project found: `<path>`. Using it for the paper.\"* — then\nre-run with `--project` automatically.\n\n**If the discovery summary shows irrelevant files after filtering:** ask the\nuser whether to include or exclude them before continuing to Phase 2. Err on\nthe side of inclusion — the extraction prompt is conservative.\n\n---\n\n## Phase 2 — Extraction (LLM-assisted)\n\nProcess discovered logs in **batches** (group by agent type; keep batches under\n~50 KB of raw text to stay within context limits):\n\nFor each batch:\n\n1. **Read** the log files in the batch (the script's `--list` output tells you\n   which file paths to read).\n2. **Apply the extraction prompt** from `references/extraction-prompt.md` as\n   your system message.\n3. **Pass the raw log text** as the user message.\n4. **Collect the structured JSON** the LLM returns (see schema in the prompt).\n5. **Append** to `workspace/ara/raw_experiments.json`.\n\nAfter all batches:\n\n```bash\npython skills/agent-research-aggregator/scripts/extract_experiments.py \\\n    --discovered workspace/ara/discovered_logs.json \\\n    --out workspace/ara/raw_experiments.json \\\n    --validate-only\n```\n\nRun this in `--validate-only` mode to check the combined JSON is well-formed\nand meets the minimum schema (`experiments` array non-empty, each entry has\n`hypothesis` or `method` or `results`). Fix any malformed entries before Phase 3.\n\n---\n\n## Phase 3 — Synthesis (LLM-assisted)\n\nConsolidate possibly-redundant experiment records from multiple agent caches into\na single coherent research narrative. This is ONE LLM call.\n\n**System message:** Use `references/synthesis-prompt.md` verbatim.\n\n**User message:**\n```\n<raw_experiments>\n{contents of workspace/ara/raw_experiments.json}\n</raw_experiments>\n```\n\nThe LLM must return a `synthesis.json` with keys:\n- `research_question` — the overarching question being investigated\n- `hypothesis` — the core proposed solution / claim\n- `method_summary` — how the approach works (concise, no data leakage)\n- `key_contributions` — 2–5 bullet strings\n- `experimental_setup` — datasets, metrics, baselines, implementation notes\n- `results_tables` — array of `{title, headers[], rows[]}` markdown-table objects\n- `qualitative_observations` — free-form text blocks (what worked, what didn't,\n  failure modes, ablation insights)\n- `iteration_history` — ordered list of `{iteration_id, change_description,\n  outcome}` entries if multiple iterations are detected\n- `open_questions` — questions that remain unanswered in the logs\n\nSave to `workspace/ara/synthesis.json`.\n\n> **Note:** By this point, the user has already selected a single project in\n> Phase 1.5. The synthesis should represent one coherent research thread. If\n> the LLM still surfaces multiple disconnected research questions, flag this\n> as a data quality warning in the audit report (Phase 5) but do not re-ask\n> for project selection — that decision was made earlier.\n\n---\n\n## Phase 4 — Formatting (deterministic)\n\nConvert `synthesis.json` into PaperOrchestra input files:\n\n```bash\npython skills/agent-research-aggregator/scripts/format_po_inputs.py \\\n    --synthesis workspace/ara/synthesis.json \\\n    --out workspace/inputs/\n```\n\nThis generates two files:\n\n### `workspace/inputs/idea.md` (Sparse variant)\n\nFollows the PaperOrchestra Sparse Idea format (arXiv:2604.05018, §3.1):\n\n```markdown\n# [Synthesized Research Title]\n\n## Problem\n<2–4 sentence problem statement derived from research_question>\n\n## Hypothesis\n<hypothesis from synthesis>\n\n## Method\n<method_summary from synthesis>\n\n## Key Contributions\n<key_contributions as bullet list>\n\n## Open Questions\n<open_questions, if any>\n```\n\n### `workspace/inputs/experimental_log.md`\n\nFollows the PaperOrchestra Experimental Log format (App. D.3):\n\n```markdown\n## 1. Experimental Setup\n<experimental_setup from synthesis, formatted as prose + sub-bullets>\n\n## 2. Raw Numeric Data\n<results_tables converted to GitHub-Flavored Markdown tables>\n\n## 3. Qualitative Observations\n<qualitative_observations from synthesis>\n\n### Iteration History\n<iteration_history as an ordered narrative, if present>\n```\n\nAfter running the script, **review both files** with the user:\n\n1. Read `workspace/inputs/idea.md` aloud and ask: \"Does this accurately capture\n   your research question and method?\"\n2. Read the table headers from `workspace/inputs/experimental_log.md` and ask:\n   \"Are these the correct metrics and baselines?\"\n\nRevise based on feedback before proceeding to PaperOrchestra.\n\n---\n\n## Phase 5 — Audit Report (deterministic)\n\n```bash\npython skills/agent-research-aggregator/scripts/format_po_inputs.py \\\n    --synthesis workspace/ara/synthesis.json \\\n    --out workspace/inputs/ \\\n    --report workspace/ara/aggregation_report.md\n```\n\nThe `--report` flag makes the script also write `aggregation_report.md`, which\ncontains:\n\n- Number of agent caches scanned, files read, batches processed\n- Per-agent breakdown (files found per agent type)\n- Experiment records extracted (count, date range)\n- Iterations detected (count, convergence direction)\n- Data quality warnings (gaps, low-confidence extractions, conflicting numbers)\n- Files written and their sizes\n\nShow the report to the user. If the data quality section lists warnings, discuss\nthem before running paper-orchestra — garbage in, garbage out.\n\n---\n\n## Handoff to PaperOrchestra\n\nOnce the user has confirmed `idea.md` and `experimental_log.md`, the workspace\nis ready for the paper-orchestra pipeline. You still need:\n\n| File | Status | Action |\n|---|---|---|\n| `workspace/inputs/idea.md` | ✓ generated | user review recommended |\n| `workspace/inputs/experimental_log.md` | ✓ generated | user review recommended |\n| `workspace/inputs/template.tex` | **MISSING** | ask user to provide their conference LaTeX template |\n| `workspace/inputs/conference_guidelines.md` | **MISSING** | ask user to provide (page limit, deadline, formatting rules) |\n\nTell the user exactly which two files are still needed, then offer to run\n`paper-orchestra` once they supply them.\n\n---\n\n## Error handling\n\n| Situation | Action |\n|---|---|\n| Cache directory does not exist | Skip silently; note in report |\n| File is binary or non-text | Skip; note in report |\n| File > 200 KB | Truncate at 200 KB; note in report with path |\n| LLM extraction returns malformed JSON | Re-prompt once with the parse error appended; if still malformed, log the batch as `status: failed` and continue |\n| Synthesis returns > 1 `research_question` | Log as data quality warning in audit report; do not re-ask for project (was selected in Phase 1.5) |\n| `results_tables` is empty after synthesis | Warn the user — PaperOrchestra's section-writing agent needs numeric data |\n\n---\n\n## Hard rules (never violate)\n\n1. **Never write to agent cache directories.** This skill is read-only on `.claude/`, `.cursor/`, `.antigravity/`, `.openclaw/`.\n2. **Never include personal information** (emails, names, credentials, API keys) in generated `idea.md` or `experimental_log.md`. The extraction prompt instructs the LLM to strip PII; double-check before handoff.\n3. **Never fabricate results.** If a metric appears in only one log with low confidence, mark it `[UNVERIFIED]` in the table rather than silently including it.\n4. **Never proceed past Phase 1 without user confirmation** of the discovered file list if the scan found > 50 files.\n\n---\n\n## Quick reference\n\n```bash\n# Phase 1: discover all projects (exits with code 2 — project selection required)\npython skills/agent-research-aggregator/scripts/discover_logs.py \\\n    --search-roots . ~ --out workspace/ara/discovered_logs.json\n\n# Phase 1.5: re-run with chosen project (exits 0)\npython skills/agent-research-aggregator/scripts/discover_logs.py \\\n    --search-roots . ~ \\\n    --project \"/home/user/projects/my-chosen-project\" \\\n    --out workspace/ara/discovered_logs.json\n\n# ... (Phase 2: LLM extraction calls, see above) ...\n\npython skills/agent-research-aggregator/scripts/extract_experiments.py \\\n    --discovered workspace/ara/discovered_logs.json \\\n    --out workspace/ara/raw_experiments.json --validate-only\n\n# ... (Phase 3: LLM synthesis call, see above) ...\n\npython skills/agent-research-aggregator/scripts/format_po_inputs.py \\\n    --synthesis workspace/ara/synthesis.json \\\n    --out workspace/inputs/ \\\n    --report workspace/ara/aggregation_report.md\n```","author":"@Ar9av","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/Ar9av/PaperOrchestra/tree/main/skills/agent-research-aggregator","license":"MIT","category":"writing","lang":"en","tokens":3140,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/extraction-prompt.md","size":4567,"sha256":"5b2a52fad4ce9465008adc49179b8959597b9e39c89c945b81d710e04fca24b0"},{"path":"references/log-formats.md","size":5615,"sha256":"aba7efe7f2f005af920012f7f4bba4e2b981709a3bac57c9bd6b6919f0bb125a"},{"path":"references/synthesis-prompt.md","size":5236,"sha256":"f7bca6add0277973cd28b5b6283d44733599dd5f6f2fd5e1062bb8cae5681108"},{"path":"scripts/discover_logs.py","size":16062,"sha256":"9eaeef682783900ab6b3c6e8d66ad5e8e0cc6e163551d1fb99b12ac8052a74af"},{"path":"scripts/extract_experiments.py","size":8070,"sha256":"320fa10ec4117efc41b429870bc42ccf4cf8908952db32e2a788eec215402f17"},{"path":"scripts/format_po_inputs.py","size":13765,"sha256":"0acccd072d88feac7b5e4ba4274d5f10cf843bfd5f80eb719659676558a6e43e"}],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"code.credentials","kind":"dangerous-code","where":"scripts/discover_logs.py:125","excerpt":"credentials.json","message":"reads credential files","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":true,"networkEndpoints":[]}}