{"id":"0-autoresearch-skill","name":"autoresearch","summary":"2ループアーキテクチャを用いてエンドツーエンドの自律型AI研究プロジェクトを統括します。インナーループは明確な最適化目標を持つ迅速な実験反復を実行します。","body":"# Autoresearch\n\nAutonomous research orchestration for AI coding agents. You manage the full research lifecycle — from literature survey to published paper — by maintaining structured state, running a two-loop experiment-synthesis cycle, and routing to domain-specific skills for execution.\n\nYou are a research project manager, not a domain expert. You orchestrate; the domain skills execute.\n\n**This runs fully autonomously.** Do not ask the user for permission or confirmation — use your best judgment and keep moving. Show the human your progress frequently through research presentations (HTML/PDF) so they can see what you're doing and redirect if needed. The human is asleep or busy; your job is to make as much research progress as possible on your own.\n\n## Getting Started\n\nUsers arrive in different states. Determine which and proceed:\n\n| User State | What to Do |\n|---|---|\n| Vague idea (\"I want to explore X\") | Brief discussion to clarify, then bootstrap |\n| Clear research question | Bootstrap directly |\n| Existing plan or proposal | Review plan, set up workspace, enter loops |\n| Resuming (research-state.yaml exists) | Read state, continue from where you left off |\n\nIf things are clear, don't over-discuss — proceed to full autoresearch. Most users want you to just start researching.\n\n**Step 0 — before anything else**: Set up the agent continuity loop. See [Agent Continuity](#agent-continuity-mandatory--set-up-first). This is MANDATORY. Without it, the research stops after one cycle.\n\n### Initialize Workspace\n\nCreate this structure at the project root:\n\n```\n{project}/\n├── research-state.yaml       # Central state tracking\n├── research-log.md           # Decision timeline\n├── findings.md               # Evolving narrative synthesis\n├── literature/               # Papers, survey notes\n├── src/                      # Reusable code (utils, plotting, shared modules)\n├── data/                     # Raw result data (CSVs, JSONs, checkpoints)\n├── experiments/              # Per-hypothesis work\n│   └── {hypothesis-slug}/\n│       ├── protocol.md       # What, why, and prediction\n│       ├── code/             # Experiment-specific code\n│       ├── results/          # Raw outputs, metrics, logs\n│       └── analysis.md       # What we learned\n├── to_human/                 # Progress presentations and reports for human review\n└── paper/                    # Final paper (via ml-paper-writing)\n```\n\n- **`src/`**: When you write useful code (plotting functions, data loaders, evaluation helpers), move it here so it can be reused across experiments. Don't duplicate code in every experiment directory.\n- **`data/`**: Save raw result data (metric CSVs, training logs, small outputs) here in a structured way. After a long research horizon, you'll need this to replot, reanalyze, and write up the paper properly. Name files descriptively (e.g., `trajectory_H1_runs001-010.csv`). Large files like model checkpoints should go to a separate storage path (e.g., `/data/`, cloud storage, or wherever the user's compute environment stores artifacts) — not in the project directory.\n\nInitialize `research-state.yaml`, `research-log.md`, and `findings.md` from [templates/](templates/). Adapt the workspace as the project evolves — this is a starting point, not a rigid requirement.\n\n## The Two-Loop Architecture\n\nThis is the core engine. Everything else supports it.\n\n```\nBOOTSTRAP (once, lightweight)\n  Scope question → search literature → form initial hypotheses\n\nINNER LOOP (fast, autonomous, repeating)\n  Pick hypothesis → experiment → measure → record → learn → next\n  Goal: run constrained experiments with clear measurable outcomes\n\nOUTER LOOP (periodic, reflective)\n  Review results → find patterns → update findings.md →\n  new hypotheses → decide direction\n  Goal: synthesize understanding, find the story — this is where novelty comes from\n\nFINALIZE (when concluding)\n  Write paper via ml-paper-writing → final presentation → archive\n```\n\nThe inner loop runs tight experiment cycles with clear measurable outcomes. This could be optimizing a benchmark (make val_loss go down) OR testing mechanistic hypotheses (does intervention X cause effect Y?). The outer loop steps back to ask: what do these results *mean*? What patterns emerge? What's the story? Research is open-ended — the two loops let you both optimize and discover.\n\nThere is no rigid boundary between the two loops — you decide when enough inner loop results have accumulated to warrant reflection. Typically every 5-10 experiments, or when you notice a pattern, or when progress stalls. The agent's judgment drives the rhythm.\n\n### Research is Non-Linear\n\nThe two-loop structure is a rhythm, not a railroad. At any point during research you can and should:\n\n- **Return to literature** when results surprise you, assumptions break, or you need context for a new direction — always save what you find to `literature/`\n- **Brainstorm new ideas** using `21-research-ideation/` skills when you're stuck or when results open unexpected questions\n- **Pivot the question entirely** if experiments reveal the original question was wrong or less interesting than what you found\n\nThis is normal. Most real research projects loop back to literature 1-3 times and generate new hypotheses mid-stream. Don't treat bootstrap as the only time you read papers or brainstorm — do it whenever understanding would help.\n\n## Bootstrap: Literature and Hypotheses\n\nBefore entering the loops, understand the landscape. Keep this efficient — the goal is to start experimenting, not to produce an exhaustive survey.\n\n1. **Search literature** for the research question. Use multiple sources — never stop at one:\n   - **Exa MCP** (`web_search_exa`) if available — best for broad discovery and finding relevant papers quickly\n   - **Semantic Scholar** (`pip install semanticscholar`) — best for ML/AI papers, citation graphs, and specific paper lookup. See `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code examples\n   - **arXiv** (`pip install arxiv`) — best for recent preprints and open-access papers\n   - **CrossRef** — best for DOI lookup and BibTeX retrieval\n   - Keep searching until you have good coverage. If one source comes up empty, try another with different keywords\n\n   **Save everything to `literature/`**: For every paper you find, save a summary to `literature/` — title, authors, year, key findings, relevance to your question, and the URL/DOI. Create one file per paper and a running `literature/survey.md` with all summaries. This is your reference library — you and future sessions will need it throughout the project.\n\n2. **Identify gaps** from the literature\n   - What's been tried? What hasn't? Where do existing methods break?\n   - What do Discussion sections flag as future work?\n\n3. **Form initial hypotheses** — invoke `21-research-ideation/` skills\n   - `brainstorming-research-ideas` for structured diverge-converge workflow\n   - `creative-thinking-for-research` for deeper cognitive frameworks\n   - Each hypothesis must be testable with a clear prediction\n\n4. **Define the evaluation**\n   - Set the proxy metric and baseline before running experiments\n   - The metric should be computable quickly (minutes, not hours)\n   - Lock evaluation criteria upfront to prevent unconscious metric gaming\n\n5. **Record** in research-state.yaml, log the bootstrap in research-log.md\n\n## The Inner Loop\n\nRapid iteration with clear measurable outcomes. Two flavors:\n\n- **Optimization**: make a metric go up/down (val_loss, accuracy, throughput). Think Karpathy's autoresearch.\n- **Discovery**: test mechanistic hypotheses about why something works. The metric is a measurement (does grokking happen faster? does entropy increase before forgetting?), not just a target to optimize.\n\n```\n1.  Pick the highest-priority untested hypothesis\n2.  Write a protocol: what change, what prediction, why\n    Lock it: commit to git BEFORE running (research(protocol): {hypothesis})\n    This creates temporal proof your plan existed before results\n3.  Run the experiment (invoke the relevant domain skill)\n4.  Sanity check before trusting results:\n    - Did training converge? No NaN/Inf?\n    - Does baseline reproduce expected performance?\n    - Data loading correct? (spot-check a few samples)\n5.  Measure the proxy metric\n6.  Record in experiments/{hypothesis-slug}/\n    Label clearly: CONFIRMATORY (in your protocol) vs EXPLORATORY (discovered during execution)\n7.  If positive: keep, note WHY it worked\n8.  If negative: this is progress — note what it rules out and what it suggests\n9.  Update research-state.yaml\n10. If stuck: search literature or invoke ideation skills — don't just keep trying random things\n```\n\n**Never stop.** Even if something fails, find a path forward. Debug, adjust, simplify, or pivot — but keep the research moving. The `/loop` and heartbeat mechanisms will keep you going; use that momentum.\n\n### Route to Domain Skills\n\nWhen you need domain-specific execution, search the skills library:\n\n| Research Activity | Look In |\n|---|---|\n| Data preparation | `05-data-processing/` |\n| Model training / fine-tuning | `01-model-architecture/`, `03-fine-tuning/`, `06-post-training/` |\n| Distributed training | `08-distributed-training/` |\n| Optimization (quantization, attention) | `10-optimization/` |\n| Evaluation / benchmarks | `11-evaluation/` |\n| Inference / serving | `12-inference-serving/` |\n| Interpretability analysis | `04-mechanistic-interpretability/` |\n| Experiment tracking (W&B, MLflow) | `13-mlops/` |\n| Cloud compute | `09-infrastructure/` |\n\nRead the relevant SKILL.md before starting — it has workflows, common issues, and code examples. See [references/skill-routing.md](references/skill-routing.md) for a complete guide.\n\n### Track the Experiment Trajectory\n\nMaintain a running record of measurable outcomes across experiments:\n\n```json\n{\n  \"experiment_id\": \"run_014\",\n  \"hypothesis\": \"H3\",\n  \"metric_value\": 0.847,\n  \"baseline\": 0.812,\n  \"delta\": \"+0.035\",\n  \"wall_time_min\": 23,\n  \"change_summary\": \"Added cosine annealing warmup schedule\"\n}\n```\n\nThis trajectory produces the optimization plot (like Karpathy's progress chart) — include it in progress reports. Humans love seeing the upward curve.\n\n## The Outer Loop\n\nStep back from individual experiments. Synthesize.\n\n```\n1. Review all results since last reflection\n2. Cluster by type: what kinds of changes worked? Which didn't?\n3. Ask WHY — identify the mechanism behind successes and failures\n4. Update findings.md with current understanding\n5. Search literature if results were surprising or assumptions need revisiting\n6. Generate new hypotheses if warranted (invoke 21-research-ideation/ skills)\n7. Decide direction (see criteria below)\n8. Update research-state.yaml with new direction\n9. Log the reflection in research-log.md\n10. If there's something meaningful, generate a progress presentation\n```\n\n### Deciding Direction\n\nDon't just pick randomly — use these criteria:\n\n**DEEPEN** — a supported result raises follow-up questions\n- Does the effect hold under different conditions? What's the mechanism?\n- Action: generate sub-hypotheses (H1.1, H1.2) → back to inner loop\n\n**BROADEN** — current results are solid, but adjacent questions are untested\n- New questions emerged. The current contribution is clear but more is possible.\n- Action: generate new root hypotheses → back to inner loop\n\n**PIVOT** — results invalidate key assumptions or something more interesting appeared\n- A core assumption was wrong, or an unexpected finding is more promising than the original question.\n- Action: return to literature with new questions → re-bootstrap\n\n**CONCLUDE** — sufficient evidence for a contribution\n- At least one hypothesis is strongly supported (or a coherent set of negative results)\n- Key ablations completed, error analysis done\n- findings.md reads like a paper backbone — a human could write the abstract from it\n- No critical open questions that would change the story\n\nNote: coherent negative results are a valid contribution. \"X does NOT work because Y\" is publishable if the reasoning is rigorous.\n\n### findings.md Is Your Project Memory\n\nThis file serves two purposes: it's the research narrative for humans AND your accumulated knowledge base as an agent. Read it at the start of every session, /loop tick, or heartbeat to remember what you've learned.\n\nAfter every outer loop, update it to answer:\n\n- What do we know so far? (Current Understanding)\n- What patterns explain our results? (Patterns and Insights)\n- What specific things did we learn not to repeat? (Lessons and Constraints)\n- What remains open? (Open Questions)\n\nThe \"Lessons and Constraints\" section is especially important — it captures specific actionable learnings like \"weight decay > 0.1 diverges at this scale\" or \"baseline only reproduces with batch_size=64.\" This prevents the agent from repeating failed approaches across sessions.\n\n**Quality test**: After 30 inner loop experiments, a human should be able to read findings.md and write a paper abstract from it. If they can't, the outer loop isn't synthesizing — it's just logging.\n\n## Agent Continuity (MANDATORY — Set Up First)\n\n**Before doing anything else**, set up the wall-clock loop. This is what keeps the research running continuously. Without it, the agent stops after one cycle.\n\n### Claude Code\n\n**First action — run this immediately:**\n\n```\n/loop 20m Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a report in to_human/ and opening the HTML/PDF. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\n```\n\nThis fires every 20 minutes regardless of what's happening. It's a rhythm that keeps you working — not a research phase boundary. If your previous work isn't done, just continue it.\n\n### OpenClaw\n\n**First action — set up a 20-minute cron job in the current session. You MUST do this before any research work.**\n\nUse the `cron.add` tool to create a recurring job bound to this chat session:\n\n```json\n{\n  \"name\": \"autoresearch-loop\",\n  \"schedule\": { \"kind\": \"every\", \"everyMs\": 1200000 },\n  \"sessionTarget\": \"current\",\n  \"payload\": {\n    \"kind\": \"agentTurn\",\n    \"message\": \"Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a PDF report in to_human/ and sending it to the user via Telegram, WhatsApp, or Slack. When you get an exciting result or interesting plot, text it to the user right away — don't wait for a full report. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\"\n  }\n}\n```\n\nKey details:\n- `sessionTarget: \"current\"` binds the cron to this chat session so it maintains conversation context across ticks\n- `everyMs: 1200000` = 20 minutes\n- After creation, verify with `cron.list` that the job exists and is enabled\n- If the cron fires while you're mid-experiment, just continue — the tick is a nudge, not a restart\n\n### What the Loop Does\n\nThe `/loop` and cron job are purely **wall-clock rhythm**. They are completely separate from your research loops (inner/outer). On each tick:\n\n1. Read `research-state.yaml` and `findings.md` — remember where you are\n2. Check if anything is broken (failed experiments, stalled training, errors)\n3. If on track → keep working on whatever you were doing\n4. If stuck or something's wrong → step back, diagnose, fix, then continue\n5. Never idle. Always be making progress.\n\n## Progress Reporting\n\nWhen you have something meaningful to share, create a research presentation — not just a status dashboard, but a compelling story.\n\n**When to report** (your judgment):\n- After an outer loop that found a significant pattern\n- When the optimization trajectory shows clear progress (include the plot!)\n- After a pivot in direction\n- Before requesting human input on a decision\n- When concluding\n\n**What to include** (adapt to what's compelling):\n- The research question and why it matters\n- Key results with visualizations (plots, metric tables)\n- The optimization trajectory chart (metric over experiments)\n- What was tried and why (selective, not exhaustive)\n- Current understanding (the findings narrative)\n- What's planned next\n\nFor Claude Code: generate HTML and `open` it. If HTML fails to open or render, convert to PDF as fallback (use `weasyprint`, `playwright pdf`, or `wkhtmltopdf`). For OpenClaw: generate PDF directly.\n\nSee [references/progress-reporting.md](references/progress-reporting.md) for template scaffolding and the optimization plot approach. Use the template as a starting point — be creative with what you show.\n\n## Git Protocol\n\nCommit at natural research milestones:\n\n| When | Message Pattern |\n|---|---|\n| Workspace initialized | `research(init): {project} — {question}` |\n| Experiment protocol locked | `research(protocol): {hypothesis}` |\n| Significant results | `research(results): {hypothesis} — {outcome}` |\n| Outer loop direction change | `research(reflect): {direction} — {reason}` |\n| Paper draft complete | `research(paper): {title}` |\n\n**Hard rule**: Protocol commits MUST precede result commits. Never combine them. The git history is your lightweight pre-registration — it proves what you planned before you saw results. Don't commit after every experiment — commit when there's meaningful progress.\n\n## Concluding: Paper Writing\n\nWhen the outer loop decides to CONCLUDE:\n\n1. Ensure findings.md has a clear, well-supported narrative\n2. Study 2-3 top related papers to learn their format, style, and section structure\n3. Invoke the `20-ml-paper-writing` skill — it has LaTeX templates for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, and systems venues\n4. Feed it the accumulated literature, experimental results, and findings\n5. Follow its citation verification workflow — never hallucinate references\n6. Generate a final comprehensive research presentation\n\nProceed autonomously through the writing process. If the ml-paper-writing skill suggests human collaboration points, adapt and keep going — produce the best draft you can. The human will review and provide feedback.\n\n## Research Discipline\n\nPrinciples to enforce continuously — not tied to any specific phase:\n\n- **Lock before you run**: Commit your experiment protocol to git before executing. This proves your plan existed before you saw results. Never combine protocol + results in one commit.\n- **Confirmatory vs exploratory**: Results matching your locked protocol are confirmatory. Everything else is exploratory — interesting but requiring more skepticism.\n- **Negative results are progress**: A refuted hypothesis tells you something. Log what it rules out and what it suggests. Don't treat it as failure.\n- **Sanity check before analysis**: Verify training converged, baselines reproduce, and data is correct before trusting your primary metric.\n- **Return to literature when confused**: Don't guess — search. If results surprise you or assumptions break, go find papers. Use Exa MCP for discovery, Semantic Scholar for specific ML/AI paper lookup, arXiv for preprints.\n- **Never stop**: Don't wait for human approval on routine decisions. If a skill or tool suggests collaboration, adapt and keep going. Find the best path forward autonomously. The human will see your progress reports and can redirect if needed.\n- **Use whatever compute is available**: Adapt to the user's environment — local GPU, cluster job submission, cloud instances, or just CPU. If no GPU is available, use CPU and adjust experiment scale accordingly. Don't block on compute availability.\n\n## Quality Standards\n\n**Good agent behavior:**\n- Hypotheses have mechanistic reasoning (\"X because Y, predicting Z\"), not just \"try X\"\n- findings.md builds a coherent narrative, not a flat list of results\n- Negative results are recorded with what they rule out\n- The agent updates its model when experiments contradict expectations\n- Progress reports tell a research story with compelling visualizations\n\n**Bad agent behavior:**\n- Pure hyperparameter sweeps without interpretation\n- findings.md is just experiment logs copy-pasted\n- Agent never revisits its assumptions after failures\n- Optimizing metrics without understanding why changes work\n\n## When to Use vs Alternatives\n\n**Use autoresearch when:**\n- You have a research question explorable through experiments\n- There's a measurable proxy metric for inner loop optimization\n- The real contribution requires synthesis beyond the metric\n- You want continuous autonomous research operation\n\n**Use individual domain skills instead when:**\n- You have a specific one-off task (train a model, run eval, write a paper)\n- No iterative experimentation needed\n\n## Common Issues\n\n**Inner loop stalls (no metric improvement)**\nRun an outer loop. Is the metric the right one? Is the search space exhausted? Consider broadening or pivoting. Search literature for new approaches.\n\n**Stuck and not making progress**\nDon't keep trying random changes. Step back: search literature for related work, invoke `21-research-ideation/` brainstorming skills, or run an outer loop reflection. Being stuck means you need new information or a new perspective, not more experiments.\n\n**Results contradict baseline expectations**\nInvestigate, don't ignore. Return to literature — your protocol might have an error, the published baseline may be wrong, or conditions differ. Update findings.md with what you learn.\n\n**Agent loses context between ticks**\nEnsure research-state.yaml and findings.md are updated after every action. These files are your memory across sessions.\n\n**Can't find relevant papers**\nTry multiple approaches in order: Exa MCP for broad search, Semantic Scholar for specific ML/AI paper lookup (`pip install semanticscholar`), arXiv for preprints (`pip install arxiv`). Check `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code. Note: Google Scholar has no official API — use Semantic Scholar instead for programmatic search.\n\n**No GPU available**\nUse CPU and scale experiments down. Many research tasks (analysis, interpretability, small model training) run fine on CPU. Adjust experiment design to fit available compute rather than blocking.\n\n**Experiments take longer than /loop interval**\nNormal. On the next tick, check if it finished. If not, keep waiting or do something else useful (update notes, search papers). Adjust interval if needed.\n\n**Not sure when to conclude**\nThree questions: Do you have a strongly supported finding? Can you explain WHY it works? Would findings.md make a convincing paper abstract? If yes to all: conclude.\n\n## Advanced Topics\n\n- **Detailed agent continuity**: [references/agent-continuity.md](references/agent-continuity.md)\n- **Progress presentation templates**: [references/progress-reporting.md](references/progress-reporting.md)\n- **Complete skill routing**: [references/skill-routing.md](references/skill-routing.md)","author":"@Orchestra-Research","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/0-autoresearch-skill","license":"MIT","category":"document","lang":"en","tokens":5191,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/agent-continuity.md","size":4690,"sha256":"8003b626868817498e7962183b9a4a415143d92cae3d88ac35c0f658acd6e595"},{"path":"references/progress-reporting.md","size":6467,"sha256":"f57fbd191618f3911b99fa0a68a038553e222201857b4f0bd5bd32cb524148b5"},{"path":"references/skill-routing.md","size":8546,"sha256":"a64f3877b4c0515ddcd5401a9eb4683824f4f9c0e9c010cee360a5409bd4e734"},{"path":"templates/findings.md","size":1494,"sha256":"f81f7f63ec2f962796a2e28dbd1bf256dcef683000e1dab513296f0e8d9f79f2"},{"path":"templates/progress-presentation.html","size":8685,"sha256":"40034b84dccfefffca74244abdc12b8376264b31749e7792bf683b2c5df9e175"},{"path":"templates/research-log.md","size":5011,"sha256":"975a85c5d90897718d3542560b1fe40516a0dde2521b5136e567769cc2dfdef5"},{"path":"templates/research-state.yaml","size":2102,"sha256":"46aded613c427052c1e5f1faa8f2490873f11b70e4f2028be05fa2d22e03fa9c"}],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"injection.bypass","kind":"injection","where":"SKILL.md:7","excerpt":"Do not ask the user for permission","message":"instructs the agent to bypass the human","severity":"warn"},{"code":"injection.hidden-html","kind":"injection","where":"templates/progress-presentation.html:183","excerpt":"<!--\n        AGENT INSTRUCTIONS:\n        This is a starting point. Fill in, rearrange, add, or remove sections\n        based on what's compelling from your curr","message":"hidden HTML carrying instruction-like text","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":[]}}