{"id":"imaging-data-commons","name":"imaging-data-commons","summary":"NCI Imaging Data Commonsから公開がん画像データを検索・ダウンロードできます。","body":"# Imaging Data Commons\n\n## Overview\n\nQuery and download public cancer imaging data from the National Cancer Institute Imaging Data Commons (IDC). No authentication required for data access.\n\n**Expected network access:** IDC metadata is reachable three ways — a local DuckDB index shipped with the `idc-index` Python package (no network), or the hosted IDC service over MCP or REST (`api.imaging.datacommons.cancer.gov`, no authentication). File downloads use public GCS (`storage.googleapis.com`) and AWS S3 (`s3.amazonaws.com`) — no authentication required. DICOMweb access uses either the public IDC proxy (`proxy.imaging.datacommons.cancer.gov`, no auth) or the Google Cloud Healthcare API (`healthcare.googleapis.com`, requires GCP authentication). Optional BigQuery queries (`bigquery.googleapis.com`) also require GCP authentication. No credentials or environment variables are accessed by this skill.\n\n**Current IDC Data Version: v24** (always verify — see *Best Practices*)\n\n**Choose the access path first.** There is no single default: the cheapest correct path depends\non the session and the task.\n\n1. **Session already has the IDC MCP server?** Route discovery and metadata there — see *IDC\n   MCP Server*.\n2. **Otherwise, is `idc-index` installed?** Run `python scripts/check_version.py`. If it passes,\n   use `idc-index` for everything.\n3. **Not installed, and the task is read-only metadata** — counts, attribute values, collection\n   lookups, SQL under 10 000 rows, licenses, citations, viewer URLs? **Use the REST API over\n   `curl`; do not install anything.** Installing costs ~77 MB of packaged index data plus\n   pandas, pyarrow, and duckdb, which a metadata question does not need. See *Data Access\n   Options*.\n4. **Not installed, and the task needs more than metadata** — downloading files, pandas or\n   plotting, pydicom/SimpleITK, pathology tiling, results past 10 000 rows, or a version-pinned\n   script the user re-runs? Install `idc-index`: `check_version.py` exits non-zero and prints\n   the exact install command for the running interpreter. Prefer a virtual environment, then\n   restart Python.\n\n`idc-index` ([GitHub](https://github.com/imagingdatacommons/idc-index)) is still the most\ncapable path and the only one that moves image bytes; the rule is just not to pay for it before\nthe task calls for it. `check_version.py` never installs anything itself — it also flags a newer\n`idc-index` or skill release when one exists.\n\n**Setup for the `idc-index` path:**\n\n```python\nfrom idc_index import IDCClient\nclient = IDCClient()\n\n# Verify IDC data version (should be \"v24\")\nprint(f\"IDC data version: {client.get_idc_version()}\")\n```\n\n**Core workflow:** query metadata with `client.sql_query()` → download with\n`client.download_from_selection()` → visualize with `client.get_viewer_URL()`. Python examples\nbelow assume this `client`; *Data Access Options* has the REST equivalents. For current data\nscale, run the summary query in `references/sql_patterns.md` or `GET /v3/stats`.\n\n## IDC MCP Server\n\nIDC operates a hosted MCP server at `https://api.imaging.datacommons.cancer.gov/mcp`\n(streamable HTTP, no authentication). Where it is available it complements — it does not\nreplace — the `idc-index` workflow below.\n\n**Identify it** by the MCP resource `idc://guide`, or by three or more of the tool names\n`build_cohort`, `get_cohort_urls`, `list_analysis_results`, and `get_idc_version`. Generic\nnames such as `run_sql` are not evidence on their own. If identification is ambiguous, use\n`idc-index`.\n\n**If this session has the server**, treat it as authoritative for discovery and metadata —\nIDC version, counts, attribute values, cohort building, metadata SQL — and follow the\nserver's own instructions rather than re-deriving them from this file. Its data version is\nwhatever the server reports: call `get_idc_version` instead of relying on the version pinned\nin this file.\n\nReturn here for what the server does not do: downloading files, local pandas/notebook\nanalysis, DICOMweb, BigQuery, digital pathology tiling, and reproducible scripts. Hand off by\npassing SeriesInstanceUIDs from the server to `client.download_from_selection(...)`, and run\n`scripts/check_version.py` at that point.\n\n**If it is not available**, the identical service is reachable with no configuration as a REST\nAPI at `https://api.imaging.datacommons.cancer.gov/v3` — use it for read-only metadata rather\nthan installing `idc-index`, per the routing gate in *Overview*. Suggest connecting the MCP\nserver at most once, only for repeated interactive discovery, and never change the user's\nconfiguration yourself.\n\nSee `references/mcp_guide.md` for the tool inventory, handoff patterns, and per-host notes.\n\n## When to Use This Skill\n\n- Finding publicly available radiology (CT, MR, PET) or pathology (slide microscopy) images\n- Selecting image subsets by cancer type, modality, anatomical site, or other metadata\n- Downloading DICOM data from IDC\n- Checking data licenses before use in research or commercial applications\n- Visualizing medical images in a browser without local DICOM viewer software\n\n## Quick Navigation\n\nInline below: the MCP/REST routing rules, the IDC data model, the index tables and how they\njoin, the core API patterns (query, download, visualize, license, cite), best practices, and\ntroubleshooting.\n\n**Reference Guides (load on demand):**\n\n| Guide | When to Load |\n|-------|--------------|\n| `index_tables_guide.md` | Complex JOINs, schema discovery, DataFrame access |\n| `use_cases.md` | End-to-end workflows: training datasets, batch downloads, DICOM reading with pydicom/SimpleITK, pipeline integration |\n| `sql_patterns.md` | Quick SQL patterns for filter discovery, annotations, size estimation |\n| `clinical_data_guide.md` | Clinical/tabular data, imaging+clinical joins, value mapping |\n| `licensing_and_citation.md` | Commercial-use questions, mixed-license cohorts, citation formats |\n| `cloud_storage_guide.md` | Direct S3/GCS access, versioning, UUID mapping |\n| `dicomweb_guide.md` | DICOMweb endpoints, PACS integration |\n| `digital_pathology_guide.md` | Slide microscopy (SM), annotations (ANN), pathology workflows |\n| `bigquery_guide.md` | Full DICOM metadata, private elements (requires GCP) |\n| `cli_guide.md` | Command-line tools (`idc download`, manifest files) |\n| `parquet_access_guide.md` | Direct Parquet queries via GCS (no idc-index install needed) |\n| `mcp_guide.md` | Hosted IDC MCP server: tool inventory, identification, handoff to `idc-index` |\n| `rest_api_guide.md` | Hosted IDC REST API: endpoints, filter syntax, SQL over HTTP, manifests |\n\n## IDC Data Model\n\nIDC adds two grouping levels above the standard DICOM hierarchy (Patient → Study → Series → Instance):\n\n- **collection_id**: Groups patients by disease, modality, or research focus (e.g., `tcga_luad`, `nlst`). A patient belongs to exactly one collection.\n- **analysis_result_id**: Identifies derived objects (segmentations, annotations, radiomics features) across one or more original collections. Use it to find AI-generated or expert annotations, while `collection_id` finds original imaging data (which may itself include deposited annotations).\n\n**Key identifiers for queries:**\n| Identifier | Scope | Use for |\n|------------|-------|---------|\n| `collection_id` | Dataset grouping | Filtering by project/study |\n| `PatientID` | Patient | Grouping images by patient |\n| `StudyInstanceUID` | DICOM study | Grouping of related series, visualization |\n| `SeriesInstanceUID` | DICOM series | Grouping of related series, visualization |\n\n## Index Tables\n\nThe `idc-index` package provides multiple metadata index tables, accessible via SQL or as pandas DataFrames. The REST API exposes the same tables through `GET /tables` and `POST /sql`.\n\n**Important:** `client.indices_overview` is the authoritative source for current table descriptions, available columns, and their types — query it when writing SQL or exploring data structure. It also answers \"which table contains column X\"; see `references/index_tables_guide.md` for that search pattern and full schema discovery.\n\n### Available Tables\n\nAlways call `client.fetch_index(\"table_name\")` before querying any index table — it is safe and idempotent for all tables, including those loaded automatically at startup.\n\n| Family | Tables | Granularity |\n|--------|--------|-------------|\n| Core | `index` (primary metadata for all current data), `collections_index`, `analysis_results_index` | series / collection / analysis result |\n| Modality acquisition parameters | `ct_index`, `mr_index`, `pt_index`, `contrast_index` | 1 row = 1 series of that modality |\n| Derived objects | `seg_index`, `rtstruct_index`, `ann_index`, `ann_group_index` | 1 row = 1 series (or annotation group) |\n| Microscopy | `sm_index`, `sm_instance_index` | 1 row = 1 SM series / instance |\n| Geometry, clinical, history | `volume_geometry_index`, `clinical_index`, `version_metadata_index`, `prior_versions_index` | see guide |\n\n`references/index_tables_guide.md` has the full inventory with each table's columns and\ncontents — load it when you need to know what a specialized table actually holds.\n\n**`prior_versions_index` is for reproducibility only.** It contains series permanently *removed*\nfrom IDC, with zero overlap with `index`. Use it only to reproduce work against a prior IDC\nversion. Do NOT use it for version history or \"what's new\" questions — those use\n`series_init_idc_version` / `series_revised_idc_version` in the main `index` table, which are\nnot equivalent to this table's `min_idc_version` / `max_idc_version`.\n\n### Joining Tables\n\n**`SeriesInstanceUID` is the universal join key** for all series-level specialized tables: `sm_index`, `sm_instance_index`, `seg_index`, `ann_index`, `ann_group_index`, `contrast_index`, `volume_geometry_index`, `rtstruct_index`, `ct_index`, `mr_index`, `pt_index`. Always join these to `index` on `SeriesInstanceUID`. The exceptions below use different column names.\n\n| Join Column | Tables | Use Case |\n|-------------|--------|----------|\n| `collection_id` | index, prior_versions_index, collections_index, clinical_index | Link series to collection metadata or clinical data |\n| `analysis_result_id` | index, analysis_results_index | Link series to analysis result metadata (annotations, segmentations) |\n| `source_DOI` | index, analysis_results_index | Link by publication DOI |\n| `segmented_SeriesInstanceUID` | seg_index → index | Link segmentation to its source image series (`seg_index.segmented_SeriesInstanceUID = index.SeriesInstanceUID`) |\n| `referenced_SeriesInstanceUID` | ann_index → index, rtstruct_index → index | Link annotation or RTSTRUCT to its source image series |\n\n**Note:** `subjects`, `updated`, and `description` appear in multiple tables but have different meanings (counts vs identifiers, different update contexts). Joining `prior_versions_index` to `index` on `SeriesInstanceUID` always returns zero rows — see the warning above.\n\nFor detailed join examples, schema discovery patterns, key columns reference, and DataFrame access, see `references/index_tables_guide.md`.\n\n### Clinical Data Access\n\nClinical (non-imaging) attributes — staging, demographics, therapy — live in per-collection\ntables. `client.fetch_index(\"clinical_index\")` loads the dictionary mapping columns to\ncollections; `client.get_clinical_table(name)` returns one table as a DataFrame.\n\nSee `references/clinical_data_guide.md` for the discovery workflow, coded-value mapping, and\njoining clinical data with imaging.\n\n## Data Access Options\n\n| Method | Auth | Best For | Reference |\n|--------|------|----------|-----------|\n| `idc-index` | No | Downloads, pandas analysis, unbounded queries — the most capable path | This document |\n| IDC MCP server | No | Discovery, cohort building, metadata when the session already has it | `mcp_guide.md` |\n| IDC REST API | No | Metadata with no install, from any language or shell — the default when `idc-index` is absent | `rest_api_guide.md` |\n| Direct Parquet (GCS) | No | Version-pinned queries, or results past the REST row cap | `parquet_access_guide.md` |\n| Cloud storage (S3/GCS) | No | Direct file access, bulk transfer, custom pipelines | `cloud_storage_guide.md` |\n| DICOMweb via IDC proxy | No | Tool and PACS integration; daily quota, so testing and moderate use | `dicomweb_guide.md` |\n| DICOMweb via Google Healthcare | Yes (GCP) | The same DICOMweb API at production volume, without the proxy quota | `dicomweb_guide.md` |\n| SlicerIDCBrowser | No | 3D visualization and analysis in 3D Slicer | https://github.com/ImagingDataCommons/SlicerIDCBrowser |\n| BigQuery | Yes (GCP) | Full DICOM metadata, private elements, SR measurements — last resort | `bigquery_guide.md` |\n\n**The IDC Portal (https://portal.imaging.datacommons.cancer.gov/) is interactive only** —\nbrowser-based exploration, manual cohort selection, and download. Unlike every option above it\nhas no programmatic interface, so point a user there to browse or click through data\nthemselves; never use it as a step in a script or workflow.\n\n**REST API — the no-install metadata path**\n\n`https://api.imaging.datacommons.cancer.gov/v3`, no authentication: discovery, cohort counts and\nmanifests, read-only SQL, clinical tables, viewer URLs, licenses, citations. It is the same\nservice as the MCP server over plain HTTP, so it needs no configuration. It never moves image\nbytes — switch to `idc-index` to download, to get a DataFrame, or for results past 10 000 rows.\n\n```bash\nB=https://api.imaging.datacommons.cancer.gov/v3\ncurl -s $B/version   # idc_version, idc_index_data_version, api_version\ncurl -s $B/stats     # collections, patients, studies, series, instances, size_TB\ncurl -s \"$B/attributes/Modality/values?limit=5\"   # real filter values, with counts\ncurl -s $B/sql -H 'content-type: application/json' \\\n  -d '{\"sql\":\"SELECT collection_id, COUNT(*) n FROM index GROUP BY 1 ORDER BY n DESC LIMIT 3\"}'\ncurl -s $B/cohort/counts -H 'content-type: application/json' \\\n  -d '{\"filters\":{\"terms\":{\"collection_id\":[\"rider_pilot\"]}}}'\n```\n\n**The filter object always goes under `filters`** — on `cohort/counts`, `cohort/manifest`,\n`cohort/manifest.txt`, `licenses`, and `citations` alike. A bare filter or an unrecognized key is\na 422 naming the fix; an unfiltered series-enumerating request is a 400, not the whole archive.\nEvery filtered response echoes `filters_applied` and `warnings` — read them, because they name\nany predicate the server dropped. A zero count with empty `warnings` therefore means the filter\nmatched nothing, not that a value was miscased; miscasing produces a warning that says so.\n\n`POST /sql` takes one read-only `SELECT`/`WITH` over the tables `idc-index` exposes plus\n`clinical.<table>`; `max_rows` defaults to 5 000, caps at 10 000, and `truncated` flags clipping.\n`GET /attributes` lists the 19 filterable attributes — clinical values, segmented anatomy, and\nacquisition parameters are not among them and need SQL. There is no rate limit or quota. **Use\nv3 only:** V1 and V2 are superseded and scheduled for shutdown, so port any `/v1/`- or\n`Modality_btw`-style example a user brings rather than extending it.\n\nBoth sides build on `idc-index-data`, so compare the API's `idc_index_data_version` against local\n`idc_index_data.__version__` before mixing them: the **major is the IDC data release** (`24.x.y`\nserves `v24`), so differing minor/patch means the series are identical. If the API is a whole\nrelease ahead, `idc-index` **cannot download the extra series** — it silently skips what its own\nindex does not list — so either upgrade it (run `scripts/check_version.py` for the right command)\nor transfer directly from the bucket with `s5cmd --no-sign-request`.\n\nSee `references/rest_api_guide.md` for the endpoint reference, filter grounding, limits, and the\nmanifest-based download flow.\n\n**Cloud storage organization**\n\nAll DICOM files live in public buckets mirrored between AWS S3 and GCS, organized by CRDC UUIDs\n(not DICOM UIDs) to support versioning, as `<crdc_series_uuid>/<crdc_instance_uuid>.dcm`. Access\nis free (no egress fees) via AWS CLI, gsutil, or s5cmd with anonymous access; use the\n`series_aws_url` column for S3 URLs. Note that `idc-open-data-cr` / `idc-open-cr` (~4% of data)\nis commercial-use restricted (CC BY-NC). See `references/cloud_storage_guide.md` for the full\nbucket list and UUID mapping.\n\n**DICOMweb access**\n\nIDC data is available via DICOMweb (Google Cloud Healthcare API) for PACS integration and\nDICOMweb-compatible tools: a public proxy (no auth, daily quota) for testing and moderate\nqueries, or Google Healthcare (GCP auth) for production volumes. See\n`references/dicomweb_guide.md`.\n\n**Direct Parquet access**\n\nThe idc-index metadata tables are also published as Parquet on a public GCS bucket\n(`idc-index-data-artifacts`), queryable with DuckDB or pandas. This needs DuckDB installed\nand cannot reach the per-collection clinical tables, so prefer REST `/sql` for ad-hoc metadata;\nchoose Parquet to pin a data version or for results past the REST row cap. See\n`references/parquet_access_guide.md`.\n\n## Core Capabilities\n\nThe patterns below are the ones that go wrong when recalled from memory rather than checked.\nWorked examples for each area live in the reference guides named inline.\n\n### 1. Discovery — enumerate values before filtering on them\n\nFiltering on a guessed `Modality` or `BodyPartExamined` string is the most common cause of an\nempty result set. Enumerate first:\n\n```python\nmodalities = client.sql_query(\"\"\"\n    SELECT DISTINCT Modality, COUNT(*) as series_count\n    FROM index\n    GROUP BY Modality\n    ORDER BY series_count DESC\n\"\"\")\nprint(modalities)\n```\n\nThe same pattern works for any filter column, optionally narrowed by another —\n`BodyPartExamined` within a `Modality`, `Manufacturer`, `collection_id`. On the REST path this\ngrounding is a single call — `GET /attributes/{attr}/values` returns values with counts — and the\ncohort endpoints report a miscased value in `warnings` rather than as an empty result.\n\nTwo indices carry curated collection-level metadata the primary `index` does not, both\nrequiring `client.fetch_index(...)` first: `collections_index` (cancer types, tumor locations,\nspecies, subject counts) and `analysis_results_index` (derived datasets — AI segmentations,\nexpert annotations, radiomics — with their source collections and modalities).\n\n**Cancer type lives in `collections_index.cancer_types`, not in `index`** — filtering by\ncancer type requires a join:\n\n```python\nclient.fetch_index(\"collections_index\")\nresults = client.sql_query(\"\"\"\n    SELECT i.collection_id, i.PatientID, i.SeriesInstanceUID, i.Modality\n    FROM index i\n    JOIN collections_index c ON i.collection_id = c.collection_id\n    WHERE c.cancer_types LIKE '%Breast%'\n      AND i.Modality = 'MR'\n    LIMIT 20\n\"\"\")\n```\n\n`client.sql_query()` returns a pandas DataFrame. Confirm column names with\n`client.get_index_schema('index')` or `client.indices_overview` before writing a query rather\nthan assuming them.\n\nSee `references/sql_patterns.md` for filter-value discovery, annotation and segmentation\nqueries, size estimation, clinical linking, and version tracking (\"what's new in vX\" — use\n`series_init_idc_version` / `series_revised_idc_version` in `index`, never\n`prior_versions_index`).\n\n### 2. Downloading DICOM files\n\n**The two download methods take their first two arguments in opposite order.** This is the\nmost common source of broken IDC code — check it rather than recalling it:\n\n| Method | First arg | Second arg | Use when |\n|--------|-----------|------------|----------|\n| `download_from_selection` | `downloadDir` (required) | filter kwargs (optional) | Filtering by collection, patient, study, or series |\n| `download_dicom_series` | `seriesInstanceUID` (required) | `downloadDir` (required) | Downloading specific series by UID only |\n\n**`download_from_selection` takes filter keyword arguments, NOT a DataFrame.** The name\n\"from_selection\" refers to filtering the IDC index by criteria — not to accepting a pandas\nDataFrame. To download query results, extract the UIDs into a list first:\n\n```python\n# Step 1: Query for series UIDs\nseries_df = client.sql_query(\"\"\"\n    SELECT SeriesInstanceUID\n    FROM index\n    WHERE Modality = 'CT'\n      AND BodyPartExamined = 'CHEST'\n      AND collection_id = 'nlst'\n    LIMIT 5\n\"\"\")\n\n# Step 2: Extract UIDs as a list from the DataFrame\nuids = list(series_df['SeriesInstanceUID'].values)\n\n# Step 3: Pass the list to download_from_selection (NOT the DataFrame itself)\nclient.download_from_selection(\n    downloadDir=\"./data/lung_ct\",\n    seriesInstanceUID=uids       # list of strings, not a DataFrame\n)\n\n# Alternative: download_dicom_series has seriesInstanceUID as FIRST arg (different order!)\nclient.download_dicom_series(\n    seriesInstanceUID=uids,      # FIRST arg here\n    downloadDir=\"./data/lung_ct\"\n)\n\n# Whole collection: downloadDir is still the FIRST positional argument\nclient.download_from_selection(downloadDir=\"./data/rider\", collection_id=\"rider_pilot\")\n```\n\nBoth methods default to AWS; pass `source_bucket_location=\"gcs\"` to pull from Google Storage.\n\n**Downloaded files are named `<crdc_instance_uuid>.dcm`, not by SOPInstanceUID.** The DICOM\nUIDs are preserved inside the file metadata, not in the filename. Use the `crdc_instance_uuid`\ncolumn to map files back to the series they came from.\n\n`idc download <collection|series-uid|manifest> --download-dir ./data` does the same from a\nshell. See `references/cli_guide.md` for the `dirTemplate` hierarchy options (Python default:\n`%collection_id/%PatientID/%StudyInstanceUID/%Modality_%SeriesInstanceUID`; `dirTemplate=\"\"`\nflattens), manifest downloads with resume, and dry-run size estimation.\n\n### 3. Visualizing IDC images\n\n```python\nviewer_url = client.get_viewer_URL(seriesInstanceUID=uid)        # one series\nviewer_url = client.get_viewer_URL(studyInstanceUID=study_uid)   # all series in a study\n```\n\nReturns a browser URL — nothing is downloaded. The method selects OHIF v3 for radiology or\nSLIM for slide microscopy automatically. Viewing by study is useful when a single DICOM Study\nholds several Series (T1, T2, and DWI from one MRI session).\n\n### 4. Licenses and citations — obligations, not optional steps\n\nIDC data carries license terms and attribution requirements that follow it into any downstream\npublication or product, and neither is inferable from the pixel data. **Check the license\nbefore use, and generate citations for whatever you download.**\n\n```python\n# License breakdown for a selection\nlicenses = client.sql_query(\"\"\"\n    SELECT DISTINCT collection_id, license_short_name,\n           COUNT(DISTINCT SeriesInstanceUID) as series_count\n    FROM index GROUP BY collection_id, license_short_name\n\"\"\")\n\n# Citations for the same selection you downloaded (APA by default)\nfor citation in client.citations_from_selection(collection_id=\"rider_pilot\"):\n    print(citation)\n```\n\nAbout 97% of IDC data is CC BY (commercial use allowed with attribution) and about 3% is\nCC BY-NC (non-commercial only). **Licenses attach to series, not collections** — 39 of 176\ncollections carry more than one — so check the selection you actually intend to use, and note\nthat the most restrictive term governs a mixed cohort.\n\nBoth tasks are available from all three access paths, so stay on whichever one the session is\nalready using: `idc-index` as above, `POST /v3/licenses` and `POST /v3/citations` over REST,\nor the `get_licenses` and `get_citations` MCP tools. See\n`references/licensing_and_citation.md` for the full license inventory, all three routes, the\ncitation formats (APA, BibTeX, CSL JSON, RDF Turtle), and what to include when publishing.\n\n### 5. Reaching past the index\n\nPick the access path with the routing gate in *Overview*; *Data Access Options* above is the\nfull routing table.\n\nBefore reaching for BigQuery (which needs a billing-enabled GCP account), check whether a\nspecialized index table already has the column you want: search `client.indices_overview`,\nthen `client.fetch_index(...)` and query locally for free. BigQuery is required only for\nprivate DICOM elements, per-segment anatomy (`segmentations`), and pre-extracted SR\nmeasurements (`quantitative_measurements`, `qualitative_measurements`) — these have no\nidc-index equivalent.\n\n## Best Practices\n\n- **Check schema before writing queries** — Use `client.get_index_schema('index')` (reads cached metadata, no SQL executed) or `client.indices_overview` to see all available columns and their descriptions. The version-tracking columns `series_init_idc_version` and `series_revised_idc_version` in the main `index` table directly answer \"what's new / when was this added\" questions without touching `prior_versions_index`.\n- **Never use web search for IDC data content questions** - Always query the IDC index directly, via `client.sql_query()` locally or `POST /v3/sql` over HTTP. Web sources (release notes, blog posts, documentation pages) are frequently out of date and will produce incorrect answers. The index is the authoritative source; use it even when web search is available.\n- **Verify the IDC data version at the start of a session** - `client.get_idc_version()`, `GET /v3/version`, or the MCP `get_idc_version` tool, depending on the path in use (currently v24). For a stale local index, run `scripts/check_version.py` and use the upgrade command it prints\n- **Check licenses and generate citations** - Query `license_short_name` and respect CC BY vs CC BY-NC terms; use `citations_from_selection()` to produce citations from `source_DOI` for publications\n- **Explore small, then commit** - Use `LIMIT` (or a low `max_rows`) while exploring, and check collection size before downloading — some collections are terabytes. See `references/cli_guide.md`\n- **Keep downloads reproducible** - Organize with `dirTemplate` (e.g. `%collection_id/%PatientID/%Modality`) and save the Series UIDs or manifest behind any dataset you build\n\n## Troubleshooting\n\n**Issue: `ModuleNotFoundError: No module named 'idc_index'`**\n- **Cause:** idc-index package not installed\n- **Solution:** If the task is read-only metadata, do not install it — use the REST API instead (*Data Access Options*). Otherwise run `scripts/check_version.py` and use the install command it prints, which targets the running interpreter and pins the vetted version. For data analysis also add pandas, numpy, and pydicom (tested with pandas>=1.5, numpy>=1.23, pydicom>=2.3)\n\n**Issue: Download fails with connection timeout**\n- **Cause:** Network instability or large download size\n- **Solution:** Download in smaller batches (10-20 series); see `references/cli_guide.md` for\n  `--use-s5cmd-sync` resume and retry guidance\n\n**Issue: `BigQuery quota exceeded` or billing errors**\n- **Cause:** BigQuery requires billing-enabled GCP project\n- **Solution:** Use idc-index mini-index for simple queries (no billing required), or see `references/bigquery_guide.md` for cost optimization tips\n\n**Issue: Series UID not found or no data returned**\n- **Cause:** Typo in UID, data not in the current IDC version, or wrong field name\n- **Solution:** Test with `LIMIT 5` first, check field names against `client.indices_overview`,\n  and confirm the series is in the current version (some old data is deprecated)\n\n**Issue: Column not found in `index` table (e.g., `SliceThickness`, `PixelSpacing`, `KVP`, `EchoTime`, `InjectedDose`)**\n- **Cause:** The `index` table contains series-level metadata only; modality-specific acquisition and reconstruction parameters live in dedicated tables (`ct_index`, `mr_index`, `pt_index`)\n- **Solution:** Search `client.indices_overview` for the column to find its table — the loop is under *Finding which table contains a column* in `references/index_tables_guide.md` — then fetch and join on `SeriesInstanceUID`:\n  ```python\n  client.fetch_index(\"ct_index\")\n  result = client.sql_query(\"\"\"\n      SELECT i.SeriesInstanceUID, i.Modality, c.SliceThickness, c.KVP, c.PixelSpacing_row_mm\n      FROM index i\n      JOIN ct_index c USING (SeriesInstanceUID)\n      WHERE i.collection_id = 'your_collection'\n  \"\"\")\n  ```\n\n**Issue: Downloaded DICOM files won't open**\n- **Cause:** Corrupted download, or an object type the viewer does not handle — SEG, RTSTRUCT,\n  SR, and slide microscopy all need specialized tools\n- **Solution:** Check `Modality` and `SOPClassUID` first, validate with\n  `pydicom.dcmread(file, force=True)`, try another viewer (3D Slicer, QuPath for pathology),\n  then re-download\n\n## Resources\n\nReference guides and their decision triggers are listed in *Quick Navigation* above.\n\n- **IDC Portal**: https://portal.imaging.datacommons.cancer.gov/explore/\n- **Documentation**: https://learn.canceridc.dev/ — **Tutorials**: https://github.com/ImagingDataCommons/IDC-Tutorials\n- **User Forum**: https://discourse.canceridc.dev/ — **idc-index**: https://github.com/ImagingDataCommons/idc-index\n- **[indices_reference](https://idc-index.readthedocs.io/en/latest/indices_reference.html)** — external index-table docs (may be ahead of the installed version)\n- **Citation**: Fedorov, A., et al. \"National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence.\" RadioGraphics 43.12 (2023). https://doi.org/10.1148/rg.230180\n- **Skill updates**: [releases page](https://github.com/ImagingDataCommons/imaging-data-commons-skill/releases); watch the repository (Watch → Custom → Releases)","author":"@K-Dense-AI","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/imaging-data-commons","license":"MIT","category":"coding","lang":"en","tokens":6985,"stars":0,"calls30d":1,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/bigquery_guide.md","size":33640,"sha256":"97294c1736fc8d70badd005ff49bfa2b44f24cf21ba5897361424382bcfed2eb"},{"path":"references/cli_guide.md","size":8425,"sha256":"523cc1385a09815d60dd4ba3ecc4cb4684f080b5ba54d43f1317a95c7e68f379"},{"path":"references/clinical_data_guide.md","size":12143,"sha256":"b8787828142cae8227e2d2264d7d1bebe8c43b3b3ef136d50327150f1962548a"},{"path":"references/cloud_storage_guide.md","size":14065,"sha256":"226b8d00d85619ef14d2f39b8bdf3fed2e4f0d6c681823a459af6968b618c973"},{"path":"references/dicomweb_guide.md","size":14998,"sha256":"b58f39766c4384dd2b18224de530548654b8b03a316b5bd920fd265aa99f8360"},{"path":"references/digital_pathology_guide.md","size":15357,"sha256":"0e2afc2908503c56bd9e348fca601d899c7544068204dc105054ec77f7a368e5"},{"path":"references/index_tables_guide.md","size":12389,"sha256":"502c4d5b8b09f6147ac203374cf1d3f39d218c01068a503089458f9c87acfd8e"},{"path":"references/licensing_and_citation.md","size":9658,"sha256":"8a2fe37fb0b368d9048a13830508099c89072be67fd62668499a2b283f2b7310"},{"path":"references/mcp_guide.md","size":8766,"sha256":"139b2463af3dd8e463f3577c397bd2e4b2b1635ebc90a8ae44e5ab6e6a68740d"},{"path":"references/parquet_access_guide.md","size":8635,"sha256":"f2730d475a23577d49364f30930783ed9e54f086eaff4dd09a938e28977f22c1"},{"path":"references/rest_api_guide.md","size":32028,"sha256":"d1e92a6653c1ed713d31262e78a1e2044f03be5c3a3deda04428ec919e6be1b8"},{"path":"references/sql_patterns.md","size":16734,"sha256":"2b1132e4ad5fccdff45af66ae7ca9a17fea8ed214c81422f7cd828327dd6de3a"},{"path":"references/use_cases.md","size":7970,"sha256":"525a8e3f330f219f3a425ce3eb74c9105715e3d26680f52d6c52d10387e34cad"},{"path":"scripts/check_version.py","size":5381,"sha256":"3297b5b2ff900b5002b9a53a1ad86fe395e5cc6cbe4de970862f765873ed85f9"}],"requires":{"mcp":["idc","claude_ai_IDC_MCP_prod"],"tools":[]},"safety":{"flags":[{"code":"net.endpoints","kind":"exfiltration","excerpt":"api.github.com, api.imaging.datacommons.cancer.gov, boto3.amazonaws.com, cloud.google.com, datastudio.google.com, dicom.nema.org, dicomweb-client.readthedocs.io, discourse.canceridc.dev","message":"bundled scripts reach 27 external host(s)","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":true,"networkEndpoints":["api.github.com","api.imaging.datacommons.cancer.gov","boto3.amazonaws.com","cloud.google.com","datastudio.google.com","dicom.nema.org","dicomweb-client.readthedocs.io","discourse.canceridc.dev","docs.aws.amazon.com","docs.cloud.google.com","docs.gdc.cancer.gov","doi.org","gdc.cancer.gov","healthcare.googleapis.com","idc-index.readthedocs.io","learn.canceridc.dev","portal.imaging.datacommons.cancer.gov","proxy.imaging.datacommons.cancer.gov","qupath.github.io","registry.opendata.aws","storage.googleapis.com","viewer.imaging.datacommons.cancer.gov","web.archive.org","www.dicomstandard.org","www.gehealthcare.com","www.kaggle.com","www.siemens-healthineers.com"]}}