{"id":"modal","name":"modal-serverless-gpu","summary":"MLワークロードを実行するためのサーバーレスGPUクラウドプラットフォーム。インフラ管理なしでオンデマンドGPUアクセスが必要な場合、MLモデルをAPIとして展開する場合、または自動スケーリング付きのバッチジョブを実行する場合に使います。","body":"# Modal Serverless GPU\n\nComprehensive guide to running ML workloads on Modal's serverless GPU cloud platform.\n\n## When to use Modal\n\n**Use Modal when:**\n- Running GPU-intensive ML workloads without managing infrastructure\n- Deploying ML models as auto-scaling APIs\n- Running batch processing jobs (training, inference, data processing)\n- Need pay-per-second GPU pricing without idle costs\n- Prototyping ML applications quickly\n- Running scheduled jobs (cron-like workloads)\n\n**Key features:**\n- **Serverless GPUs**: T4, L4, A10G, L40S, A100, H100, H200, B200 on-demand\n- **Python-native**: Define infrastructure in Python code, no YAML\n- **Auto-scaling**: Scale to zero, scale to 100+ GPUs instantly\n- **Sub-second cold starts**: Rust-based infrastructure for fast container launches\n- **Container caching**: Image layers cached for rapid iteration\n- **Web endpoints**: Deploy functions as REST APIs with zero-downtime updates\n\n**Use alternatives instead:**\n- **RunPod**: For longer-running pods with persistent state\n- **Lambda Labs**: For reserved GPU instances\n- **SkyPilot**: For multi-cloud orchestration and cost optimization\n- **Kubernetes**: For complex multi-service architectures\n\n## Quick start\n\n### Installation\n\n```bash\npip install modal\nmodal setup  # Opens browser for authentication\n```\n\n### Hello World with GPU\n\n```python\nimport modal\n\napp = modal.App(\"hello-gpu\")\n\n@app.function(gpu=\"T4\")\ndef gpu_info():\n    import subprocess\n    return subprocess.run([\"nvidia-smi\"], capture_output=True, text=True).stdout\n\n@app.local_entrypoint()\ndef main():\n    print(gpu_info.remote())\n```\n\nRun: `modal run hello_gpu.py`\n\n### Basic inference endpoint\n\n```python\nimport modal\n\napp = modal.App(\"text-generation\")\nimage = modal.Image.debian_slim().pip_install(\"transformers\", \"torch\", \"accelerate\")\n\n@app.cls(gpu=\"A10G\", image=image)\nclass TextGenerator:\n    @modal.enter()\n    def load_model(self):\n        from transformers import pipeline\n        self.pipe = pipeline(\"text-generation\", model=\"gpt2\", device=0)\n\n    @modal.method()\n    def generate(self, prompt: str) -> str:\n        return self.pipe(prompt, max_length=100)[0][\"generated_text\"]\n\n@app.local_entrypoint()\ndef main():\n    print(TextGenerator().generate.remote(\"Hello, world\"))\n```\n\n## Core concepts\n\n### Key components\n\n| Component | Purpose |\n|-----------|---------|\n| `App` | Container for functions and resources |\n| `Function` | Serverless function with compute specs |\n| `Cls` | Class-based functions with lifecycle hooks |\n| `Image` | Container image definition |\n| `Volume` | Persistent storage for models/data |\n| `Secret` | Secure credential storage |\n\n### Execution modes\n\n| Command | Description |\n|---------|-------------|\n| `modal run script.py` | Execute and exit |\n| `modal serve script.py` | Development with live reload |\n| `modal deploy script.py` | Persistent cloud deployment |\n\n## GPU configuration\n\n### Available GPUs\n\n| GPU | VRAM | Best For |\n|-----|------|----------|\n| `T4` | 16GB | Budget inference, small models |\n| `L4` | 24GB | Inference, Ada Lovelace arch |\n| `A10G` | 24GB | Training/inference, 3.3x faster than T4 |\n| `L40S` | 48GB | Recommended for inference (best cost/perf) |\n| `A100-40GB` | 40GB | Large model training |\n| `A100-80GB` | 80GB | Very large models |\n| `H100` | 80GB | Fastest, FP8 + Transformer Engine |\n| `H200` | 141GB | Auto-upgrade from H100, 4.8TB/s bandwidth |\n| `B200` | Latest | Blackwell architecture |\n\n### GPU specification patterns\n\n```python\n# Single GPU\n@app.function(gpu=\"A100\")\n\n# Specific memory variant\n@app.function(gpu=\"A100-80GB\")\n\n# Multiple GPUs (up to 8)\n@app.function(gpu=\"H100:4\")\n\n# GPU with fallbacks\n@app.function(gpu=[\"H100\", \"A100\", \"L40S\"])\n\n# Any available GPU\n@app.function(gpu=\"any\")\n```\n\n## Container images\n\n```python\n# Basic image with pip\nimage = modal.Image.debian_slim(python_version=\"3.11\").pip_install(\n    \"torch==2.1.0\", \"transformers==4.36.0\", \"accelerate\"\n)\n\n# From CUDA base\nimage = modal.Image.from_registry(\n    \"nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04\",\n    add_python=\"3.11\"\n).pip_install(\"torch\", \"transformers\")\n\n# With system packages\nimage = modal.Image.debian_slim().apt_install(\"git\", \"ffmpeg\").pip_install(\"whisper\")\n```\n\n## Persistent storage\n\n```python\nvolume = modal.Volume.from_name(\"model-cache\", create_if_missing=True)\n\n@app.function(gpu=\"A10G\", volumes={\"/models\": volume})\ndef load_model():\n    import os\n    model_path = \"/models/llama-7b\"\n    if not os.path.exists(model_path):\n        model = download_model()\n        model.save_pretrained(model_path)\n        volume.commit()  # Persist changes\n    return load_from_path(model_path)\n```\n\n## Web endpoints\n\n### FastAPI endpoint decorator\n\n```python\n@app.function()\n@modal.fastapi_endpoint(method=\"POST\")\ndef predict(text: str) -> dict:\n    return {\"result\": model.predict(text)}\n```\n\n### Full ASGI app\n\n```python\nfrom fastapi import FastAPI\nweb_app = FastAPI()\n\n@web_app.post(\"/predict\")\nasync def predict(text: str):\n    return {\"result\": await model.predict.remote.aio(text)}\n\n@app.function()\n@modal.asgi_app()\ndef fastapi_app():\n    return web_app\n```\n\n### Web endpoint types\n\n| Decorator | Use Case |\n|-----------|----------|\n| `@modal.fastapi_endpoint()` | Simple function → API |\n| `@modal.asgi_app()` | Full FastAPI/Starlette apps |\n| `@modal.wsgi_app()` | Django/Flask apps |\n| `@modal.web_server(port)` | Arbitrary HTTP servers |\n\n## Dynamic batching\n\n```python\n@app.function()\n@modal.batched(max_batch_size=32, wait_ms=100)\nasync def batch_predict(inputs: list[str]) -> list[dict]:\n    # Inputs automatically batched\n    return model.batch_predict(inputs)\n```\n\n## Secrets management\n\n```bash\n# Create secret\nmodal secret create huggingface HF_TOKEN=hf_xxx\n```\n\n```python\n@app.function(secrets=[modal.Secret.from_name(\"huggingface\")])\ndef download_model():\n    import os\n    token = os.environ[\"HF_TOKEN\"]\n```\n\n## Scheduling\n\n```python\n@app.function(schedule=modal.Cron(\"0 0 * * *\"))  # Daily midnight\ndef daily_job():\n    pass\n\n@app.function(schedule=modal.Period(hours=1))\ndef hourly_job():\n    pass\n```\n\n## Performance optimization\n\n### Cold start mitigation\n\n```python\n@app.function(\n    container_idle_timeout=300,  # Keep warm 5 min\n    allow_concurrent_inputs=10,  # Handle concurrent requests\n)\ndef inference():\n    pass\n```\n\n### Model loading best practices\n\n```python\n@app.cls(gpu=\"A100\")\nclass Model:\n    @modal.enter()  # Run once at container start\n    def load(self):\n        self.model = load_model()  # Load during warm-up\n\n    @modal.method()\n    def predict(self, x):\n        return self.model(x)\n```\n\n## Parallel processing\n\n```python\n@app.function()\ndef process_item(item):\n    return expensive_computation(item)\n\n@app.function()\ndef run_parallel():\n    items = list(range(1000))\n    # Fan out to parallel containers\n    results = list(process_item.map(items))\n    return results\n```\n\n## Common configuration\n\n```python\n@app.function(\n    gpu=\"A100\",\n    memory=32768,              # 32GB RAM\n    cpu=4,                     # 4 CPU cores\n    timeout=3600,              # 1 hour max\n    container_idle_timeout=120,# Keep warm 2 min\n    retries=3,                 # Retry on failure\n    concurrency_limit=10,      # Max concurrent containers\n)\ndef my_function():\n    pass\n```\n\n## Debugging\n\n```python\n# Test locally\nif __name__ == \"__main__\":\n    result = my_function.local()\n\n# View logs\n# modal app logs my-app\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Cold start latency | Increase `container_idle_timeout`, use `@modal.enter()` |\n| GPU OOM | Use larger GPU (`A100-80GB`), enable gradient checkpointing |\n| Image build fails | Pin dependency versions, check CUDA compatibility |\n| Timeout errors | Increase `timeout`, add checkpointing |\n\n## References\n\n- **[Advanced Usage](references/advanced-usage.md)** - Multi-GPU, distributed training, cost optimization\n- **[Troubleshooting](references/troubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **Documentation**: https://modal.com/docs\n- **Examples**: https://github.com/modal-labs/modal-examples\n- **Pricing**: https://modal.com/pricing\n- **Discord**: https://discord.gg/modal","author":"@Orchestra-Research","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/09-infrastructure/modal","license":"MIT","category":"coding","lang":"en","tokens":2055,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/advanced-usage.md","size":10903,"sha256":"1e65ec0c842b7fb0bbda4a091ea19069d749992c4f95cd8bd966863a26addf0d"},{"path":"references/troubleshooting.md","size":10516,"sha256":"df2d4f66c8301c9ffc067d9170271b2d31c0dcd56bba5d83ea07882475110ccb"}],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"code.eval","kind":"dangerous-code","where":"references/advanced-usage.md:478","excerpt":"exec(","message":"evaluates code at runtime","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":["discord.gg","download.pytorch.org","modal.com","status.modal.com","your-workspace--my-app-predict.modal.run"]}}