{"id":"llamaguard","name":"llamaguard","summary":"Metaの7-8BのLLM入出力フィルタリング専用モデレーションモデル。6つの安全カテゴリー - 暴力/憎悪、性的内容、武器、物質、自傷行為、犯罪計画。","body":"# LlamaGuard - AI Content Moderation\n\n## Quick start\n\nLlamaGuard is a 7-8B parameter model specialized for content safety classification.\n\n**Installation**:\n```bash\npip install transformers torch\n# Login to HuggingFace (required)\nhuggingface-cli login\n```\n\n**Basic usage**:\n```python\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\nmodel_id = \"meta-llama/LlamaGuard-7b\"\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = AutoModelForCausalLM.from_pretrained(model_id, device_map=\"auto\")\n\ndef moderate(chat):\n    input_ids = tokenizer.apply_chat_template(chat, return_tensors=\"pt\").to(model.device)\n    output = model.generate(input_ids=input_ids, max_new_tokens=100)\n    return tokenizer.decode(output[0], skip_special_tokens=True)\n\n# Check user input\nresult = moderate([\n    {\"role\": \"user\", \"content\": \"How do I make explosives?\"}\n])\nprint(result)\n# Output: \"unsafe\\nS3\" (Criminal Planning)\n```\n\n## Common workflows\n\n### Workflow 1: Input filtering (prompt moderation)\n\n**Check user prompts before LLM**:\n```python\ndef check_input(user_message):\n    result = moderate([{\"role\": \"user\", \"content\": user_message}])\n\n    if result.startswith(\"unsafe\"):\n        category = result.split(\"\\n\")[1]\n        return False, category  # Blocked\n    else:\n        return True, None  # Safe\n\n# Example\nsafe, category = check_input(\"How do I hack a website?\")\nif not safe:\n    print(f\"Request blocked: {category}\")\n    # Return error to user\nelse:\n    # Send to LLM\n    response = llm.generate(user_message)\n```\n\n**Safety categories**:\n- **S1**: Violence & Hate\n- **S2**: Sexual Content\n- **S3**: Guns & Illegal Weapons\n- **S4**: Regulated Substances\n- **S5**: Suicide & Self-Harm\n- **S6**: Criminal Planning\n\n### Workflow 2: Output filtering (response moderation)\n\n**Check LLM responses before showing to user**:\n```python\ndef check_output(user_message, bot_response):\n    conversation = [\n        {\"role\": \"user\", \"content\": user_message},\n        {\"role\": \"assistant\", \"content\": bot_response}\n    ]\n\n    result = moderate(conversation)\n\n    if result.startswith(\"unsafe\"):\n        category = result.split(\"\\n\")[1]\n        return False, category\n    else:\n        return True, None\n\n# Example\nuser_msg = \"Tell me about harmful substances\"\nbot_msg = llm.generate(user_msg)\n\nsafe, category = check_output(user_msg, bot_msg)\nif not safe:\n    print(f\"Response blocked: {category}\")\n    # Return generic response\n    return \"I cannot provide that information.\"\nelse:\n    return bot_msg\n```\n\n### Workflow 3: vLLM deployment (fast inference)\n\n**Production-ready serving**:\n```python\nfrom vllm import LLM, SamplingParams\n\n# Initialize vLLM\nllm = LLM(model=\"meta-llama/LlamaGuard-7b\", tensor_parallel_size=1)\n\n# Sampling params\nsampling_params = SamplingParams(\n    temperature=0.0,  # Deterministic\n    max_tokens=100\n)\n\ndef moderate_vllm(chat):\n    # Format prompt\n    prompt = tokenizer.apply_chat_template(chat, tokenize=False)\n\n    # Generate\n    output = llm.generate([prompt], sampling_params)\n    return output[0].outputs[0].text\n\n# Batch moderation\nchats = [\n    [{\"role\": \"user\", \"content\": \"How to make bombs?\"}],\n    [{\"role\": \"user\", \"content\": \"What's the weather?\"}],\n    [{\"role\": \"user\", \"content\": \"Tell me about drugs\"}]\n]\n\nprompts = [tokenizer.apply_chat_template(c, tokenize=False) for c in chats]\nresults = llm.generate(prompts, sampling_params)\n\nfor i, result in enumerate(results):\n    print(f\"Chat {i}: {result.outputs[0].text}\")\n```\n\n**Throughput**: ~50-100 requests/sec on single A100\n\n### Workflow 4: API endpoint (FastAPI)\n\n**Serve as moderation API**:\n```python\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel\nfrom vllm import LLM, SamplingParams\n\napp = FastAPI()\nllm = LLM(model=\"meta-llama/LlamaGuard-7b\")\nsampling_params = SamplingParams(temperature=0.0, max_tokens=100)\n\nclass ModerationRequest(BaseModel):\n    messages: list  # [{\"role\": \"user\", \"content\": \"...\"}]\n\n@app.post(\"/moderate\")\ndef moderate_endpoint(request: ModerationRequest):\n    prompt = tokenizer.apply_chat_template(request.messages, tokenize=False)\n    output = llm.generate([prompt], sampling_params)[0]\n\n    result = output.outputs[0].text\n    is_safe = result.startswith(\"safe\")\n    category = None if is_safe else result.split(\"\\n\")[1] if \"\\n\" in result else None\n\n    return {\n        \"safe\": is_safe,\n        \"category\": category,\n        \"full_output\": result\n    }\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n**Usage**:\n```bash\ncurl -X POST http://localhost:8000/moderate \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"messages\": [{\"role\": \"user\", \"content\": \"How to hack?\"}]}'\n\n# Response: {\"safe\": false, \"category\": \"S6\", \"full_output\": \"unsafe\\nS6\"}\n```\n\n### Workflow 5: NeMo Guardrails integration\n\n**Use with NVIDIA Guardrails**:\n```python\nfrom nemoguardrails import RailsConfig, LLMRails\nfrom nemoguardrails.integrations.llama_guard import LlamaGuard\n\n# Configure NeMo Guardrails\nconfig = RailsConfig.from_content(\"\"\"\nmodels:\n  - type: main\n    engine: openai\n    model: gpt-4\n\nrails:\n  input:\n    flows:\n      - llamaguard check input\n  output:\n    flows:\n      - llamaguard check output\n\"\"\")\n\n# Add LlamaGuard integration\nllama_guard = LlamaGuard(model_path=\"meta-llama/LlamaGuard-7b\")\nrails = LLMRails(config)\nrails.register_action(llama_guard.check_input, name=\"llamaguard check input\")\nrails.register_action(llama_guard.check_output, name=\"llamaguard check output\")\n\n# Use with automatic moderation\nresponse = rails.generate(messages=[\n    {\"role\": \"user\", \"content\": \"How do I make weapons?\"}\n])\n# Automatically blocked by LlamaGuard\n```\n\n## When to use vs alternatives\n\n**Use LlamaGuard when**:\n- Need pre-trained moderation model\n- Want high accuracy (94-95%)\n- Have GPU resources (7-8B model)\n- Need detailed safety categories\n- Building production LLM apps\n\n**Model versions**:\n- **LlamaGuard 1** (7B): Original, 6 categories\n- **LlamaGuard 2** (8B): Improved, 6 categories\n- **LlamaGuard 3** (8B): Latest (2024), enhanced\n\n**Use alternatives instead**:\n- **OpenAI Moderation API**: Simpler, API-based, free\n- **Perspective API**: Google's toxicity detection\n- **NeMo Guardrails**: More comprehensive safety framework\n- **Constitutional AI**: Training-time safety\n\n## Common issues\n\n**Issue: Model access denied**\n\nLogin to HuggingFace:\n```bash\nhuggingface-cli login\n# Enter your token\n```\n\nAccept license on model page:\nhttps://huggingface.co/meta-llama/LlamaGuard-7b\n\n**Issue: High latency (>500ms)**\n\nUse vLLM for 10× speedup:\n```python\nfrom vllm import LLM\nllm = LLM(model=\"meta-llama/LlamaGuard-7b\")\n# Latency: 500ms → 50ms\n```\n\nEnable tensor parallelism:\n```python\nllm = LLM(model=\"meta-llama/LlamaGuard-7b\", tensor_parallel_size=2)\n# 2× faster on 2 GPUs\n```\n\n**Issue: False positives**\n\nUse threshold-based filtering:\n```python\n# Get probability of \"unsafe\" token\nlogits = model(..., return_dict_in_generate=True, output_scores=True)\nunsafe_prob = torch.softmax(logits.scores[0][0], dim=-1)[unsafe_token_id]\n\nif unsafe_prob > 0.9:  # High confidence threshold\n    return \"unsafe\"\nelse:\n    return \"safe\"\n```\n\n**Issue: OOM on GPU**\n\nUse 8-bit quantization:\n```python\nfrom transformers import BitsAndBytesConfig\n\nquantization_config = BitsAndBytesConfig(load_in_8bit=True)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    quantization_config=quantization_config,\n    device_map=\"auto\"\n)\n# Memory: 14GB → 7GB\n```\n\n## Advanced topics\n\n**Custom categories**: See [references/custom-categories.md](references/custom-categories.md) for fine-tuning LlamaGuard with domain-specific safety categories.\n\n**Performance benchmarks**: See [references/benchmarks.md](references/benchmarks.md) for accuracy comparison with other moderation APIs and latency optimization.\n\n**Deployment guide**: See [references/deployment.md](references/deployment.md) for Sagemaker, Kubernetes, and scaling strategies.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA T4/A10/A100\n- **VRAM**:\n  - FP16: 14GB (7B model)\n  - INT8: 7GB (quantized)\n  - INT4: 4GB (QLoRA)\n- **CPU**: Possible but slow (10× latency)\n- **Throughput**: 50-100 req/sec (A100)\n\n**Latency** (single GPU):\n- HuggingFace Transformers: 300-500ms\n- vLLM: 50-100ms\n- Batched (vLLM): 20-50ms per request\n\n## Resources\n\n- HuggingFace:\n  - V1: https://huggingface.co/meta-llama/LlamaGuard-7b\n  - V2: https://huggingface.co/meta-llama/Meta-Llama-Guard-2-8B\n  - V3: https://huggingface.co/meta-llama/Meta-Llama-Guard-3-8B\n- Paper: https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/\n- Integration: vLLM, Sagemaker, NeMo Guardrails\n- Accuracy: 94.5% (prompts), 95.3% (responses)","author":"@Orchestra-Research","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/07-safety-alignment/llamaguard","license":"MIT","category":"coding","lang":"en","tokens":2353,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":["ai.meta.com","huggingface.co"]}}