Outlines

orchestra-research/ai-research-skills/16-prompt-engineering/outlines

作者 orchestra-research773a52944ba4MIT13K 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫3 個月前更新

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

僅含說明AI & Agents
AI 產生的概覽

指導使用 Outlines 函式庫，透過本機模型進行受結構約束的文字生成。

功能
這個技能說明如何使用 Outlines 函式庫，把語言模型的輸出限制成合法的 JSON、XML、正規表達式或程式碼結構。內容涵蓋 Pydantic 模型整合，choice、JSON、正規表達式和數值產生器，以及 Transformers、llama.cpp、vLLM 和 OpenAI 的後端設定。也列出資料擷取、分類、結構化表單和批次處理等常見模式，並提供最佳實務和與其他方案的比較。
適用情境
當你需要語言模型輸出保證合法的結構化結果時使用，例如符合結構描述的 JSON，或固定集合中的某個值。它適合使用本機模型、重視型別安全與文法約束生成以及推論速度的情境。
執行需求
需要 outlines Python 套件，以及選用的 transformers、llama-cpp-python 或 vllm 等後端，並使用 pydantic 定義模型。依後端不同，還需要本機模型權重或 API 金鑰。這個技能不含指令碼，只有說明文件和參考文件。

Outlines: Structured Text Generation

When to Use This Skill

Use Outlines when you need to:

  • Guarantee valid JSON/XML/code structure during generation
  • Use Pydantic models for type-safe outputs
  • Support local models (Transformers, llama.cpp, vLLM)
  • Maximize inference speed with zero-overhead structured generation
  • Generate against JSON schemas automatically
  • Control token sampling at the grammar level

GitHub Stars: 8,000+ | From: dottxt.ai (formerly .txt)

Installation

bash
# Base installationpip install outlines
# With specific backendspip install outlines transformers  # Hugging Face modelspip install outlines llama-cpp-python  # llama.cpppip install outlines vllm  # vLLM for high-throughput

Quick Start

Basic Example: Classification

python
import outlinesfrom typing import Literal
# Load modelmodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate with type constraintprompt = "Sentiment of 'This product is amazing!': "generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])sentiment = generator(prompt)
print(sentiment)  # "positive" (guaranteed one of these)

With Pydantic Models

python
from pydantic import BaseModelimport outlines
class User(BaseModel):    name: str    age: int    email: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate structured outputprompt = "Extract user: John Doe, 30 years old, [email protected]"generator = outlines.generate.json(model, User)user = generator(prompt)
print(user.name)   # "John Doe"print(user.age)    # 30print(user.email)  # "[email protected]"

Core Concepts

1. Constrained Token Sampling

Outlines uses Finite State Machines (FSM) to constrain token generation at the logit level.

How it works:

  1. Convert schema (JSON/Pydantic/regex) to context-free grammar (CFG)
  2. Transform CFG into Finite State Machine (FSM)
  3. Filter invalid tokens at each step during generation
  4. Fast-forward when only one valid token exists

Benefits:

  • Zero overhead: Filtering happens at token level
  • Speed improvement: Fast-forward through deterministic paths
  • Guaranteed validity: Invalid outputs impossible
python
import outlines
# Pydantic model -> JSON schema -> CFG -> FSMclass Person(BaseModel):    name: str    age: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Behind the scenes:# 1. Person -> JSON schema# 2. JSON schema -> CFG# 3. CFG -> FSM# 4. FSM filters tokens during generation
generator = outlines.generate.json(model, Person)result = generator("Generate person: Alice, 25")

2. Structured Generators

Outlines provides specialized generators for different output types.

Choice Generator
python
# Multiple choice selectiongenerator = outlines.generate.choice(    model,    ["positive", "negative", "neutral"])
sentiment = generator("Review: This is great!")# Result: One of the three choices
JSON Generator
python
from pydantic import BaseModel
class Product(BaseModel):    name: str    price: float    in_stock: bool
# Generate valid JSON matching schemagenerator = outlines.generate.json(model, Product)product = generator("Extract: iPhone 15, $999, available")
# Guaranteed valid Product instanceprint(type(product))  # <class '__main__.Product'>
Regex Generator
python
# Generate text matching regexgenerator = outlines.generate.regex(    model,    r"[0-9]{3}-[0-9]{3}-[0-9]{4}"  # Phone number pattern)
phone = generator("Generate phone number:")# Result: "555-123-4567" (guaranteed to match pattern)
Integer/Float Generators
python
# Generate specific numeric typesint_generator = outlines.generate.integer(model)age = int_generator("Person's age:")  # Guaranteed integer
float_generator = outlines.generate.float(model)price = float_generator("Product price:")  # Guaranteed float

3. Model Backends

Outlines supports multiple local and API-based backends.

Transformers (Hugging Face)
python
import outlines
# Load from Hugging Facemodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda"  # Or "cpu")
# Use with any generatorgenerator = outlines.generate.json(model, YourModel)
llama.cpp
python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b-instruct.Q4_K_M.gguf",    n_gpu_layers=35)
generator = outlines.generate.json(model, YourModel)
vLLM (High Throughput)
python
# For production deploymentsmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    tensor_parallel_size=2  # Multi-GPU)
generator = outlines.generate.json(model, YourModel)
OpenAI (Limited Support)
python
# Basic OpenAI supportmodel = outlines.models.openai(    "gpt-4o-mini",    api_key="your-api-key")
# Note: Some features limited with API modelsgenerator = outlines.generate.json(model, YourModel)

4. Pydantic Integration

Outlines has first-class Pydantic support with automatic schema translation.

Basic Models
python
from pydantic import BaseModel, Field
class Article(BaseModel):    title: str = Field(description="Article title")    author: str = Field(description="Author name")    word_count: int = Field(description="Number of words", gt=0)    tags: list[str] = Field(description="List of tags")
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, Article)
article = generator("Generate article about AI")print(article.title)print(article.word_count)  # Guaranteed > 0
Nested Models
python
class Address(BaseModel):    street: str    city: str    country: str
class Person(BaseModel):    name: str    age: int    address: Address  # Nested model
generator = outlines.generate.json(model, Person)person = generator("Generate person in New York")
print(person.address.city)  # "New York"
Enums and Literals
python
from enum import Enumfrom typing import Literal
class Status(str, Enum):    PENDING = "pending"    APPROVED = "approved"    REJECTED = "rejected"
class Application(BaseModel):    applicant: str    status: Status  # Must be one of enum values    priority: Literal["low", "medium", "high"]  # Must be one of literals
generator = outlines.generate.json(model, Application)app = generator("Generate application")
print(app.status)  # Status.PENDING (or APPROVED/REJECTED)

Common Patterns

Pattern 1: Data Extraction

python
from pydantic import BaseModelimport outlines
class CompanyInfo(BaseModel):    name: str    founded_year: int    industry: str    employees: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, CompanyInfo)
text = """Apple Inc. was founded in 1976 in the technology industry.The company employs approximately 164,000 people worldwide."""
prompt = f"Extract company information:\n{text}\n\nCompany:"company = generator(prompt)
print(f"Name: {company.name}")print(f"Founded: {company.founded_year}")print(f"Industry: {company.industry}")print(f"Employees: {company.employees}")

Pattern 2: Classification

python
from typing import Literalimport outlines
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Binary classificationgenerator = outlines.generate.choice(model, ["spam", "not_spam"])result = generator("Email: Buy now! 50% off!")
# Multi-class classificationcategories = ["technology", "business", "sports", "entertainment"]category_gen = outlines.generate.choice(model, categories)category = category_gen("Article: Apple announces new iPhone...")
# With confidenceclass Classification(BaseModel):    label: Literal["positive", "negative", "neutral"]    confidence: float
classifier = outlines.generate.json(model, Classification)result = classifier("Review: This product is okay, nothing special")

Pattern 3: Structured Forms

python
class UserProfile(BaseModel):    full_name: str    age: int    email: str    phone: str    country: str    interests: list[str]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, UserProfile)
prompt = """Extract user profile from:Name: Alice JohnsonAge: 28Email: [email protected]Phone: 555-0123Country: USAInterests: hiking, photography, cooking"""
profile = generator(prompt)print(profile.full_name)print(profile.interests)  # ["hiking", "photography", "cooking"]

Pattern 4: Multi-Entity Extraction

python
class Entity(BaseModel):    name: str    type: Literal["PERSON", "ORGANIZATION", "LOCATION"]
class DocumentEntities(BaseModel):    entities: list[Entity]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, DocumentEntities)
text = "Tim Cook met with Satya Nadella at Microsoft headquarters in Redmond."prompt = f"Extract entities from: {text}"
result = generator(prompt)for entity in result.entities:    print(f"{entity.name} ({entity.type})")

Pattern 5: Code Generation

python
class PythonFunction(BaseModel):    function_name: str    parameters: list[str]    docstring: str    body: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, PythonFunction)
prompt = "Generate a Python function to calculate factorial"func = generator(prompt)
print(f"def {func.function_name}({', '.join(func.parameters)}):")print(f'    """{func.docstring}"""')print(f"    {func.body}")

Pattern 6: Batch Processing

python
def batch_extract(texts: list[str], schema: type[BaseModel]):    """Extract structured data from multiple texts."""    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")    generator = outlines.generate.json(model, schema)
    results = []    for text in texts:        result = generator(f"Extract from: {text}")        results.append(result)
    return results
class Person(BaseModel):    name: str    age: int
texts = [    "John is 30 years old",    "Alice is 25 years old",    "Bob is 40 years old"]
people = batch_extract(texts, Person)for person in people:    print(f"{person.name}: {person.age}")

Backend Configuration

Transformers

python
import outlines
# Basic usagemodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# GPU configurationmodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda",    model_kwargs={"torch_dtype": "float16"})
# Popular modelsmodel = outlines.models.transformers("meta-llama/Llama-3.1-8B-Instruct")model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.3")model = outlines.models.transformers("Qwen/Qwen2.5-7B-Instruct")

llama.cpp

python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b.Q4_K_M.gguf",    n_ctx=4096,         # Context window    n_gpu_layers=35,    # GPU layers    n_threads=8         # CPU threads)
# Full GPU offloadmodel = outlines.models.llamacpp(    "./models/model.gguf",    n_gpu_layers=-1  # All layers on GPU)

vLLM (Production)

python
# Single GPUmodel = outlines.models.vllm("meta-llama/Llama-3.1-8B-Instruct")
# Multi-GPUmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-70B-Instruct",    tensor_parallel_size=4  # 4 GPUs)
# With quantizationmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    quantization="awq"  # Or "gptq")

Best Practices

1. Use Specific Types

python
# ✅ Good: Specific typesclass Product(BaseModel):    name: str    price: float  # Not str    quantity: int  # Not str    in_stock: bool  # Not str
# ❌ Bad: Everything as stringclass Product(BaseModel):    name: str    price: str  # Should be float    quantity: str  # Should be int

2. Add Constraints

python
from pydantic import Field
# ✅ Good: With constraintsclass User(BaseModel):    name: str = Field(min_length=1, max_length=100)    age: int = Field(ge=0, le=120)    email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")
# ❌ Bad: No constraintsclass User(BaseModel):    name: str    age: int    email: str

3. Use Enums for Categories

python
# ✅ Good: Enum for fixed setclass Priority(str, Enum):    LOW = "low"    MEDIUM = "medium"    HIGH = "high"
class Task(BaseModel):    title: str    priority: Priority
# ❌ Bad: Free-form stringclass Task(BaseModel):    title: str    priority: str  # Can be anything

4. Provide Context in Prompts

python
# ✅ Good: Clear contextprompt = """Extract product information from the following text.Text: iPhone 15 Pro costs $999 and is currently in stock.Product:"""
# ❌ Bad: Minimal contextprompt = "iPhone 15 Pro costs $999 and is currently in stock."

5. Handle Optional Fields

python
from typing import Optional
# ✅ Good: Optional fields for incomplete dataclass Article(BaseModel):    title: str  # Required    author: Optional[str] = None  # Optional    date: Optional[str] = None  # Optional    tags: list[str] = []  # Default empty list
# Can succeed even if author/date missing

Comparison to Alternatives

FeatureOutlinesInstructorGuidanceLMQL
Pydantic Support✅ Native✅ Native❌ No❌ No
JSON Schema✅ Yes✅ Yes⚠️ Limited✅ Yes
Regex Constraints✅ Yes❌ No✅ Yes✅ Yes
Local Models✅ Full⚠️ Limited✅ Full✅ Full
API Models⚠️ Limited✅ Full✅ Full✅ Full
Zero Overhead✅ Yes❌ No⚠️ Partial✅ Yes
Automatic Retrying❌ No✅ Yes❌ No❌ No
Learning CurveLowLowLowHigh

When to choose Outlines:

  • Using local models (Transformers, llama.cpp, vLLM)
  • Need maximum inference speed
  • Want Pydantic model support
  • Require zero-overhead structured generation
  • Control token sampling process

When to choose alternatives:

  • Instructor: Need API models with automatic retrying
  • Guidance: Need token healing and complex workflows
  • LMQL: Prefer declarative query syntax

Performance Characteristics

Speed:

  • Zero overhead: Structured generation as fast as unconstrained
  • Fast-forward optimization: Skips deterministic tokens
  • 1.2-2x faster than post-generation validation approaches

Memory:

  • FSM compiled once per schema (cached)
  • Minimal runtime overhead
  • Efficient with vLLM for high throughput

Accuracy:

  • 100% valid outputs (guaranteed by FSM)
  • No retry loops needed
  • Deterministic token filtering

Resources

See Also

  • references/json_generation.md - Comprehensive JSON and Pydantic patterns
  • references/backends.md - Backend-specific configuration
  • references/examples.md - Production-ready examples

來源與署名

來源：orchestra-research/ai-research-skills位於16-prompt-engineering/outlines提交773a529

授權條款： MIT

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架

更多來自 orchestra-research/ai-research-skills 的技能

Speculative Decoding

orchestra-research

指導使用推測解碼、Medusa 多頭預測與前瞻解碼來加速大型語言模型推論。

AI & Agents13K3 個月前更新

Systems Paper Writing

orchestra-research

為 OSDI、SOSP、ASPLOS、NSDI 等系統會議論文寫作提供段落層級結構與篇幅分配指引。

Writing & Content13K3 個月前更新

Moe Training

orchestra-research

指導使用 DeepSpeed 或 HuggingFace 訓練與推論混合專家模型，涵蓋路由、負載平衡與專家平行。

AI & Agents13K3 個月前更新

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

待分類13K3 個月前更新

Model Pruning

orchestra-research

指導使用 Wanda、SparseGPT、幅度剪枝與 N:M 方法壓縮 LLM，縮小模型體積並加速推論。

AI & Agents13K3 個月前更新

Ml Paper Writing

orchestra-research

指導撰寫可發表的機器學習／人工智慧會議論文，從研究倉庫到 LaTeX 投稿定稿。

Writing & Content13K3 個月前更新

更多AI & Agents技能

Ppt Template Creator

anthropics

精選

將使用者的 PowerPoint 範本轉換為可重用的技能，用來產生品牌簡報。

AI & Agents39K2 週前更新

Skill Development

anthropics

精選

指導建立 Claude Code 外掛技能，涵蓋結構、描述、漸進式揭露與驗證。

AI & Agents37K今天更新

Plugin Structure

anthropics

精選

說明 Claude Code 外掛的結構、資訊清單與元件配置方式。

AI & Agents37K今天更新

Command Development

anthropics

精選

指導建立 Claude Code 斜線命令，涵蓋結構、YAML frontmatter、參數與外掛功能。

AI & Agents37K今天更新

Claude Md Improver

anthropics

精選

稽核儲存庫中的 CLAUDE.md 檔案、評估其品質，並在取得核准後套用針對性的改進。

AI & Agents37K今天更新

Find Skills

vercel-labs

精選

指導透過 npx skills 命令列工具探索、評估並安裝代理技能。

AI & Agents33K今天更新