Outlines

orchestra-research/ai-research-skills/16-prompt-engineering/outlines

作者 orchestra-research773a52944ba4MIT13K 个星标收录于 2026年10月8日更新于 2026年10月8日仓库3个月前更新

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

仅含说明AI & Agents
AI 生成的概览

指导使用 Outlines 库，通过本地模型进行受结构约束的文本生成。

功能
该技能说明如何使用 Outlines 库将语言模型输出限制为合法的 JSON、XML、正则或代码结构。内容涵盖 Pydantic 模型集成，choice、JSON、正则和数值生成器，以及 Transformers、llama.cpp、vLLM 和 OpenAI 的后端配置。还列出数据抽取、分类、结构化表单和批处理等常见模式，并给出最佳实践和与其他方案的对比。
适用场景
当你需要语言模型输出保证合法的结构化结果时使用，例如符合模式的 JSON 或固定集合中的取值。它适合使用本地模型、重视类型安全和语法约束生成以及推理速度的场景。
运行要求
需要 outlines Python 包，以及可选的 transformers、llama-cpp-python 或 vllm 等后端，并使用 pydantic 定义模型。根据后端不同，还需要本地模型权重或 API 密钥。该技能不包含脚本，只有说明文档和参考文档。

Outlines: Structured Text Generation

When to Use This Skill

Use Outlines when you need to:

  • Guarantee valid JSON/XML/code structure during generation
  • Use Pydantic models for type-safe outputs
  • Support local models (Transformers, llama.cpp, vLLM)
  • Maximize inference speed with zero-overhead structured generation
  • Generate against JSON schemas automatically
  • Control token sampling at the grammar level

GitHub Stars: 8,000+ | From: dottxt.ai (formerly .txt)

Installation

bash
# Base installationpip install outlines
# With specific backendspip install outlines transformers  # Hugging Face modelspip install outlines llama-cpp-python  # llama.cpppip install outlines vllm  # vLLM for high-throughput

Quick Start

Basic Example: Classification

python
import outlinesfrom typing import Literal
# Load modelmodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate with type constraintprompt = "Sentiment of 'This product is amazing!': "generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])sentiment = generator(prompt)
print(sentiment)  # "positive" (guaranteed one of these)

With Pydantic Models

python
from pydantic import BaseModelimport outlines
class User(BaseModel):    name: str    age: int    email: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate structured outputprompt = "Extract user: John Doe, 30 years old, [email protected]"generator = outlines.generate.json(model, User)user = generator(prompt)
print(user.name)   # "John Doe"print(user.age)    # 30print(user.email)  # "[email protected]"

Core Concepts

1. Constrained Token Sampling

Outlines uses Finite State Machines (FSM) to constrain token generation at the logit level.

How it works:

  1. Convert schema (JSON/Pydantic/regex) to context-free grammar (CFG)
  2. Transform CFG into Finite State Machine (FSM)
  3. Filter invalid tokens at each step during generation
  4. Fast-forward when only one valid token exists

Benefits:

  • Zero overhead: Filtering happens at token level
  • Speed improvement: Fast-forward through deterministic paths
  • Guaranteed validity: Invalid outputs impossible
python
import outlines
# Pydantic model -> JSON schema -> CFG -> FSMclass Person(BaseModel):    name: str    age: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Behind the scenes:# 1. Person -> JSON schema# 2. JSON schema -> CFG# 3. CFG -> FSM# 4. FSM filters tokens during generation
generator = outlines.generate.json(model, Person)result = generator("Generate person: Alice, 25")

2. Structured Generators

Outlines provides specialized generators for different output types.

Choice Generator
python
# Multiple choice selectiongenerator = outlines.generate.choice(    model,    ["positive", "negative", "neutral"])
sentiment = generator("Review: This is great!")# Result: One of the three choices
JSON Generator
python
from pydantic import BaseModel
class Product(BaseModel):    name: str    price: float    in_stock: bool
# Generate valid JSON matching schemagenerator = outlines.generate.json(model, Product)product = generator("Extract: iPhone 15, $999, available")
# Guaranteed valid Product instanceprint(type(product))  # <class '__main__.Product'>
Regex Generator
python
# Generate text matching regexgenerator = outlines.generate.regex(    model,    r"[0-9]{3}-[0-9]{3}-[0-9]{4}"  # Phone number pattern)
phone = generator("Generate phone number:")# Result: "555-123-4567" (guaranteed to match pattern)
Integer/Float Generators
python
# Generate specific numeric typesint_generator = outlines.generate.integer(model)age = int_generator("Person's age:")  # Guaranteed integer
float_generator = outlines.generate.float(model)price = float_generator("Product price:")  # Guaranteed float

3. Model Backends

Outlines supports multiple local and API-based backends.

Transformers (Hugging Face)
python
import outlines
# Load from Hugging Facemodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda"  # Or "cpu")
# Use with any generatorgenerator = outlines.generate.json(model, YourModel)
llama.cpp
python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b-instruct.Q4_K_M.gguf",    n_gpu_layers=35)
generator = outlines.generate.json(model, YourModel)
vLLM (High Throughput)
python
# For production deploymentsmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    tensor_parallel_size=2  # Multi-GPU)
generator = outlines.generate.json(model, YourModel)
OpenAI (Limited Support)
python
# Basic OpenAI supportmodel = outlines.models.openai(    "gpt-4o-mini",    api_key="your-api-key")
# Note: Some features limited with API modelsgenerator = outlines.generate.json(model, YourModel)

4. Pydantic Integration

Outlines has first-class Pydantic support with automatic schema translation.

Basic Models
python
from pydantic import BaseModel, Field
class Article(BaseModel):    title: str = Field(description="Article title")    author: str = Field(description="Author name")    word_count: int = Field(description="Number of words", gt=0)    tags: list[str] = Field(description="List of tags")
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, Article)
article = generator("Generate article about AI")print(article.title)print(article.word_count)  # Guaranteed > 0
Nested Models
python
class Address(BaseModel):    street: str    city: str    country: str
class Person(BaseModel):    name: str    age: int    address: Address  # Nested model
generator = outlines.generate.json(model, Person)person = generator("Generate person in New York")
print(person.address.city)  # "New York"
Enums and Literals
python
from enum import Enumfrom typing import Literal
class Status(str, Enum):    PENDING = "pending"    APPROVED = "approved"    REJECTED = "rejected"
class Application(BaseModel):    applicant: str    status: Status  # Must be one of enum values    priority: Literal["low", "medium", "high"]  # Must be one of literals
generator = outlines.generate.json(model, Application)app = generator("Generate application")
print(app.status)  # Status.PENDING (or APPROVED/REJECTED)

Common Patterns

Pattern 1: Data Extraction

python
from pydantic import BaseModelimport outlines
class CompanyInfo(BaseModel):    name: str    founded_year: int    industry: str    employees: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, CompanyInfo)
text = """Apple Inc. was founded in 1976 in the technology industry.The company employs approximately 164,000 people worldwide."""
prompt = f"Extract company information:\n{text}\n\nCompany:"company = generator(prompt)
print(f"Name: {company.name}")print(f"Founded: {company.founded_year}")print(f"Industry: {company.industry}")print(f"Employees: {company.employees}")

Pattern 2: Classification

python
from typing import Literalimport outlines
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Binary classificationgenerator = outlines.generate.choice(model, ["spam", "not_spam"])result = generator("Email: Buy now! 50% off!")
# Multi-class classificationcategories = ["technology", "business", "sports", "entertainment"]category_gen = outlines.generate.choice(model, categories)category = category_gen("Article: Apple announces new iPhone...")
# With confidenceclass Classification(BaseModel):    label: Literal["positive", "negative", "neutral"]    confidence: float
classifier = outlines.generate.json(model, Classification)result = classifier("Review: This product is okay, nothing special")

Pattern 3: Structured Forms

python
class UserProfile(BaseModel):    full_name: str    age: int    email: str    phone: str    country: str    interests: list[str]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, UserProfile)
prompt = """Extract user profile from:Name: Alice JohnsonAge: 28Email: [email protected]Phone: 555-0123Country: USAInterests: hiking, photography, cooking"""
profile = generator(prompt)print(profile.full_name)print(profile.interests)  # ["hiking", "photography", "cooking"]

Pattern 4: Multi-Entity Extraction

python
class Entity(BaseModel):    name: str    type: Literal["PERSON", "ORGANIZATION", "LOCATION"]
class DocumentEntities(BaseModel):    entities: list[Entity]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, DocumentEntities)
text = "Tim Cook met with Satya Nadella at Microsoft headquarters in Redmond."prompt = f"Extract entities from: {text}"
result = generator(prompt)for entity in result.entities:    print(f"{entity.name} ({entity.type})")

Pattern 5: Code Generation

python
class PythonFunction(BaseModel):    function_name: str    parameters: list[str]    docstring: str    body: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, PythonFunction)
prompt = "Generate a Python function to calculate factorial"func = generator(prompt)
print(f"def {func.function_name}({', '.join(func.parameters)}):")print(f'    """{func.docstring}"""')print(f"    {func.body}")

Pattern 6: Batch Processing

python
def batch_extract(texts: list[str], schema: type[BaseModel]):    """Extract structured data from multiple texts."""    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")    generator = outlines.generate.json(model, schema)
    results = []    for text in texts:        result = generator(f"Extract from: {text}")        results.append(result)
    return results
class Person(BaseModel):    name: str    age: int
texts = [    "John is 30 years old",    "Alice is 25 years old",    "Bob is 40 years old"]
people = batch_extract(texts, Person)for person in people:    print(f"{person.name}: {person.age}")

Backend Configuration

Transformers

python
import outlines
# Basic usagemodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# GPU configurationmodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda",    model_kwargs={"torch_dtype": "float16"})
# Popular modelsmodel = outlines.models.transformers("meta-llama/Llama-3.1-8B-Instruct")model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.3")model = outlines.models.transformers("Qwen/Qwen2.5-7B-Instruct")

llama.cpp

python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b.Q4_K_M.gguf",    n_ctx=4096,         # Context window    n_gpu_layers=35,    # GPU layers    n_threads=8         # CPU threads)
# Full GPU offloadmodel = outlines.models.llamacpp(    "./models/model.gguf",    n_gpu_layers=-1  # All layers on GPU)

vLLM (Production)

python
# Single GPUmodel = outlines.models.vllm("meta-llama/Llama-3.1-8B-Instruct")
# Multi-GPUmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-70B-Instruct",    tensor_parallel_size=4  # 4 GPUs)
# With quantizationmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    quantization="awq"  # Or "gptq")

Best Practices

1. Use Specific Types

python
# ✅ Good: Specific typesclass Product(BaseModel):    name: str    price: float  # Not str    quantity: int  # Not str    in_stock: bool  # Not str
# ❌ Bad: Everything as stringclass Product(BaseModel):    name: str    price: str  # Should be float    quantity: str  # Should be int

2. Add Constraints

python
from pydantic import Field
# ✅ Good: With constraintsclass User(BaseModel):    name: str = Field(min_length=1, max_length=100)    age: int = Field(ge=0, le=120)    email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")
# ❌ Bad: No constraintsclass User(BaseModel):    name: str    age: int    email: str

3. Use Enums for Categories

python
# ✅ Good: Enum for fixed setclass Priority(str, Enum):    LOW = "low"    MEDIUM = "medium"    HIGH = "high"
class Task(BaseModel):    title: str    priority: Priority
# ❌ Bad: Free-form stringclass Task(BaseModel):    title: str    priority: str  # Can be anything

4. Provide Context in Prompts

python
# ✅ Good: Clear contextprompt = """Extract product information from the following text.Text: iPhone 15 Pro costs $999 and is currently in stock.Product:"""
# ❌ Bad: Minimal contextprompt = "iPhone 15 Pro costs $999 and is currently in stock."

5. Handle Optional Fields

python
from typing import Optional
# ✅ Good: Optional fields for incomplete dataclass Article(BaseModel):    title: str  # Required    author: Optional[str] = None  # Optional    date: Optional[str] = None  # Optional    tags: list[str] = []  # Default empty list
# Can succeed even if author/date missing

Comparison to Alternatives

FeatureOutlinesInstructorGuidanceLMQL
Pydantic Support✅ Native✅ Native❌ No❌ No
JSON Schema✅ Yes✅ Yes⚠️ Limited✅ Yes
Regex Constraints✅ Yes❌ No✅ Yes✅ Yes
Local Models✅ Full⚠️ Limited✅ Full✅ Full
API Models⚠️ Limited✅ Full✅ Full✅ Full
Zero Overhead✅ Yes❌ No⚠️ Partial✅ Yes
Automatic Retrying❌ No✅ Yes❌ No❌ No
Learning CurveLowLowLowHigh

When to choose Outlines:

  • Using local models (Transformers, llama.cpp, vLLM)
  • Need maximum inference speed
  • Want Pydantic model support
  • Require zero-overhead structured generation
  • Control token sampling process

When to choose alternatives:

  • Instructor: Need API models with automatic retrying
  • Guidance: Need token healing and complex workflows
  • LMQL: Prefer declarative query syntax

Performance Characteristics

Speed:

  • Zero overhead: Structured generation as fast as unconstrained
  • Fast-forward optimization: Skips deterministic tokens
  • 1.2-2x faster than post-generation validation approaches

Memory:

  • FSM compiled once per schema (cached)
  • Minimal runtime overhead
  • Efficient with vLLM for high throughput

Accuracy:

  • 100% valid outputs (guaranteed by FSM)
  • No retry loops needed
  • Deterministic token filtering

Resources

See Also

  • references/json_generation.md - Comprehensive JSON and Pydantic patterns
  • references/backends.md - Backend-specific configuration
  • references/examples.md - Production-ready examples

来源与署名

来源：orchestra-research/ai-research-skills位于16-prompt-engineering/outlines提交773a529

许可证： MIT

内容归原作者所有。SourceWeft 从公开仓库中收录这些内容。

举报或申请下架

更多来自 orchestra-research/ai-research-skills 的技能

Speculative Decoding

orchestra-research

指导使用推测解码、Medusa 多头预测和前瞻解码来加速大语言模型推理。

AI & Agents13K3个月前更新

Systems Paper Writing

orchestra-research

为 OSDI、SOSP、ASPLOS、NSDI 等系统会议论文写作提供段落级结构与篇幅分配指导。

Writing & Content13K3个月前更新

Moe Training

orchestra-research

指导使用 DeepSpeed 或 HuggingFace 训练与推理混合专家模型，涵盖路由、负载均衡与专家并行。

AI & Agents13K3个月前更新

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

待分类13K3个月前更新

Model Pruning

orchestra-research

指导使用 Wanda、SparseGPT、幅度剪枝和 N:M 方法压缩 LLM，减小模型体积并加速推理。

AI & Agents13K3个月前更新

Ml Paper Writing

orchestra-research

指导撰写可发表的机器学习/人工智能会议论文，从研究仓库到 LaTeX 投稿定稿。

Writing & Content13K3个月前更新

更多AI & Agents技能

Ppt Template Creator

anthropics

精选

将用户的 PowerPoint 模板转化为可复用技能，用于生成品牌演示文稿。

AI & Agents39K2周前更新

Skill Development

anthropics

精选

指导创建 Claude Code 插件技能，涵盖结构、描述、渐进式披露与验证。

AI & Agents37K今天更新

Plugin Structure

anthropics

精选

指导 Claude Code 插件的结构、清单与组件布局。

AI & Agents37K今天更新

Command Development

anthropics

精选

指导创建 Claude Code 斜杠命令，涵盖结构、YAML frontmatter、参数与插件功能。

AI & Agents37K今天更新

Claude Md Improver

anthropics

精选

审查仓库中的 CLAUDE.md 文件，评估其质量，并在获得批准后应用有针对性的改进。

AI & Agents37K今天更新

Find Skills

vercel-labs

精选

指导通过 npx skills 命令行工具发现、评估并安装智能体技能。

AI & Agents33K今天更新