Outlines

orchestra-research/ai-research-skills/16-prompt-engineering/outlines

by orchestra-research773a52944ba4MIT13K starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated 3 months ago

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

Instructions onlyAI & Agents
AI-generated overview

Guides use of the Outlines library for structured, schema-constrained text generation with local models.

What it does
This skill documents how to use the Outlines library to constrain language model output to valid JSON, XML, regex or code structures. It covers Pydantic model integration, choice, JSON, regex and numeric generators, and backend configuration for Transformers, llama.cpp, vLLM and OpenAI. It also lists common patterns such as data extraction, classification, structured forms and batch processing, plus best practices and comparisons with alternatives.
When to use it
Use it when you need guaranteed-valid structured output from a language model, such as JSON matching a schema or a value from a fixed set. It suits work with local models where type-safe, grammar-constrained generation and inference speed matter.
Requirements
Requires the outlines Python package, plus optional backends such as transformers, llama-cpp-python or vllm, and pydantic for model definitions. Local model weights or an API key are needed depending on the backend. It ships no scripts; it is instructions plus reference documents.

Outlines: Structured Text Generation

When to Use This Skill

Use Outlines when you need to:

  • Guarantee valid JSON/XML/code structure during generation
  • Use Pydantic models for type-safe outputs
  • Support local models (Transformers, llama.cpp, vLLM)
  • Maximize inference speed with zero-overhead structured generation
  • Generate against JSON schemas automatically
  • Control token sampling at the grammar level

GitHub Stars: 8,000+ | From: dottxt.ai (formerly .txt)

Installation

bash
# Base installationpip install outlines
# With specific backendspip install outlines transformers  # Hugging Face modelspip install outlines llama-cpp-python  # llama.cpppip install outlines vllm  # vLLM for high-throughput

Quick Start

Basic Example: Classification

python
import outlinesfrom typing import Literal
# Load modelmodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate with type constraintprompt = "Sentiment of 'This product is amazing!': "generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])sentiment = generator(prompt)
print(sentiment)  # "positive" (guaranteed one of these)

With Pydantic Models

python
from pydantic import BaseModelimport outlines
class User(BaseModel):    name: str    age: int    email: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate structured outputprompt = "Extract user: John Doe, 30 years old, [email protected]"generator = outlines.generate.json(model, User)user = generator(prompt)
print(user.name)   # "John Doe"print(user.age)    # 30print(user.email)  # "[email protected]"

Core Concepts

1. Constrained Token Sampling

Outlines uses Finite State Machines (FSM) to constrain token generation at the logit level.

How it works:

  1. Convert schema (JSON/Pydantic/regex) to context-free grammar (CFG)
  2. Transform CFG into Finite State Machine (FSM)
  3. Filter invalid tokens at each step during generation
  4. Fast-forward when only one valid token exists

Benefits:

  • Zero overhead: Filtering happens at token level
  • Speed improvement: Fast-forward through deterministic paths
  • Guaranteed validity: Invalid outputs impossible
python
import outlines
# Pydantic model -> JSON schema -> CFG -> FSMclass Person(BaseModel):    name: str    age: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Behind the scenes:# 1. Person -> JSON schema# 2. JSON schema -> CFG# 3. CFG -> FSM# 4. FSM filters tokens during generation
generator = outlines.generate.json(model, Person)result = generator("Generate person: Alice, 25")

2. Structured Generators

Outlines provides specialized generators for different output types.

Choice Generator
python
# Multiple choice selectiongenerator = outlines.generate.choice(    model,    ["positive", "negative", "neutral"])
sentiment = generator("Review: This is great!")# Result: One of the three choices
JSON Generator
python
from pydantic import BaseModel
class Product(BaseModel):    name: str    price: float    in_stock: bool
# Generate valid JSON matching schemagenerator = outlines.generate.json(model, Product)product = generator("Extract: iPhone 15, $999, available")
# Guaranteed valid Product instanceprint(type(product))  # <class '__main__.Product'>
Regex Generator
python
# Generate text matching regexgenerator = outlines.generate.regex(    model,    r"[0-9]{3}-[0-9]{3}-[0-9]{4}"  # Phone number pattern)
phone = generator("Generate phone number:")# Result: "555-123-4567" (guaranteed to match pattern)
Integer/Float Generators
python
# Generate specific numeric typesint_generator = outlines.generate.integer(model)age = int_generator("Person's age:")  # Guaranteed integer
float_generator = outlines.generate.float(model)price = float_generator("Product price:")  # Guaranteed float

3. Model Backends

Outlines supports multiple local and API-based backends.

Transformers (Hugging Face)
python
import outlines
# Load from Hugging Facemodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda"  # Or "cpu")
# Use with any generatorgenerator = outlines.generate.json(model, YourModel)
llama.cpp
python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b-instruct.Q4_K_M.gguf",    n_gpu_layers=35)
generator = outlines.generate.json(model, YourModel)
vLLM (High Throughput)
python
# For production deploymentsmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    tensor_parallel_size=2  # Multi-GPU)
generator = outlines.generate.json(model, YourModel)
OpenAI (Limited Support)
python
# Basic OpenAI supportmodel = outlines.models.openai(    "gpt-4o-mini",    api_key="your-api-key")
# Note: Some features limited with API modelsgenerator = outlines.generate.json(model, YourModel)

4. Pydantic Integration

Outlines has first-class Pydantic support with automatic schema translation.

Basic Models
python
from pydantic import BaseModel, Field
class Article(BaseModel):    title: str = Field(description="Article title")    author: str = Field(description="Author name")    word_count: int = Field(description="Number of words", gt=0)    tags: list[str] = Field(description="List of tags")
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, Article)
article = generator("Generate article about AI")print(article.title)print(article.word_count)  # Guaranteed > 0
Nested Models
python
class Address(BaseModel):    street: str    city: str    country: str
class Person(BaseModel):    name: str    age: int    address: Address  # Nested model
generator = outlines.generate.json(model, Person)person = generator("Generate person in New York")
print(person.address.city)  # "New York"
Enums and Literals
python
from enum import Enumfrom typing import Literal
class Status(str, Enum):    PENDING = "pending"    APPROVED = "approved"    REJECTED = "rejected"
class Application(BaseModel):    applicant: str    status: Status  # Must be one of enum values    priority: Literal["low", "medium", "high"]  # Must be one of literals
generator = outlines.generate.json(model, Application)app = generator("Generate application")
print(app.status)  # Status.PENDING (or APPROVED/REJECTED)

Common Patterns

Pattern 1: Data Extraction

python
from pydantic import BaseModelimport outlines
class CompanyInfo(BaseModel):    name: str    founded_year: int    industry: str    employees: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, CompanyInfo)
text = """Apple Inc. was founded in 1976 in the technology industry.The company employs approximately 164,000 people worldwide."""
prompt = f"Extract company information:\n{text}\n\nCompany:"company = generator(prompt)
print(f"Name: {company.name}")print(f"Founded: {company.founded_year}")print(f"Industry: {company.industry}")print(f"Employees: {company.employees}")

Pattern 2: Classification

python
from typing import Literalimport outlines
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Binary classificationgenerator = outlines.generate.choice(model, ["spam", "not_spam"])result = generator("Email: Buy now! 50% off!")
# Multi-class classificationcategories = ["technology", "business", "sports", "entertainment"]category_gen = outlines.generate.choice(model, categories)category = category_gen("Article: Apple announces new iPhone...")
# With confidenceclass Classification(BaseModel):    label: Literal["positive", "negative", "neutral"]    confidence: float
classifier = outlines.generate.json(model, Classification)result = classifier("Review: This product is okay, nothing special")

Pattern 3: Structured Forms

python
class UserProfile(BaseModel):    full_name: str    age: int    email: str    phone: str    country: str    interests: list[str]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, UserProfile)
prompt = """Extract user profile from:Name: Alice JohnsonAge: 28Email: [email protected]Phone: 555-0123Country: USAInterests: hiking, photography, cooking"""
profile = generator(prompt)print(profile.full_name)print(profile.interests)  # ["hiking", "photography", "cooking"]

Pattern 4: Multi-Entity Extraction

python
class Entity(BaseModel):    name: str    type: Literal["PERSON", "ORGANIZATION", "LOCATION"]
class DocumentEntities(BaseModel):    entities: list[Entity]
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, DocumentEntities)
text = "Tim Cook met with Satya Nadella at Microsoft headquarters in Redmond."prompt = f"Extract entities from: {text}"
result = generator(prompt)for entity in result.entities:    print(f"{entity.name} ({entity.type})")

Pattern 5: Code Generation

python
class PythonFunction(BaseModel):    function_name: str    parameters: list[str]    docstring: str    body: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")generator = outlines.generate.json(model, PythonFunction)
prompt = "Generate a Python function to calculate factorial"func = generator(prompt)
print(f"def {func.function_name}({', '.join(func.parameters)}):")print(f'    """{func.docstring}"""')print(f"    {func.body}")

Pattern 6: Batch Processing

python
def batch_extract(texts: list[str], schema: type[BaseModel]):    """Extract structured data from multiple texts."""    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")    generator = outlines.generate.json(model, schema)
    results = []    for text in texts:        result = generator(f"Extract from: {text}")        results.append(result)
    return results
class Person(BaseModel):    name: str    age: int
texts = [    "John is 30 years old",    "Alice is 25 years old",    "Bob is 40 years old"]
people = batch_extract(texts, Person)for person in people:    print(f"{person.name}: {person.age}")

Backend Configuration

Transformers

python
import outlines
# Basic usagemodel = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# GPU configurationmodel = outlines.models.transformers(    "microsoft/Phi-3-mini-4k-instruct",    device="cuda",    model_kwargs={"torch_dtype": "float16"})
# Popular modelsmodel = outlines.models.transformers("meta-llama/Llama-3.1-8B-Instruct")model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.3")model = outlines.models.transformers("Qwen/Qwen2.5-7B-Instruct")

llama.cpp

python
# Load GGUF modelmodel = outlines.models.llamacpp(    "./models/llama-3.1-8b.Q4_K_M.gguf",    n_ctx=4096,         # Context window    n_gpu_layers=35,    # GPU layers    n_threads=8         # CPU threads)
# Full GPU offloadmodel = outlines.models.llamacpp(    "./models/model.gguf",    n_gpu_layers=-1  # All layers on GPU)

vLLM (Production)

python
# Single GPUmodel = outlines.models.vllm("meta-llama/Llama-3.1-8B-Instruct")
# Multi-GPUmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-70B-Instruct",    tensor_parallel_size=4  # 4 GPUs)
# With quantizationmodel = outlines.models.vllm(    "meta-llama/Llama-3.1-8B-Instruct",    quantization="awq"  # Or "gptq")

Best Practices

1. Use Specific Types

python
# ✅ Good: Specific typesclass Product(BaseModel):    name: str    price: float  # Not str    quantity: int  # Not str    in_stock: bool  # Not str
# ❌ Bad: Everything as stringclass Product(BaseModel):    name: str    price: str  # Should be float    quantity: str  # Should be int

2. Add Constraints

python
from pydantic import Field
# ✅ Good: With constraintsclass User(BaseModel):    name: str = Field(min_length=1, max_length=100)    age: int = Field(ge=0, le=120)    email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")
# ❌ Bad: No constraintsclass User(BaseModel):    name: str    age: int    email: str

3. Use Enums for Categories

python
# ✅ Good: Enum for fixed setclass Priority(str, Enum):    LOW = "low"    MEDIUM = "medium"    HIGH = "high"
class Task(BaseModel):    title: str    priority: Priority
# ❌ Bad: Free-form stringclass Task(BaseModel):    title: str    priority: str  # Can be anything

4. Provide Context in Prompts

python
# ✅ Good: Clear contextprompt = """Extract product information from the following text.Text: iPhone 15 Pro costs $999 and is currently in stock.Product:"""
# ❌ Bad: Minimal contextprompt = "iPhone 15 Pro costs $999 and is currently in stock."

5. Handle Optional Fields

python
from typing import Optional
# ✅ Good: Optional fields for incomplete dataclass Article(BaseModel):    title: str  # Required    author: Optional[str] = None  # Optional    date: Optional[str] = None  # Optional    tags: list[str] = []  # Default empty list
# Can succeed even if author/date missing

Comparison to Alternatives

FeatureOutlinesInstructorGuidanceLMQL
Pydantic Support✅ Native✅ Native❌ No❌ No
JSON Schema✅ Yes✅ Yes⚠️ Limited✅ Yes
Regex Constraints✅ Yes❌ No✅ Yes✅ Yes
Local Models✅ Full⚠️ Limited✅ Full✅ Full
API Models⚠️ Limited✅ Full✅ Full✅ Full
Zero Overhead✅ Yes❌ No⚠️ Partial✅ Yes
Automatic Retrying❌ No✅ Yes❌ No❌ No
Learning CurveLowLowLowHigh

When to choose Outlines:

  • Using local models (Transformers, llama.cpp, vLLM)
  • Need maximum inference speed
  • Want Pydantic model support
  • Require zero-overhead structured generation
  • Control token sampling process

When to choose alternatives:

  • Instructor: Need API models with automatic retrying
  • Guidance: Need token healing and complex workflows
  • LMQL: Prefer declarative query syntax

Performance Characteristics

Speed:

  • Zero overhead: Structured generation as fast as unconstrained
  • Fast-forward optimization: Skips deterministic tokens
  • 1.2-2x faster than post-generation validation approaches

Memory:

  • FSM compiled once per schema (cached)
  • Minimal runtime overhead
  • Efficient with vLLM for high throughput

Accuracy:

  • 100% valid outputs (guaranteed by FSM)
  • No retry loops needed
  • Deterministic token filtering

Resources

See Also

  • references/json_generation.md - Comprehensive JSON and Pydantic patterns
  • references/backends.md - Backend-specific configuration
  • references/examples.md - Production-ready examples

Source and attribution

Source:orchestra-research/ai-research-skillsin16-prompt-engineering/outlinesat commit773a529

License: MIT

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

More from orchestra-research/ai-research-skills

Speculative Decoding

orchestra-research

Guides accelerating LLM inference with speculative decoding, Medusa multi-head prediction, and lookahead decoding.

AI & Agents13Kupdated 3 months ago

Systems Paper Writing

orchestra-research

Guides paragraph-level structure and page budgets for writing systems conference papers such as OSDI, SOSP, ASPLOS and NSDI.

Writing & Content13Kupdated 3 months ago

Moe Training

orchestra-research

Guides training and inference of Mixture of Experts models with DeepSpeed or HuggingFace, covering routing, load balancing and expert parallelism.

AI & Agents13Kupdated 3 months ago

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

Awaiting classification13Kupdated 3 months ago

Model Pruning

orchestra-research

Guides LLM pruning with Wanda, SparseGPT, magnitude and N:M methods to cut model size and speed inference.

AI & Agents13Kupdated 3 months ago

Ml Paper Writing

orchestra-research

Guides writing publication-ready ML/AI conference papers, from research repo to camera-ready LaTeX submission.

Writing & Content13Kupdated 3 months ago

More in AI & Agents

Ppt Template Creator

anthropics

Featured

Turns a user's PowerPoint template into a reusable skill that generates branded presentations.

AI & Agents39Kupdated 2 weeks ago

Skill Development

anthropics

Featured

Guides creation of Claude Code plugin skills, covering structure, descriptions, progressive disclosure and validation.

AI & Agents37Kupdated today

Plugin Structure

anthropics

Featured

Guides the structure, manifest, and component layout of Claude Code plugins.

AI & Agents37Kupdated today

Command Development

anthropics

Featured

Guides creation of Claude Code slash commands, covering structure, YAML frontmatter, arguments and plugin features.

AI & Agents37Kupdated today

Claude Md Improver

anthropics

Featured

Audits CLAUDE.md files in a repository, scores their quality, and applies approved targeted improvements.

AI & Agents37Kupdated today

Find Skills

vercel-labs

Featured

Guides discovery, quality checking and installation of agent skills via the npx skills CLI.

AI & Agents33Kupdated today