Guidance

orchestra-research/ai-research-skills/16-prompt-engineering/guidance

作者 orchestra-research773a52944ba4MIT13K 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫3 個月前更新

Control LLM output with regex and grammars, guarantee valid JSON/XML/code generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework

僅含說明AI & Agents
AI 產生的概覽

指導使用 Microsoft 的 Guidance 函式庫，以正規表示式、文法和選項約束大型語言模型輸出，產生有效的結構化內容。

功能
此技能說明如何使用 Guidance 框架，透過正規表示式約束、上下文無關文法、選項清單和 token healing 來控制大型語言模型輸出。內容涵蓋安裝、Anthropic、OpenAI、Transformers 與 llama.cpp 的後端設定，以及可重複使用的 @guidance 函式。並提供 JSON 產生、分類、多步推理、ReAct 代理和實體擷取等常見模式。
適用情境
當你需要保證輸出為有效的 JSON、XML 或程式碼，或需要強制日期、電子郵件、ID 等格式時使用。也適合以 Python 風格控制流程建構多步大型語言模型工作流程，並相較於無約束提示降低延遲。
執行需求
需要 guidance Python 套件；使用本機模型時還需 transformers 或 llama.cpp 擴充。API 後端需要 ANTHROPIC_API_KEY 或 OPENAI_API_KEY 等憑證以及網路存取。此技能不含指令碼，只有參考文件。

Guidance: Constrained LLM Generation

When to Use This Skill

Use Guidance when you need to:

  • Control LLM output syntax with regex or grammars
  • Guarantee valid JSON/XML/code generation
  • Reduce latency vs traditional prompting approaches
  • Enforce structured formats (dates, emails, IDs, etc.)
  • Build multi-step workflows with Pythonic control flow
  • Prevent invalid outputs through grammatical constraints

GitHub Stars: 18,000+ | From: Microsoft Research

Installation

bash
# Base installationpip install guidance
# With specific backendspip install guidance[transformers]  # Hugging Face modelspip install guidance[llama_cpp]     # llama.cpp models

Quick Start

Basic Example: Structured Generation

python
from guidance import models, gen
# Load model (supports OpenAI, Transformers, llama.cpp)lm = models.OpenAI("gpt-4")
# Generate with constraintsresult = lm + "The capital of France is " + gen("capital", max_tokens=5)
print(result["capital"])  # "Paris"

With Anthropic Claude

python
from guidance import models, gen, system, user, assistant
# Configure Claudelm = models.Anthropic("claude-sonnet-4-5-20250929")
# Use context managers for chat formatwith system():    lm += "You are a helpful assistant."
with user():    lm += "What is the capital of France?"
with assistant():    lm += gen(max_tokens=20)

Core Concepts

1. Context Managers

Guidance uses Pythonic context managers for chat-style interactions.

python
from guidance import system, user, assistant, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# System messagewith system():    lm += "You are a JSON generation expert."
# User messagewith user():    lm += "Generate a person object with name and age."
# Assistant responsewith assistant():    lm += gen("response", max_tokens=100)
print(lm["response"])

Benefits:

  • Natural chat flow
  • Clear role separation
  • Easy to read and maintain

2. Constrained Generation

Guidance ensures outputs match specified patterns using regex or grammars.

Regex Constraints
python
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Constrain to valid email formatlm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
# Constrain to date format (YYYY-MM-DD)lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}")
# Constrain to phone numberlm += "Phone: " + gen("phone", regex=r"\d{3}-\d{3}-\d{4}")
print(lm["email"])  # Guaranteed valid emailprint(lm["date"])   # Guaranteed YYYY-MM-DD format

How it works:

  • Regex converted to grammar at token level
  • Invalid tokens filtered during generation
  • Model can only produce matching outputs
Selection Constraints
python
from guidance import models, gen, select
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Constrain to specific choiceslm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")
# Multiple-choice selectionlm += "Best answer: " + select(    ["A) Paris", "B) London", "C) Berlin", "D) Madrid"],    name="answer")
print(lm["sentiment"])  # One of: positive, negative, neutralprint(lm["answer"])     # One of: A, B, C, or D

3. Token Healing

Guidance automatically "heals" token boundaries between prompt and generation.

Problem: Tokenization creates unnatural boundaries.

python
# Without token healingprompt = "The capital of France is "# Last token: " is "# First generated token might be " Par" (with leading space)# Result: "The capital of France is  Paris" (double space!)

Solution: Guidance backs up one token and regenerates.

python
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# Token healing enabled by defaultlm += "The capital of France is " + gen("capital", max_tokens=5)# Result: "The capital of France is Paris" (correct spacing)

Benefits:

  • Natural text boundaries
  • No awkward spacing issues
  • Better model performance (sees natural token sequences)

4. Grammar-Based Generation

Define complex structures using context-free grammars.

python
from guidance import models, gen
lm = models.Anthropic("claude-sonnet-4-5-20250929")
# JSON grammar (simplified)json_grammar = """{    "name": <gen name regex="[A-Za-z ]+" max_tokens=20>,    "age": <gen age regex="[0-9]+" max_tokens=3>,    "email": <gen email regex="[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}" max_tokens=50>}"""
# Generate valid JSONlm += gen("person", grammar=json_grammar)
print(lm["person"])  # Guaranteed valid JSON structure

Use cases:

  • Complex structured outputs
  • Nested data structures
  • Programming language syntax
  • Domain-specific languages

5. Guidance Functions

Create reusable generation patterns with the @guidance decorator.

python
from guidance import guidance, gen, models
@guidancedef generate_person(lm):    """Generate a person with name and age."""    lm += "Name: " + gen("name", max_tokens=20, stop="\n")    lm += "\nAge: " + gen("age", regex=r"[0-9]+", max_tokens=3)    return lm
# Use the functionlm = models.Anthropic("claude-sonnet-4-5-20250929")lm = generate_person(lm)
print(lm["name"])print(lm["age"])

Stateful Functions:

python
@guidance(stateless=False)def react_agent(lm, question, tools, max_rounds=5):    """ReAct agent with tool use."""    lm += f"Question: {question}\n\n"
    for i in range(max_rounds):        # Thought        lm += f"Thought {i+1}: " + gen("thought", stop="\n")
        # Action        lm += "\nAction: " + select(list(tools.keys()), name="action")
        # Execute tool        tool_result = tools[lm["action"]]()        lm += f"\nObservation: {tool_result}\n\n"
        # Check if done        lm += "Done? " + select(["Yes", "No"], name="done")        if lm["done"] == "Yes":            break
    # Final answer    lm += "\nFinal Answer: " + gen("answer", max_tokens=100)    return lm

Backend Configuration

Anthropic Claude

python
from guidance import models
lm = models.Anthropic(    model="claude-sonnet-4-5-20250929",    api_key="your-api-key"  # Or set ANTHROPIC_API_KEY env var)

OpenAI

python
lm = models.OpenAI(    model="gpt-4o-mini",    api_key="your-api-key"  # Or set OPENAI_API_KEY env var)

Local Models (Transformers)

python
from guidance.models import Transformers
lm = Transformers(    "microsoft/Phi-4-mini-instruct",    device="cuda"  # Or "cpu")

Local Models (llama.cpp)

python
from guidance.models import LlamaCpp
lm = LlamaCpp(    model_path="/path/to/model.gguf",    n_ctx=4096,    n_gpu_layers=35)

Common Patterns

Pattern 1: JSON Generation

python
from guidance import models, gen, system, user, assistant
lm = models.Anthropic("claude-sonnet-4-5-20250929")
with system():    lm += "You generate valid JSON."
with user():    lm += "Generate a user profile with name, age, and email."
with assistant():    lm += """{    "name": """ + gen("name", regex=r'"[A-Za-z ]+"', max_tokens=30) + """,    "age": """ + gen("age", regex=r"[0-9]+", max_tokens=3) + """,    "email": """ + gen("email", regex=r'"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"', max_tokens=50) + """}"""
print(lm)  # Valid JSON guaranteed

Pattern 2: Classification

python
from guidance import models, gen, select
lm = models.Anthropic("claude-sonnet-4-5-20250929")
text = "This product is amazing! I love it."
lm += f"Text: {text}\n"lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")lm += "\nConfidence: " + gen("confidence", regex=r"[0-9]+", max_tokens=3) + "%"
print(f"Sentiment: {lm['sentiment']}")print(f"Confidence: {lm['confidence']}%")

Pattern 3: Multi-Step Reasoning

python
from guidance import models, gen, guidance
@guidancedef chain_of_thought(lm, question):    """Generate answer with step-by-step reasoning."""    lm += f"Question: {question}\n\n"
    # Generate multiple reasoning steps    for i in range(3):        lm += f"Step {i+1}: " + gen(f"step_{i+1}", stop="\n", max_tokens=100) + "\n"
    # Final answer    lm += "\nTherefore, the answer is: " + gen("answer", max_tokens=50)
    return lm
lm = models.Anthropic("claude-sonnet-4-5-20250929")lm = chain_of_thought(lm, "What is 15% of 200?")
print(lm["answer"])

Pattern 4: ReAct Agent

python
from guidance import models, gen, select, guidance
@guidance(stateless=False)def react_agent(lm, question):    """ReAct agent with tool use."""    tools = {        "calculator": lambda expr: eval(expr),        "search": lambda query: f"Search results for: {query}",    }
    lm += f"Question: {question}\n\n"
    for round in range(5):        # Thought        lm += f"Thought: " + gen("thought", stop="\n") + "\n"
        # Action selection        lm += "Action: " + select(["calculator", "search", "answer"], name="action")
        if lm["action"] == "answer":            lm += "\nFinal Answer: " + gen("answer", max_tokens=100)            break
        # Action input        lm += "\nAction Input: " + gen("action_input", stop="\n") + "\n"
        # Execute tool        if lm["action"] in tools:            result = tools[lm["action"]](lm["action_input"])            lm += f"Observation: {result}\n\n"
    return lm
lm = models.Anthropic("claude-sonnet-4-5-20250929")lm = react_agent(lm, "What is 25 * 4 + 10?")print(lm["answer"])

Pattern 5: Data Extraction

python
from guidance import models, gen, guidance
@guidancedef extract_entities(lm, text):    """Extract structured entities from text."""    lm += f"Text: {text}\n\n"
    # Extract person    lm += "Person: " + gen("person", stop="\n", max_tokens=30) + "\n"
    # Extract organization    lm += "Organization: " + gen("organization", stop="\n", max_tokens=30) + "\n"
    # Extract date    lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}", max_tokens=10) + "\n"
    # Extract location    lm += "Location: " + gen("location", stop="\n", max_tokens=30) + "\n"
    return lm
text = "Tim Cook announced at Apple Park on 2024-09-15 in Cupertino."
lm = models.Anthropic("claude-sonnet-4-5-20250929")lm = extract_entities(lm, text)
print(f"Person: {lm['person']}")print(f"Organization: {lm['organization']}")print(f"Date: {lm['date']}")print(f"Location: {lm['location']}")

Best Practices

1. Use Regex for Format Validation

python
# ✅ Good: Regex ensures valid formatlm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
# ❌ Bad: Free generation may produce invalid emailslm += "Email: " + gen("email", max_tokens=50)

2. Use select() for Fixed Categories

python
# ✅ Good: Guaranteed valid categorylm += "Status: " + select(["pending", "approved", "rejected"], name="status")
# ❌ Bad: May generate typos or invalid valueslm += "Status: " + gen("status", max_tokens=20)

3. Leverage Token Healing

python
# Token healing is enabled by default# No special action needed - just concatenate naturallylm += "The capital is " + gen("capital")  # Automatic healing

4. Use stop Sequences

python
# ✅ Good: Stop at newline for single-line outputslm += "Name: " + gen("name", stop="\n")
# ❌ Bad: May generate multiple lineslm += "Name: " + gen("name", max_tokens=50)

5. Create Reusable Functions

python
# ✅ Good: Reusable pattern@guidancedef generate_person(lm):    lm += "Name: " + gen("name", stop="\n")    lm += "\nAge: " + gen("age", regex=r"[0-9]+")    return lm
# Use multiple timeslm = generate_person(lm)lm += "\n\n"lm = generate_person(lm)

6. Balance Constraints

python
# ✅ Good: Reasonable constraintslm += gen("name", regex=r"[A-Za-z ]+", max_tokens=30)
# ❌ Too strict: May fail or be very slowlm += gen("name", regex=r"^(John|Jane)$", max_tokens=10)

Comparison to Alternatives

FeatureGuidanceInstructorOutlinesLMQL
Regex Constraints✅ Yes❌ No✅ Yes✅ Yes
Grammar Support✅ CFG❌ No✅ CFG✅ CFG
Pydantic Validation❌ No✅ Yes✅ Yes❌ No
Token Healing✅ Yes❌ No✅ Yes❌ No
Local Models✅ Yes⚠️ Limited✅ Yes✅ Yes
API Models✅ Yes✅ Yes⚠️ Limited✅ Yes
Pythonic Syntax✅ Yes✅ Yes✅ Yes❌ SQL-like
Learning CurveLowLowMediumHigh

When to choose Guidance:

  • Need regex/grammar constraints
  • Want token healing
  • Building complex workflows with control flow
  • Using local models (Transformers, llama.cpp)
  • Prefer Pythonic syntax

When to choose alternatives:

  • Instructor: Need Pydantic validation with automatic retrying
  • Outlines: Need JSON schema validation
  • LMQL: Prefer declarative query syntax

Performance Characteristics

Latency Reduction:

  • 30-50% faster than traditional prompting for constrained outputs
  • Token healing reduces unnecessary regeneration
  • Grammar constraints prevent invalid token generation

Memory Usage:

  • Minimal overhead vs unconstrained generation
  • Grammar compilation cached after first use
  • Efficient token filtering at inference time

Token Efficiency:

  • Prevents wasted tokens on invalid outputs
  • No need for retry loops
  • Direct path to valid outputs

Resources

See Also

  • references/constraints.md - Comprehensive regex and grammar patterns
  • references/backends.md - Backend-specific configuration
  • references/examples.md - Production-ready examples

來源與署名

來源：orchestra-research/ai-research-skills位於16-prompt-engineering/guidance提交773a529

授權條款： MIT

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架

更多來自 orchestra-research/ai-research-skills 的技能

Speculative Decoding

orchestra-research

指導使用推測解碼、Medusa 多頭預測與前瞻解碼來加速大型語言模型推論。

AI & Agents13K3 個月前更新

Systems Paper Writing

orchestra-research

為 OSDI、SOSP、ASPLOS、NSDI 等系統會議論文寫作提供段落層級結構與篇幅分配指引。

Writing & Content13K3 個月前更新

Moe Training

orchestra-research

指導使用 DeepSpeed 或 HuggingFace 訓練與推論混合專家模型，涵蓋路由、負載平衡與專家平行。

AI & Agents13K3 個月前更新

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

待分類13K3 個月前更新

Model Pruning

orchestra-research

指導使用 Wanda、SparseGPT、幅度剪枝與 N:M 方法壓縮 LLM，縮小模型體積並加速推論。

AI & Agents13K3 個月前更新

Outlines

orchestra-research

指導使用 Outlines 函式庫，透過本機模型進行受結構約束的文字生成。

AI & Agents13K3 個月前更新

更多AI & Agents技能

Ppt Template Creator

anthropics

精選

將使用者的 PowerPoint 範本轉換為可重用的技能，用來產生品牌簡報。

AI & Agents39K2 週前更新

Skill Development

anthropics

精選

指導建立 Claude Code 外掛技能，涵蓋結構、描述、漸進式揭露與驗證。

AI & Agents37K今天更新

Plugin Structure

anthropics

精選

說明 Claude Code 外掛的結構、資訊清單與元件配置方式。

AI & Agents37K今天更新

Command Development

anthropics

精選

指導建立 Claude Code 斜線命令，涵蓋結構、YAML frontmatter、參數與外掛功能。

AI & Agents37K今天更新

Claude Md Improver

anthropics

精選

稽核儲存庫中的 CLAUDE.md 檔案、評估其品質，並在取得核准後套用針對性的改進。

AI & Agents37K今天更新

Find Skills

vercel-labs

精選

指導透過 npx skills 命令列工具探索、評估並安裝代理技能。

AI & Agents33K今天更新