Regex Vs Llm Structured Text

affaan-m/ECC/skills/regex-vs-llm-structured-text

by affaan-mef648e01899ba3e8dc6371642deaaf64b4477775No license275K starsListed Oct 9, 2026Updated Oct 9, 2026Repository updated 4 days ago

Decision framework for parsing structured text (quizzes, forms, invoices, receipts, tables) with a hybrid regex-first pipeline — regex extraction handles 95%+ cheaply, a confidence scorer flags low-confidence items, and an LLM validator fixes only the edge cases. Use when choosing between regex and LLM for text extraction, building a cheap document parser, or optimizing extraction cost and accuracy.

Instructions onlySoftware Development AI & Agents

AI-generated overview

Decision framework and code patterns for hybrid regex-first parsing of structured text, with LLM validation only for low-confidence items.

What it does: Explains when to use regex versus an LLM for extracting structure from repeating text such as quizzes, forms, invoices and receipts. It provides a decision tree, an architecture diagram, and Python code for a regex parser, a confidence scorer, an LLM validator and a combined pipeline. It also lists production metrics, best practices and anti-patterns for cost and accuracy tradeoffs.
When to use it: Use when choosing between regex and LLM extraction for structured text, building a low-cost document parser, or reducing LLM calls while keeping extraction accuracy. It fits repeating formats like quiz questions, form fields, invoices, receipts and tables. It is not aimed at free-form, highly variable text, where the framework recommends using an LLM directly.
Requirements: No scripts ship with the skill; it is instructions only. The described pipeline uses Python with the re and dataclasses modules, and the optional LLM validation step requires an Anthropic-style client and API access to a Haiku-class model.

Regex vs LLM for Structured Text Parsing

A practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.

When to Activate

Parsing structured text with repeating patterns (questions, forms, tables)
Deciding between regex and LLM for text extraction
Building hybrid pipelines that combine both approaches
Optimizing cost/accuracy tradeoffs in text processing

Decision Framework

Is the text format consistent and repeating?├── Yes (>90% follows a pattern) → Start with Regex│   ├── Regex handles 95%+ → Done, no LLM needed│   └── Regex handles <95% → Add LLM for edge cases only└── No (free-form, highly variable) → Use LLM directly

Is the text format consistent and repeating?├── Yes (>90% follows a pattern) → Start with Regex│   ├── Regex handles 95%+ → Done, no LLM needed│   └── Regex handles <95% → Add LLM for edge cases only└── No (free-form, highly variable) → Use LLM directly

Architecture Pattern

Source Text    │    ▼[Regex Parser] ─── Extracts structure (95-98% accuracy)    │    ▼[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)    │    ▼[Confidence Scorer] ─── Flags low-confidence extractions    │    ├── High confidence (≥0.95) → Direct output    │    └── Low confidence (<0.95) → [LLM Validator] → Output

Source Text    │    ▼[Regex Parser] ─── Extracts structure (95-98% accuracy)    │    ▼[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)    │    ▼[Confidence Scorer] ─── Flags low-confidence extractions    │    ├── High confidence (≥0.95) → Direct output    │    └── Low confidence (<0.95) → [LLM Validator] → Output

Implementation

1. Regex Parser (Handles the Majority)

python

import refrom dataclasses import dataclass
@dataclass(frozen=True)class ParsedItem:    id: str    text: str    choices: tuple[str, ...]    answer: str    confidence: float = 1.0
def parse_structured_text(content: str) -> list[ParsedItem]:    """Parse structured text using regex patterns."""    pattern = re.compile(        r"(?P<id>\d+)\.\s*(?P<text>.+?)\n"        r"(?P<choices>(?:[A-D]\..+?\n)+)"        r"Answer:\s*(?P<answer>[A-D])",        re.MULTILINE | re.DOTALL,    )    items = []    for match in pattern.finditer(content):        choices = tuple(            c.strip() for c in re.findall(r"[A-D]\.\s*(.+)", match.group("choices"))        )        items.append(ParsedItem(            id=match.group("id"),            text=match.group("text").strip(),            choices=choices,            answer=match.group("answer"),        ))    return items

import refrom dataclasses import dataclass
@dataclass(frozen=True)class ParsedItem:    id: str    text: str    choices: tuple[str, ...]    answer: str    confidence: float = 1.0
def parse_structured_text(content: str) -> list[ParsedItem]:    """Parse structured text using regex patterns."""    pattern = re.compile(        r"(?P<id>\d+)\.\s*(?P<text>.+?)\n"        r"(?P<choices>(?:[A-D]\..+?\n)+)"        r"Answer:\s*(?P<answer>[A-D])",        re.MULTILINE | re.DOTALL,    )    items = []    for match in pattern.finditer(content):        choices = tuple(            c.strip() for c in re.findall(r"[A-D]\.\s*(.+)", match.group("choices"))        )        items.append(ParsedItem(            id=match.group("id"),            text=match.group("text").strip(),            choices=choices,            answer=match.group("answer"),        ))    return items

2. Confidence Scoring

Flag items that may need LLM review:

python

@dataclass(frozen=True)class ConfidenceFlag:    item_id: str    score: float    reasons: tuple[str, ...]
def score_confidence(item: ParsedItem) -> ConfidenceFlag:    """Score extraction confidence and flag issues."""    reasons = []    score = 1.0
    if len(item.choices) < 3:        reasons.append("few_choices")        score -= 0.3
    if not item.answer:        reasons.append("missing_answer")        score -= 0.5
    if len(item.text) < 10:        reasons.append("short_text")        score -= 0.2
    return ConfidenceFlag(        item_id=item.id,        score=max(0.0, score),        reasons=tuple(reasons),    )
def identify_low_confidence(    items: list[ParsedItem],    threshold: float = 0.95,) -> list[ConfidenceFlag]:    """Return items below confidence threshold."""    flags = [score_confidence(item) for item in items]    return [f for f in flags if f.score < threshold]

@dataclass(frozen=True)class ConfidenceFlag:    item_id: str    score: float    reasons: tuple[str, ...]
def score_confidence(item: ParsedItem) -> ConfidenceFlag:    """Score extraction confidence and flag issues."""    reasons = []    score = 1.0
    if len(item.choices) < 3:        reasons.append("few_choices")        score -= 0.3
    if not item.answer:        reasons.append("missing_answer")        score -= 0.5
    if len(item.text) < 10:        reasons.append("short_text")        score -= 0.2
    return ConfidenceFlag(        item_id=item.id,        score=max(0.0, score),        reasons=tuple(reasons),    )
def identify_low_confidence(    items: list[ParsedItem],    threshold: float = 0.95,) -> list[ConfidenceFlag]:    """Return items below confidence threshold."""    flags = [score_confidence(item) for item in items]    return [f for f in flags if f.score < threshold]

3. LLM Validator (Edge Cases Only)

python

def validate_with_llm(    item: ParsedItem,    original_text: str,    client,) -> ParsedItem:    """Use LLM to fix low-confidence extractions."""    response = client.messages.create(        model="claude-haiku-4-5-20251001",  # Cheapest model for validation        max_tokens=500,        messages=[{            "role": "user",            "content": (                f"Extract the question, choices, and answer from this text.\n\n"                f"Text: {original_text}\n\n"                f"Current extraction: {item}\n\n"                f"Return corrected JSON if needed, or 'CORRECT' if accurate."            ),        }],    )    # Parse LLM response and return corrected item...    return corrected_item

def validate_with_llm(    item: ParsedItem,    original_text: str,    client,) -> ParsedItem:    """Use LLM to fix low-confidence extractions."""    response = client.messages.create(        model="claude-haiku-4-5-20251001",  # Cheapest model for validation        max_tokens=500,        messages=[{            "role": "user",            "content": (                f"Extract the question, choices, and answer from this text.\n\n"                f"Text: {original_text}\n\n"                f"Current extraction: {item}\n\n"                f"Return corrected JSON if needed, or 'CORRECT' if accurate."            ),        }],    )    # Parse LLM response and return corrected item...    return corrected_item

4. Hybrid Pipeline

python

def process_document(    content: str,    *,    llm_client=None,    confidence_threshold: float = 0.95,) -> list[ParsedItem]:    """Full pipeline: regex -> confidence check -> LLM for edge cases."""    # Step 1: Regex extraction (handles 95-98%)    items = parse_structured_text(content)
    # Step 2: Confidence scoring    low_confidence = identify_low_confidence(items, confidence_threshold)
    if not low_confidence or llm_client is None:        return items
    # Step 3: LLM validation (only for flagged items)    low_conf_ids = {f.item_id for f in low_confidence}    result = []    for item in items:        if item.id in low_conf_ids:            result.append(validate_with_llm(item, content, llm_client))        else:            result.append(item)
    return result

def process_document(    content: str,    *,    llm_client=None,    confidence_threshold: float = 0.95,) -> list[ParsedItem]:    """Full pipeline: regex -> confidence check -> LLM for edge cases."""    # Step 1: Regex extraction (handles 95-98%)    items = parse_structured_text(content)
    # Step 2: Confidence scoring    low_confidence = identify_low_confidence(items, confidence_threshold)
    if not low_confidence or llm_client is None:        return items
    # Step 3: LLM validation (only for flagged items)    low_conf_ids = {f.item_id for f in low_confidence}    result = []    for item in items:        if item.id in low_conf_ids:            result.append(validate_with_llm(item, content, llm_client))        else:            result.append(item)
    return result

Real-World Metrics

From a production quiz parsing pipeline (410 items):

Best Practices

Start with regex — even imperfect regex gives you a baseline to improve
Use confidence scoring to programmatically identify what needs LLM help
Use the cheapest LLM for validation (Haiku-class models are sufficient)
Never mutate parsed items — return new instances from cleaning/validation steps
TDD works well for parsers — write tests for known patterns first, then edge cases
Log metrics (regex success rate, LLM call count) to track pipeline health

Anti-Patterns to Avoid

Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)
Using regex for free-form, highly variable text (LLM is better here)
Skipping confidence scoring and hoping regex "just works"
Mutating parsed objects during cleaning/validation steps
Not testing edge cases (malformed input, missing fields, encoding issues)

When to Use

Quiz/exam question parsing
Form data extraction
Invoice/receipt processing
Document structure parsing (headers, sections, tables)
Any structured text with repeating patterns where cost matters

Source and attribution

Source:affaan-m/ECCinskills/regex-vs-llm-structured-textat commitef648e0

License: No license

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

Metric	Value
Regex success rate	98.0%
Low confidence items	8 (2.0%)
LLM calls needed	~5
Cost savings vs all-LLM	~95%
Test coverage	93%

Regex Vs Llm Structured Text

Regex vs LLM for Structured Text Parsing

When to Activate

Decision Framework

Architecture Pattern

Implementation

1. Regex Parser (Handles the Majority)

2. Confidence Scoring

3. LLM Validator (Edge Cases Only)

4. Hybrid Pipeline

Real-World Metrics

Best Practices

Anti-Patterns to Avoid

When to Use

Source and attribution

More from affaan-m/ECC

Unified Notifications Ops

Verification Loop

Ui To Vue

Unified Memory

Visa Doc Translate

Team Builder

More in Software Development

Playground

M5 Onboard

Cardputer Buddy

Web Design Guidelines

Vercel React Native Skills

Vercel React Best Practices