Grepai Chunking

GrepAI Chunking Configuration

This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase.

When to Use This Skill

• Optimizing search accuracy
• Adjusting for code style (verbose vs. concise)
• Troubleshooting search results
• Understanding how indexing works

What is Chunking?

Chunking is the process of splitting source files into smaller segments for embedding:

┌─────────────────────────────────────┐│         Large Source File           ││         (1000+ tokens)              │└─────────────────────────────────────┘                  ↓┌─────────┐ ┌─────────┐ ┌─────────┐│ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 ││ ~512    │ │ ~512    │ │ ~512    ││ tokens  │ │ tokens  │ │ tokens  │└─────────┘ └─────────┘ └─────────┘                  ↓          Each chunk gets          its own embedding

Why Chunking Matters

Embedding models have optimal input sizes:

• Too large chunks: Less precise search results
• Too small chunks: Lost context, fragmented results
• Just right: Good balance of precision and context

Configuration

Basic Settings

# .grepai/config.yamlchunking:  size: 512      # Tokens per chunk  overlap: 50    # Overlap between chunks

Understanding Parameters

Chunk Size

The target number of tokens per chunk.

Overlap

Tokens shared between adjacent chunks. Preserves context at boundaries.

Visualization

With size=512 and overlap=50:

File: auth.go (1000 tokens)
Chunk 1: tokens 1-512         ┌────────────────────────────────────┐         │ func Login(user, pass)...          │         └────────────────────────────────────┘                                    ↘                              50 token overlap                                    ↙Chunk 2: tokens 463-974         ┌────────────────────────────────────┐         │ ...validate credentials...         │         └────────────────────────────────────┘                                    ↘                              50 token overlap                                    ↙Chunk 3: tokens 925-1000         ┌──────────────┐         │ ...return    │         └──────────────┘

Recommended Settings by Language

Verbose Languages (Java, C#)

chunking:  size: 768    # Larger to capture full methods  overlap: 75

Concise Languages (Go, Python)

chunking:  size: 512    # Standard size  overlap: 50

Very Concise (Rust, Zig)

chunking:  size: 384    # Smaller for precise results  overlap: 40

Recommended Settings by Codebase

Small Functions (Microservices)

chunking:  size: 384    # Capture individual functions  overlap: 40

Large Classes (Monolith)

chunking:  size: 768    # Capture more context  overlap: 100

Mixed Codebase

chunking:  size: 512    # Balanced default  overlap: 50

How Tokens are Counted

GrepAI uses approximate token counting:

• ~4 characters = 1 token (for English text)
• Code varies based on identifiers and syntax

Example:

func calculateTotal(items []Item) float64 {    total := 0.0    for _, item := range items {        total += item.Price * float64(item.Quantity)    }    return total}

≈ 45 tokens

Impact on Index Size

Larger overlap = more chunks = larger index:

Impact on Search Quality

Too Small Chunks (size: 128)

Query: "authentication middleware"
Result: "...c.AbortWithStatus(401)..."        (Fragment, missing context)

Just Right (size: 512)

Query: "authentication middleware"
Result: "func AuthMiddleware() gin.HandlerFunc {            return func(c *gin.Context) {                token := c.GetHeader("Authorization")                if token == "" {                    c.AbortWithStatus(401)                    return                }                // validate token...            }        }"        (Complete function with context)

Too Large Chunks (size: 2048)

Query: "authentication middleware"
Result: "// Multiple unrelated functions...        func AuthMiddleware()... (your match)        func LoggingMiddleware()...        func CORSMiddleware()..."        (Too much noise)

Experimentation

Testing Different Settings

1. Try smaller chunks for more precise results:

chunking:  size: 384  overlap: 40

2. Re-index:

rm .grepai/index.gobgrepai watch

3. Test with searches:

grepai search "your query"

4. Adjust and repeat until satisfied.

Comparing Results

Before changing settings, save a search result:

grepai search "authentication" > before.txt

After changing settings and re-indexing:

grepai search "authentication" > after.txtdiff before.txt after.txt

Chunk Boundaries

GrepAI tries to split at logical boundaries:

1. Empty lines (function/class boundaries)
2. Closing braces
3. Statement ends

This means actual chunk sizes may vary slightly from the target.

Best Practices

1. Start with defaults: 512/50 works well for most codebases
2. Adjust based on code style: Verbose = larger, concise = smaller
3. Test with real queries: See what your searches return
4. Re-index after changes: Must regenerate embeddings
5. Consider overlap: Don't set to 0 unless index size is critical

Common Issues

❌ Problem: Search results are too fragmented ✅ Solution: Increase chunk size:

chunking:  size: 768

❌ Problem: Search results have too much irrelevant context ✅ Solution: Decrease chunk size:

chunking:  size: 384

❌ Problem: Results miss related code at function boundaries ✅ Solution: Increase overlap:

chunking:  overlap: 100

❌ Problem: Index is too large ✅ Solutions:

• Decrease overlap
• Increase chunk size
• Add more ignore patterns

Output Format

Chunking status:

✅ Chunking Configuration
   Size: 512 tokens   Overlap: 50 tokens
   Index Statistics:   - Total files: 245   - Total chunks: 1,234   - Avg chunks/file: 5.0   - Avg chunk size: 478 tokens
   Recommendations:   - Current settings are balanced   - Consider size: 384 for more precise results   - Consider size: 768 for more context