Altimate Data Engineering Skills

Altimate Data Engineering Skills

Skill by ara.so — Data Skills collection.

Altimate Data Engineering Skills is a collection of Claude Code skills that encode the workflows and best practices of experienced analytics engineers. These skills transform Claude from a code generator into a capable data engineering assistant by teaching how to approach tasks, not just what syntax to use.

The project demonstrates 53% accuracy on ADE-bench (43 real-world dbt tasks), 3x improvement on model creation tasks, and 84% pass rate on Snowflake query optimization.

What This Project Does

Data Engineering Skills provides:

• 7 dbt skills: Model creation, debugging, testing, documentation, migration, refactoring, incremental models
• 3 Snowflake skills: Cost analysis, query optimization by ID, query optimization by text
• 1 delegation skill: Hand off complex tasks to altimate-code CLI tool
• Workflow automation: Skills trigger automatically based on user intent
• Best practices: Encoded patterns from experienced analytics engineers

Skills are markdown files with YAML frontmatter that define trigger conditions and step-by-step workflows.

Installation

Installing Skills in Claude Code

# Add the marketplace plugin/plugin marketplace add AltimateAI/data-engineering-skills
# Install all skills/plugin install dbt-skills@data-engineering-skills/plugin install snowflake-skills@data-engineering-skills/plugin install altimate-code@data-engineering-skills

Installing Kits

Kits bundle skills, MCP servers, and instructions:

# Install altimate-code CLI (required for kits)npm install -g altimate-code
# Install kit systemaltimate-code kit install AltimateAI/data-engineering-skills
# Activate the dbt-snowflake kitaltimate-code kit activate dbt-snowflake
# Check statusaltimate-code kit status

Manual Installation

Clone and reference skills directly:

git clone https://github.com/AltimateAI/data-engineering-skills.gitcd data-engineering-skills

Available Skills

dbt Skills

creating-dbt-models: Creates new dbt models following project conventions

• Discovers existing patterns before writing
• Runs dbt build after creation (not just compile)
• Verifies output with dbt show
• Handles staging, intermediate, and mart models

debugging-dbt-errors: Troubleshoots dbt compilation and runtime errors

• Reads full error messages carefully
• Checks upstream dependencies
• Applies fixes and rebuilds
• Stops after 3 failed attempts to reassess

testing-dbt-models: Adds schema tests to models

• Studies existing test patterns in project
• Matches project testing style
• Covers uniqueness, not_null, relationships, accepted_values

documenting-dbt-models: Generates dbt model documentation

• Analyzes model logic and columns
• Creates descriptions for models and fields
• Follows project documentation patterns

migrating-sql-to-dbt: Converts legacy SQL to dbt models

• Parses raw SQL queries
• Creates proper dbt model structure
• Handles CTEs, refs, and sources

refactoring-dbt-models: Safely restructures dbt models

• Tracks all dependencies before changes
• Applies refactoring
• Verifies downstream models still work

developing-incremental-models: Creates incremental dbt models

• Selects appropriate strategy (append, merge, delete+insert)
• Designs proper unique_key
• Handles late-arriving data and edge cases

Snowflake Skills

finding-expensive-queries: Identifies costly Snowflake queries

• Finds queries by cost, time, or data scanned
• Ranks by impact
• Provides query IDs for optimization

optimizing-query-by-id: Optimizes using Snowflake query history ID

• Retrieves query profile from ID
• Applies optimization patterns
• Validates semantic preservation

optimizing-query-text: Optimizes raw SQL query text

• Profiles query execution
• Identifies bottlenecks (scans, joins, aggregations)
• Applies anti-pattern fixes
• Tests performance improvement

Delegation Skill

altimate-code: Delegates complex data tasks to altimate-code CLI

• Verifies altimate-code installation
• Invokes altimate-code run --yolo non-interactively
• Reads output file and summarizes results
• Requires: npm install -g altimate-code (Node 20+)

Skill Structure

Skills are markdown files with YAML frontmatter:

---name: creating-dbt-modelsdescription: |  Guide for creating dbt models. ALWAYS use this skill when:  (1) Creating ANY new model (staging, intermediate, mart)  (2) Task mentions "create", "build", "add" with model/table  (3) Modifying model logic or columns---

Followed by workflow instructions:

# dbt Model Development
**Read before you write. Build after you write. Verify your output.**
## Critical Rules1. ALWAYS run `dbt build` after creating models - compile is NOT enough2. ALWAYS verify output after build using `dbt show`3. If build fails 3+ times, stop and reassess your approach
## Workflow1. Discover Conventions   - Check existing models in same layer (staging/intermediate/mart)   - Note naming patterns, CTE style, column ordering   2. Write Model   - Follow discovered patterns   - Use proper refs and sources   3. Build and Verify   ```bash   dbt build --select model_name   dbt show --select model_name --limit 10

## Usage Examples
### Creating a dbt Model
```yaml# User request:"Create a staging model for raw customers data"
# Skill triggers: creating-dbt-models# AI workflow:# 1. Checks models/staging/ for naming patterns# 2. Creates models/staging/stg_customers.sql# 3. Runs: dbt build --select stg_customers# 4. Verifies: dbt show --select stg_customers

Example model created:

-- models/staging/stg_customers.sqlwith source as (    select * from {{ source('jaffle_shop', 'customers') }}),
renamed as (    select        id as customer_id,        first_name,        last_name,        email,        created_at    from source)
select * from renamed

Debugging dbt Errors

# User request:"Fix this error: Compilation Error in model customers (models/customers.sql)"
# Skill triggers: debugging-dbt-errors# AI workflow:# 1. Reads full error message# 2. Checks upstream model dependencies# 3. Identifies issue (e.g., missing ref)# 4. Applies fix# 5. Rebuilds: dbt build --select customers

Optimizing Snowflake Query

# User request:"This query is taking 5 minutes, can you optimize it?"
# Skill triggers: optimizing-query-text# AI workflow:# 1. Profiles query (EXPLAIN, execution stats)# 2. Identifies bottlenecks (table scans, inefficient joins)# 3. Applies optimizations (clustering, filters, join order)# 4. Tests and measures improvement

Example optimization:

-- Before (slow)SELECT     o.order_id,    c.customer_name,    SUM(oi.amount) as totalFROM orders oJOIN customers c ON o.customer_id = c.customer_idJOIN order_items oi ON o.order_id = oi.order_idWHERE o.order_date >= '2024-01-01'GROUP BY 1, 2;
-- After (optimized)SELECT     o.order_id,    c.customer_name,    SUM(oi.amount) as totalFROM orders oINNER JOIN order_items oi ON o.order_id = oi.order_idLEFT JOIN customers c ON o.customer_id = c.customer_idWHERE o.order_date >= '2024-01-01'  AND o.order_date < '2024-12-31'  -- Added upper bound for partition pruningGROUP BY 1, 2;-- Assumes orders table is clustered by order_date

Using Kits

# Activate dbt-snowflake kit for a projectcd /path/to/dbt-projectaltimate-code kit activate dbt-snowflake
# This configures:# - All dbt and Snowflake skills# - MCP server for dbt (live project access)# - Project-specific instructions
# Deactivate when donealtimate-code kit deactivate

Kit configuration (.altimate/kits/active.yaml):

kit: dbt-snowflakeversion: 1.0.0skills:  - creating-dbt-models  - debugging-dbt-errors  - testing-dbt-models  - documenting-dbt-models  - migrating-sql-to-dbt  - refactoring-dbt-models  - developing-incremental-models  - finding-expensive-queries  - optimizing-query-by-id  - optimizing-query-textmcp_servers:  - dbtinstructions: |  You are working on a dbt + Snowflake project.  Always check project conventions before creating models.  Run dbt build (not compile) after creating models.

Configuration

Skill Configuration

Skills are auto-triggered based on user requests. No explicit configuration needed.

MCP Integration

Skills work best with Altimate MCP server for live project access:

// claude_desktop_config.json or similar{  "mcpServers": {    "altimate-dbt": {      "command": "npx",      "args": ["-y", "@altimate/mcp-server-dbt"],      "env": {        "DBT_PROJECT_DIR": "/path/to/dbt/project",        "DBT_PROFILES_DIR": "/path/to/.dbt"      }    }  }}

MCP tools available to skills:

• dbt_project_info: Project structure, model list, sources
• dbt_model_details: Column types, dependencies, compiled SQL
• dbt_compile: Compile models without CLI
• snowflake_query_history: Recent query executions and stats
• snowflake_table_stats: Row counts, clustering info

Common Patterns

Pattern 1: Multi-Step Model Creation

# User: "Create a mart model for monthly revenue by customer"# Skills auto-chains:# 1. creating-dbt-models: Creates initial model# 2. testing-dbt-models: Adds schema tests# 3. documenting-dbt-models: Adds documentation

Pattern 2: Debug-Fix-Verify Loop

# User: "My model won't compile"# Skill: debugging-dbt-errors# Workflow:# 1. Read error# 2. Check dependencies# 3. Apply fix# 4. Rebuild# 5. If still fails, repeat up to 3x# 6. If 3x fails, stop and reassess approach

Pattern 3: Safe Refactoring

# User: "Break this large model into smaller ones"# Skill: refactoring-dbt-models# Workflow:# 1. Map all downstream dependencies# 2. Create new intermediate models# 3. Update refs in downstream models# 4. Run dbt build on entire DAG# 5. Verify no breakage

Pattern 4: Incremental Model Development

-- Pattern: merge strategy with unique_key{{ config(    materialized='incremental',    unique_key='event_id',    merge_update_columns=['status', 'updated_at']) }}
select    event_id,    user_id,    event_type,    status,    created_at,    updated_atfrom {{ source('events', 'raw_events') }}{% if is_incremental() %}    where updated_at > (select max(updated_at) from {{ this }}){% endif %}

Pattern 5: Query Optimization Workflow

# Skill: optimizing-query-text# Steps encoded:# 1. Profile query# 2. Identify bottleneck type:#    - Full table scan → Add filters/clustering#    - Inefficient join → Reorder, change type#    - Large aggregation → Pre-aggregate or partition# 3. Apply pattern-based fix# 4. Validate semantics unchanged# 5. Measure improvement

Troubleshooting

Skills Not Triggering

Problem: Skills don't activate for your request

Solution:

• Be explicit: "Create a dbt model" not "make a model"
• Check skill installation: /plugin list
• Review trigger phrases in skill YAML
• Use exact trigger phrases from skill description

dbt Build Failures

Problem: Models fail to build after creation

Solution:

• Skill will auto-retry up to 3 times
• After 3 failures, skill stops to reassess
• Check dbt debug for connection issues
• Verify refs and sources exist
• Check for SQL syntax errors

MCP Server Not Connected

Problem: Skills can't access live project data

Solution:

# Check MCP server configcat claude_desktop_config.json | grep altimate
# Verify dbt project pathexport DBT_PROJECT_DIR=/correct/pathexport DBT_PROFILES_DIR=/correct/.dbt/path
# Restart Claude Code

altimate-code Delegation Fails

Problem: "altimate-code not found" error

Solution:

# Install altimate-code CLInpm install -g altimate-code
# Verify installationaltimate-code --version
# Check Node version (requires 20+)node --version

Query Optimization No Improvement

Problem: Optimized query performs the same

Solution:

• Skill checks for anti-patterns but can't fix all issues
• May need manual clustering/indexing setup
• Check if bottleneck is data volume (consider aggregation)
• Review Snowflake warehouse size
• Use finding-expensive-queries to compare before/after

Kit Activation Issues

Problem: Kit won't activate

Solution:

# Check kit is installedaltimate-code kit list
# Reinstall if neededaltimate-code kit install AltimateAI/data-engineering-skills
# Check for conflicting active kitsaltimate-code kit status
# Deactivate others firstaltimate-code kit deactivate

Performance

Benchmark results (ADE-bench, 43 real-world dbt tasks):

• Baseline Claude: 46.5% accuracy (20/43 tasks)
• Claude + Skills: 53.5% accuracy (23/43 tasks)
• Model Creation: 3x improvement (40% → 65%)

Snowflake optimization (TPC-H SF1000, 62 queries):

• Baseline: 77.4% pass rate, 4.7% avg improvement
• With Skills: 83.9% pass rate, 16.8% avg improvement (3.6x better)

Additional Resources

• Documentation: https://docs.myaltimate.com/
• ADE-bench Framework: https://github.com/dbt-labs/ade-bench
• altimate-code CLI: https://github.com/AltimateAI/altimate-code
• dbt Slack: https://getdbt.slack.com/archives/C05KPDGRMDW
• Contact: https://app.myaltimate.com/contactus