Model Merging

orchestra-research/ai-research-skills/19-emerging-techniques/model-merging

by orchestra-research773a52944ba4MIT13K starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated 3 months ago

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

Instructions onlyAI & Agents

AI-generated overview

Guides merging fine-tuned language models with mergekit to combine capabilities without retraining.

What it does: This skill provides instructions and YAML configuration patterns for merging fine-tuned language models using mergekit. It covers merge methods such as linear, SLERP, task arithmetic, TIES-Merging, DARE, layer-wise passthrough, and MoE construction, along with weight and density tuning guidance. It also describes evaluation, quantization, and deployment steps for merged models. Reference files cover methods, examples, evaluation, and coefficient tuning.
When to use it: Use it when combining multiple fine-tuned models into one model without retraining, blending domain expertise such as math, coding, and chat. It also fits rapid experimentation with model variants and improving benchmark performance beyond a single model.
Requirements: Requires mergekit, transformers, and torch, plus access to model weights and sufficient compute or memory; merges can run on CPU. It ships no scripts, only instructions and reference documents.

Model Merging: Combining Pre-trained Models

When to Use This Skill

Use Model Merging when you need to:

Combine capabilities from multiple fine-tuned models without retraining
Create specialized models by blending domain-specific expertise (math + coding + chat)
Improve performance beyond single models (often +5-10% on benchmarks)
Reduce training costs - no GPUs needed, merges run on CPU
Experiment rapidly - create new model variants in minutes, not days
Preserve multiple skills - merge without catastrophic forgetting

Success Stories: Marcoro14-7B-slerp (best on Open LLM Leaderboard 02/2024), many top HuggingFace models use merging

Tools: mergekit (Arcee AI), LazyMergekit, Model Soup

Installation

bash

# Install mergekitgit clone https://github.com/arcee-ai/mergekit.gitcd mergekitpip install -e .
# Or via pippip install mergekit
# Optional: Transformer librarypip install transformers torch

# Install mergekitgit clone https://github.com/arcee-ai/mergekit.gitcd mergekitpip install -e .
# Or via pippip install mergekit
# Optional: Transformer librarypip install transformers torch

Quick Start

Simple Linear Merge

yaml

# config.yml - Merge two models with equal weightsmerge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.5dtype: bfloat16

# config.yml - Merge two models with equal weightsmerge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.5dtype: bfloat16

bash

# Run mergemergekit-yaml config.yml ./merged-model --cuda
# Use merged modelpython -m transformers.models.auto --model_name_or_path ./merged-model

# Run mergemergekit-yaml config.yml ./merged-model --cuda
# Use merged modelpython -m transformers.models.auto --model_name_or_path ./merged-model

SLERP Merge (Best for 2 Models)

yaml

# config.yml - Spherical interpolationmerge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # Interpolation factor (0=model1, 1=model2)dtype: bfloat16

# config.yml - Spherical interpolationmerge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # Interpolation factor (0=model1, 1=model2)dtype: bfloat16

Core Concepts

1. Merge Methods

Linear (Model Soup)

Simple weighted average of parameters
Fast, works well for similar models
Can merge 2+ models (w1 + w2 + ... = 1)

SLERP (Spherical Linear Interpolation)

Interpolates along sphere in weight space
Preserves magnitude of weight vectors
Best for merging 2 models
Smoother than linear

python

# SLERP formulamerged = (sin((1-t)*θ) / sin(θ)) * model1 + (sin(t*θ) / sin(θ)) * model2# where θ = arccos(dot(model1, model2))# t ∈ [0, 1]

# SLERP formulamerged = (sin((1-t)*θ) / sin(θ)) * model1 + (sin(t*θ) / sin(θ)) * model2# where θ = arccos(dot(model1, model2))# t ∈ [0, 1]

Task Arithmetic

Extract "task vectors" (fine-tuned - base)
Combine task vectors, add to base
Good for merging multiple specialized models (merged = base + α₁·tv₁ + α₂·tv₂)

TIES-Merging

Task arithmetic + sparsification
Resolves sign conflicts in parameters
Best for merging many task-specific models

DARE (Drop And REscale)

Randomly drops fine-tuned parameters
Rescales remaining parameters
Reduces redundancy, maintains performance

2. Configuration Structure

yaml

# Basic structuremerge_method: <method>  # linear, slerp, ties, dare_ties, task_arithmeticbase_model: <path>      # Optional: base model for task arithmetic
models:  - model: <path/to/model1>    parameters:      weight: <float>   # Merge weight      density: <float>  # For TIES/DARE
  - model: <path/to/model2>    parameters:      weight: <float>
parameters:  # Method-specific parameters
dtype: <dtype>  # bfloat16, float16, float32
# Optionalslices:  # Layer-wise mergingtokenizer:  # Tokenizer configuration

# Basic structuremerge_method: <method>  # linear, slerp, ties, dare_ties, task_arithmeticbase_model: <path>      # Optional: base model for task arithmetic
models:  - model: <path/to/model1>    parameters:      weight: <float>   # Merge weight      density: <float>  # For TIES/DARE
  - model: <path/to/model2>    parameters:      weight: <float>
parameters:  # Method-specific parameters
dtype: <dtype>  # bfloat16, float16, float32
# Optionalslices:  # Layer-wise mergingtokenizer:  # Tokenizer configuration

Merge Methods Guide

Linear Merge

Best for: Simple model combinations, equal weighting

yaml

merge_method: linearmodels:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      weight: 0.4  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.3  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      weight: 0.3dtype: bfloat16

merge_method: linearmodels:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      weight: 0.4  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.3  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      weight: 0.3dtype: bfloat16

SLERP Merge

Best for: Two models, smooth interpolation

yaml

merge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # 0.0 = first model, 1.0 = second modeldtype: bfloat16

merge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # 0.0 = first model, 1.0 = second modeldtype: bfloat16

Layer-specific SLERP:

yaml

merge_method: slerpslices:  - sources:      - model: model_a        layer_range: [0, 32]      - model: model_b        layer_range: [0, 32]parameters:  t:    - filter: self_attn    # Attention layers      value: 0.3    - filter: mlp          # MLP layers      value: 0.7    - value: 0.5           # Default for other layersdtype: bfloat16

merge_method: slerpslices:  - sources:      - model: model_a        layer_range: [0, 32]      - model: model_b        layer_range: [0, 32]parameters:  t:    - filter: self_attn    # Attention layers      value: 0.3    - filter: mlp          # MLP layers      value: 0.7    - value: 0.5           # Default for other layersdtype: bfloat16

Task Arithmetic

Best for: Combining specialized skills

yaml

merge_method: task_arithmeticbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1  # Math    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B  # Chat    parameters:      weight: 0.3  - model: ajibawa-2023/Code-Mistral-7B  # Code    parameters:      weight: 0.2dtype: bfloat16

merge_method: task_arithmeticbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1  # Math    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B  # Chat    parameters:      weight: 0.3  - model: ajibawa-2023/Code-Mistral-7B  # Code    parameters:      weight: 0.2dtype: bfloat16

TIES-Merging

Best for: Many models, resolving conflicts

yaml

merge_method: tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5  # Keep top 50% of parameters      weight: 1.0  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 1.0  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      density: 0.5      weight: 1.0parameters:  normalize: truedtype: bfloat16

merge_method: tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5  # Keep top 50% of parameters      weight: 1.0  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 1.0  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      density: 0.5      weight: 1.0parameters:  normalize: truedtype: bfloat16

DARE Merge

Best for: Reducing redundancy

yaml

merge_method: dare_tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5    # Drop 50% of deltas      weight: 0.6  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 0.4parameters:  int8_mask: true  # Use int8 for masks (saves memory)dtype: bfloat16

merge_method: dare_tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5    # Drop 50% of deltas      weight: 0.6  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 0.4parameters:  int8_mask: true  # Use int8 for masks (saves memory)dtype: bfloat16

Advanced Patterns

Layer-wise Merging

yaml

# Different models for different layersmerge_method: passthroughslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 16]   # First half  - sources:      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [16, 32]  # Second halfdtype: bfloat16

# Different models for different layersmerge_method: passthroughslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 16]   # First half  - sources:      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [16, 32]  # Second halfdtype: bfloat16

MoE from Merged Models

yaml

# Create Mixture of Expertsmerge_method: moebase_model: mistralai/Mistral-7B-v0.1experts:  - source_model: WizardLM/WizardMath-7B-V1.1    positive_prompts:      - "math"      - "calculate"  - source_model: teknium/OpenHermes-2.5-Mistral-7B    positive_prompts:      - "chat"      - "conversation"  - source_model: ajibawa-2023/Code-Mistral-7B    positive_prompts:      - "code"      - "python"dtype: bfloat16

# Create Mixture of Expertsmerge_method: moebase_model: mistralai/Mistral-7B-v0.1experts:  - source_model: WizardLM/WizardMath-7B-V1.1    positive_prompts:      - "math"      - "calculate"  - source_model: teknium/OpenHermes-2.5-Mistral-7B    positive_prompts:      - "chat"      - "conversation"  - source_model: ajibawa-2023/Code-Mistral-7B    positive_prompts:      - "code"      - "python"dtype: bfloat16

Tokenizer Merging

yaml

merge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1  - model: custom/specialized-model
tokenizer:  source: "union"  # Combine vocabularies from both models  tokens:    <|special_token|>:      source: "custom/specialized-model"

merge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1  - model: custom/specialized-model
tokenizer:  source: "union"  # Combine vocabularies from both models  tokens:    <|special_token|>:      source: "custom/specialized-model"

Best Practices

1. Model Compatibility

python

# ✅ Good: Same architecturemodels = [    "mistralai/Mistral-7B-v0.1",    "teknium/OpenHermes-2.5-Mistral-7B",  # Both Mistral 7B]
# ❌ Bad: Different architecturesmodels = [    "meta-llama/Llama-2-7b-hf",  # Llama    "mistralai/Mistral-7B-v0.1",  # Mistral (incompatible!)]

# ✅ Good: Same architecturemodels = [    "mistralai/Mistral-7B-v0.1",    "teknium/OpenHermes-2.5-Mistral-7B",  # Both Mistral 7B]
# ❌ Bad: Different architecturesmodels = [    "meta-llama/Llama-2-7b-hf",  # Llama    "mistralai/Mistral-7B-v0.1",  # Mistral (incompatible!)]

2. Weight Selection

yaml

# ✅ Good: Weights sum to 1.0models:  - model: model_a    parameters:      weight: 0.6  - model: model_b    parameters:      weight: 0.4  # 0.6 + 0.4 = 1.0
# ⚠️  Acceptable: Weights don't sum to 1 (for task arithmetic)models:  - model: model_a    parameters:      weight: 0.8  - model: model_b    parameters:      weight: 0.8  # May boost performance

# ✅ Good: Weights sum to 1.0models:  - model: model_a    parameters:      weight: 0.6  - model: model_b    parameters:      weight: 0.4  # 0.6 + 0.4 = 1.0
# ⚠️  Acceptable: Weights don't sum to 1 (for task arithmetic)models:  - model: model_a    parameters:      weight: 0.8  - model: model_b    parameters:      weight: 0.8  # May boost performance

Unsupervised Coefficient Tuning (no labeled data needed)

Instead of manual search, use generation consistency: merge with several candidate coefficients, generate responses on a small unlabeled subset, and pick the coefficient whose outputs are most similar to those of its neighbors. Consistent outputs signal a stable, well-performing merge region (AdaMMS, arXiv:2503.23733).

python

# Pseudocode — see references/coefficient-tuning.md for full implementationcandidates = [0.3, 0.4, 0.5, 0.6, 0.7]for alpha in candidates:    merged_paths[alpha] = merge_with_coefficient(alpha, model_a, model_b)    responses[alpha]    = generate_responses(merged_paths[alpha], eval_prompts)
# Score each alpha by similarity to its neighbors (alpha ± 0.1)best_alpha = max(candidates, key=lambda a: generation_consistency(a, responses))

# Pseudocode — see references/coefficient-tuning.md for full implementationcandidates = [0.3, 0.4, 0.5, 0.6, 0.7]for alpha in candidates:    merged_paths[alpha] = merge_with_coefficient(alpha, model_a, model_b)    responses[alpha]    = generate_responses(merged_paths[alpha], eval_prompts)
# Score each alpha by similarity to its neighbors (alpha ± 0.1)best_alpha = max(candidates, key=lambda a: generation_consistency(a, responses))

See references/coefficient-tuning.md [blocked] for the full algorithm, similarity metrics, multi-coefficient search, and end-to-end pipeline.

3. Method Selection

python

# Choose merge method based on use case:
# 2 models, smooth blend → SLERPmerge_method = "slerp"
# 3+ models, simple average → Linearmerge_method = "linear"
# Multiple task-specific models → Task Arithmetic or TIESmerge_method = "ties"
# Want to reduce redundancy → DAREmerge_method = "dare_ties"

# Choose merge method based on use case:
# 2 models, smooth blend → SLERPmerge_method = "slerp"
# 3+ models, simple average → Linearmerge_method = "linear"
# Multiple task-specific models → Task Arithmetic or TIESmerge_method = "ties"
# Want to reduce redundancy → DAREmerge_method = "dare_ties"

4. Density Tuning (TIES/DARE)

yaml

# Start conservative (keep more parameters)parameters:  density: 0.8  # Keep 80%
# If performance good, increase sparsityparameters:  density: 0.5  # Keep 50%
# If performance degrades, reduce sparsityparameters:  density: 0.9  # Keep 90%

# Start conservative (keep more parameters)parameters:  density: 0.8  # Keep 80%
# If performance good, increase sparsityparameters:  density: 0.5  # Keep 50%
# If performance degrades, reduce sparsityparameters:  density: 0.9  # Keep 90%

5. Layer-specific Merging

Preserve the base model's first/last layers (often best left untouched) and merge only the middle via merge_method: passthrough with slices — see the Layer-wise Merging pattern above.

Evaluation & Testing

Benchmark Merged Models

python

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Test on various taskstest_prompts = {    "math": "Calculate: 25 * 17 =",    "code": "Write a Python function to reverse a string:",    "chat": "What is the capital of France?",}
for task, prompt in test_prompts.items():    inputs = tokenizer(prompt, return_tensors="pt")    outputs = model.generate(**inputs, max_length=100)    print(f"{task}: {tokenizer.decode(outputs[0])}")

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Test on various taskstest_prompts = {    "math": "Calculate: 25 * 17 =",    "code": "Write a Python function to reverse a string:",    "chat": "What is the capital of France?",}
for task, prompt in test_prompts.items():    inputs = tokenizer(prompt, return_tensors="pt")    outputs = model.generate(**inputs, max_length=100)    print(f"{task}: {tokenizer.decode(outputs[0])}")

Common Benchmarks

Open LLM Leaderboard: General capabilities
MT-Bench: Multi-turn conversation
MMLU: Multitask accuracy
HumanEval: Code generation
GSM8K: Math reasoning

Production Deployment

Save and Upload

python

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Upload to HuggingFace Hubmodel.push_to_hub("username/my-merged-model")tokenizer.push_to_hub("username/my-merged-model")

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Upload to HuggingFace Hubmodel.push_to_hub("username/my-merged-model")tokenizer.push_to_hub("username/my-merged-model")

Quantize Merged Model

bash

# Quantize with GGUFpython convert.py ./merged-model --outtype f16 --outfile merged-model.gguf
# Quantize with GPTQpython quantize_gptq.py ./merged-model --bits 4 --group_size 128

# Quantize with GGUFpython convert.py ./merged-model --outtype f16 --outfile merged-model.gguf
# Quantize with GPTQpython quantize_gptq.py ./merged-model --bits 4 --group_size 128

Common Pitfalls

Mismatched architectures — only merge models that share the same architecture (e.g., don't mix Llama and Mistral).
Over-weighting one model (e.g., 0.95 / 0.05) — keep weights balanced, typically in the 0.3–0.7 range.
Skipping evaluation — always benchmark a merged model before deploying (see the Evaluation & Testing section above).

Resources

mergekit GitHub: https://github.com/arcee-ai/mergekit
HuggingFace Tutorial: https://huggingface.co/blog/mlabonne/merge-models
LazyMergekit: Automated merging notebook
TIES Paper: https://arxiv.org/abs/2306.01708
DARE Paper: https://arxiv.org/abs/2311.03099

Source and attribution

Source:orchestra-research/ai-research-skillsin19-emerging-techniques/model-mergingat commit773a529

License: MIT

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

More from orchestra-research/ai-research-skills

Speculative Decoding

orchestra-research

Guides accelerating LLM inference with speculative decoding, Medusa multi-head prediction, and lookahead decoding.

AI & Agents13Kupdated 3 months ago

Systems Paper Writing

orchestra-research

Guides paragraph-level structure and page budgets for writing systems conference papers such as OSDI, SOSP, ASPLOS and NSDI.

Writing & Content13Kupdated 3 months ago

Moe Training

orchestra-research

Guides training and inference of Mixture of Experts models with DeepSpeed or HuggingFace, covering routing, load balancing and expert parallelism.

AI & Agents13Kupdated 3 months ago

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

Awaiting classification13Kupdated 3 months ago

Model Pruning

orchestra-research

Guides LLM pruning with Wanda, SparseGPT, magnitude and N:M methods to cut model size and speed inference.

AI & Agents13Kupdated 3 months ago

Outlines

orchestra-research

Guides use of the Outlines library for structured, schema-constrained text generation with local models.

AI & Agents13Kupdated 3 months ago

Model Merging

Model Merging: Combining Pre-trained Models

When to Use This Skill

Installation

Quick Start

Simple Linear Merge

SLERP Merge (Best for 2 Models)

Core Concepts

1. Merge Methods

2. Configuration Structure

Merge Methods Guide

Linear Merge

SLERP Merge

Task Arithmetic

TIES-Merging

DARE Merge

Advanced Patterns

Layer-wise Merging

MoE from Merged Models

Tokenizer Merging

Best Practices

1. Model Compatibility

2. Weight Selection

3. Method Selection

4. Density Tuning (TIES/DARE)

5. Layer-specific Merging

Evaluation & Testing

Benchmark Merged Models

Common Benchmarks

Production Deployment

Save and Upload

Quantize Merged Model

Common Pitfalls

Resources

See Also

Source and attribution

More from orchestra-research/ai-research-skills

Speculative Decoding

Systems Paper Writing

Moe Training

Presenting Conference Talks

Model Pruning

Outlines

More in AI & Agents

Ppt Template Creator

Skill Development

Plugin Structure

Command Development

Claude Md Improver

Find Skills