Model Merging

orchestra-research/ai-research-skills/19-emerging-techniques/model-merging

作者 orchestra-research773a52944ba4MIT13K 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫3 個月前更新

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

僅含說明AI & Agents

AI 產生的概覽

指導使用 mergekit 合併微調模型，在不重新訓練的情況下組合多種能力。

功能: 此技能提供使用 mergekit 合併微調語言模型的說明與 YAML 設定模式。內容涵蓋線性合併、SLERP、任務算術、TIES-Merging、DARE、分層直通以及 MoE 建構等方法，並提供權重與密度調校建議。它也說明合併模型的評估、量化與部署步驟。參考檔案涵蓋方法、範例、評估與係數調校。
適用情境: 適用於在不重新訓練的情況下，將多個微調模型合併為一個模型，融合數學、程式設計、對話等領域能力。也適合快速實驗模型變體，以及讓基準表現超越單一模型。
執行需求: 需要 mergekit、transformers 與 torch，並能存取模型權重及足夠的運算或記憶體資源；合併可在 CPU 上執行。此技能不附帶指令碼，只有說明與參考文件。

Model Merging: Combining Pre-trained Models

When to Use This Skill

Use Model Merging when you need to:

Combine capabilities from multiple fine-tuned models without retraining
Create specialized models by blending domain-specific expertise (math + coding + chat)
Improve performance beyond single models (often +5-10% on benchmarks)
Reduce training costs - no GPUs needed, merges run on CPU
Experiment rapidly - create new model variants in minutes, not days
Preserve multiple skills - merge without catastrophic forgetting

Success Stories: Marcoro14-7B-slerp (best on Open LLM Leaderboard 02/2024), many top HuggingFace models use merging

Tools: mergekit (Arcee AI), LazyMergekit, Model Soup

Installation

bash

# Install mergekitgit clone https://github.com/arcee-ai/mergekit.gitcd mergekitpip install -e .
# Or via pippip install mergekit
# Optional: Transformer librarypip install transformers torch

# Install mergekitgit clone https://github.com/arcee-ai/mergekit.gitcd mergekitpip install -e .
# Or via pippip install mergekit
# Optional: Transformer librarypip install transformers torch

Quick Start

Simple Linear Merge

yaml

# config.yml - Merge two models with equal weightsmerge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.5dtype: bfloat16

# config.yml - Merge two models with equal weightsmerge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.5dtype: bfloat16

bash

# Run mergemergekit-yaml config.yml ./merged-model --cuda
# Use merged modelpython -m transformers.models.auto --model_name_or_path ./merged-model

# Run mergemergekit-yaml config.yml ./merged-model --cuda
# Use merged modelpython -m transformers.models.auto --model_name_or_path ./merged-model

SLERP Merge (Best for 2 Models)

yaml

# config.yml - Spherical interpolationmerge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # Interpolation factor (0=model1, 1=model2)dtype: bfloat16

# config.yml - Spherical interpolationmerge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # Interpolation factor (0=model1, 1=model2)dtype: bfloat16

Core Concepts

1. Merge Methods

Linear (Model Soup)

Simple weighted average of parameters
Fast, works well for similar models
Can merge 2+ models (w1 + w2 + ... = 1)

SLERP (Spherical Linear Interpolation)

Interpolates along sphere in weight space
Preserves magnitude of weight vectors
Best for merging 2 models
Smoother than linear

python

# SLERP formulamerged = (sin((1-t)*θ) / sin(θ)) * model1 + (sin(t*θ) / sin(θ)) * model2# where θ = arccos(dot(model1, model2))# t ∈ [0, 1]

# SLERP formulamerged = (sin((1-t)*θ) / sin(θ)) * model1 + (sin(t*θ) / sin(θ)) * model2# where θ = arccos(dot(model1, model2))# t ∈ [0, 1]

Task Arithmetic

Extract "task vectors" (fine-tuned - base)
Combine task vectors, add to base
Good for merging multiple specialized models (merged = base + α₁·tv₁ + α₂·tv₂)

TIES-Merging

Task arithmetic + sparsification
Resolves sign conflicts in parameters
Best for merging many task-specific models

DARE (Drop And REscale)

Randomly drops fine-tuned parameters
Rescales remaining parameters
Reduces redundancy, maintains performance

2. Configuration Structure

yaml

# Basic structuremerge_method: <method>  # linear, slerp, ties, dare_ties, task_arithmeticbase_model: <path>      # Optional: base model for task arithmetic
models:  - model: <path/to/model1>    parameters:      weight: <float>   # Merge weight      density: <float>  # For TIES/DARE
  - model: <path/to/model2>    parameters:      weight: <float>
parameters:  # Method-specific parameters
dtype: <dtype>  # bfloat16, float16, float32
# Optionalslices:  # Layer-wise mergingtokenizer:  # Tokenizer configuration

# Basic structuremerge_method: <method>  # linear, slerp, ties, dare_ties, task_arithmeticbase_model: <path>      # Optional: base model for task arithmetic
models:  - model: <path/to/model1>    parameters:      weight: <float>   # Merge weight      density: <float>  # For TIES/DARE
  - model: <path/to/model2>    parameters:      weight: <float>
parameters:  # Method-specific parameters
dtype: <dtype>  # bfloat16, float16, float32
# Optionalslices:  # Layer-wise mergingtokenizer:  # Tokenizer configuration

Merge Methods Guide

Linear Merge

Best for: Simple model combinations, equal weighting

yaml

merge_method: linearmodels:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      weight: 0.4  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.3  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      weight: 0.3dtype: bfloat16

merge_method: linearmodels:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      weight: 0.4  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      weight: 0.3  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      weight: 0.3dtype: bfloat16

SLERP Merge

Best for: Two models, smooth interpolation

yaml

merge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # 0.0 = first model, 1.0 = second modeldtype: bfloat16

merge_method: slerpslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 32]      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [0, 32]parameters:  t: 0.5  # 0.0 = first model, 1.0 = second modeldtype: bfloat16

Layer-specific SLERP:

yaml

merge_method: slerpslices:  - sources:      - model: model_a        layer_range: [0, 32]      - model: model_b        layer_range: [0, 32]parameters:  t:    - filter: self_attn    # Attention layers      value: 0.3    - filter: mlp          # MLP layers      value: 0.7    - value: 0.5           # Default for other layersdtype: bfloat16

merge_method: slerpslices:  - sources:      - model: model_a        layer_range: [0, 32]      - model: model_b        layer_range: [0, 32]parameters:  t:    - filter: self_attn    # Attention layers      value: 0.3    - filter: mlp          # MLP layers      value: 0.7    - value: 0.5           # Default for other layersdtype: bfloat16

Task Arithmetic

Best for: Combining specialized skills

yaml

merge_method: task_arithmeticbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1  # Math    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B  # Chat    parameters:      weight: 0.3  - model: ajibawa-2023/Code-Mistral-7B  # Code    parameters:      weight: 0.2dtype: bfloat16

merge_method: task_arithmeticbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1  # Math    parameters:      weight: 0.5  - model: teknium/OpenHermes-2.5-Mistral-7B  # Chat    parameters:      weight: 0.3  - model: ajibawa-2023/Code-Mistral-7B  # Code    parameters:      weight: 0.2dtype: bfloat16

TIES-Merging

Best for: Many models, resolving conflicts

yaml

merge_method: tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5  # Keep top 50% of parameters      weight: 1.0  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 1.0  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      density: 0.5      weight: 1.0parameters:  normalize: truedtype: bfloat16

merge_method: tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5  # Keep top 50% of parameters      weight: 1.0  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 1.0  - model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO    parameters:      density: 0.5      weight: 1.0parameters:  normalize: truedtype: bfloat16

DARE Merge

Best for: Reducing redundancy

yaml

merge_method: dare_tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5    # Drop 50% of deltas      weight: 0.6  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 0.4parameters:  int8_mask: true  # Use int8 for masks (saves memory)dtype: bfloat16

merge_method: dare_tiesbase_model: mistralai/Mistral-7B-v0.1models:  - model: WizardLM/WizardMath-7B-V1.1    parameters:      density: 0.5    # Drop 50% of deltas      weight: 0.6  - model: teknium/OpenHermes-2.5-Mistral-7B    parameters:      density: 0.5      weight: 0.4parameters:  int8_mask: true  # Use int8 for masks (saves memory)dtype: bfloat16

Advanced Patterns

Layer-wise Merging

yaml

# Different models for different layersmerge_method: passthroughslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 16]   # First half  - sources:      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [16, 32]  # Second halfdtype: bfloat16

# Different models for different layersmerge_method: passthroughslices:  - sources:      - model: mistralai/Mistral-7B-v0.1        layer_range: [0, 16]   # First half  - sources:      - model: teknium/OpenHermes-2.5-Mistral-7B        layer_range: [16, 32]  # Second halfdtype: bfloat16

MoE from Merged Models

yaml

# Create Mixture of Expertsmerge_method: moebase_model: mistralai/Mistral-7B-v0.1experts:  - source_model: WizardLM/WizardMath-7B-V1.1    positive_prompts:      - "math"      - "calculate"  - source_model: teknium/OpenHermes-2.5-Mistral-7B    positive_prompts:      - "chat"      - "conversation"  - source_model: ajibawa-2023/Code-Mistral-7B    positive_prompts:      - "code"      - "python"dtype: bfloat16

# Create Mixture of Expertsmerge_method: moebase_model: mistralai/Mistral-7B-v0.1experts:  - source_model: WizardLM/WizardMath-7B-V1.1    positive_prompts:      - "math"      - "calculate"  - source_model: teknium/OpenHermes-2.5-Mistral-7B    positive_prompts:      - "chat"      - "conversation"  - source_model: ajibawa-2023/Code-Mistral-7B    positive_prompts:      - "code"      - "python"dtype: bfloat16

Tokenizer Merging

yaml

merge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1  - model: custom/specialized-model
tokenizer:  source: "union"  # Combine vocabularies from both models  tokens:    <|special_token|>:      source: "custom/specialized-model"

merge_method: linearmodels:  - model: mistralai/Mistral-7B-v0.1  - model: custom/specialized-model
tokenizer:  source: "union"  # Combine vocabularies from both models  tokens:    <|special_token|>:      source: "custom/specialized-model"

Best Practices

1. Model Compatibility

python

# ✅ Good: Same architecturemodels = [    "mistralai/Mistral-7B-v0.1",    "teknium/OpenHermes-2.5-Mistral-7B",  # Both Mistral 7B]
# ❌ Bad: Different architecturesmodels = [    "meta-llama/Llama-2-7b-hf",  # Llama    "mistralai/Mistral-7B-v0.1",  # Mistral (incompatible!)]

# ✅ Good: Same architecturemodels = [    "mistralai/Mistral-7B-v0.1",    "teknium/OpenHermes-2.5-Mistral-7B",  # Both Mistral 7B]
# ❌ Bad: Different architecturesmodels = [    "meta-llama/Llama-2-7b-hf",  # Llama    "mistralai/Mistral-7B-v0.1",  # Mistral (incompatible!)]

2. Weight Selection

yaml

# ✅ Good: Weights sum to 1.0models:  - model: model_a    parameters:      weight: 0.6  - model: model_b    parameters:      weight: 0.4  # 0.6 + 0.4 = 1.0
# ⚠️  Acceptable: Weights don't sum to 1 (for task arithmetic)models:  - model: model_a    parameters:      weight: 0.8  - model: model_b    parameters:      weight: 0.8  # May boost performance

# ✅ Good: Weights sum to 1.0models:  - model: model_a    parameters:      weight: 0.6  - model: model_b    parameters:      weight: 0.4  # 0.6 + 0.4 = 1.0
# ⚠️  Acceptable: Weights don't sum to 1 (for task arithmetic)models:  - model: model_a    parameters:      weight: 0.8  - model: model_b    parameters:      weight: 0.8  # May boost performance

Unsupervised Coefficient Tuning (no labeled data needed)

Instead of manual search, use generation consistency: merge with several candidate coefficients, generate responses on a small unlabeled subset, and pick the coefficient whose outputs are most similar to those of its neighbors. Consistent outputs signal a stable, well-performing merge region (AdaMMS, arXiv:2503.23733).

python

# Pseudocode — see references/coefficient-tuning.md for full implementationcandidates = [0.3, 0.4, 0.5, 0.6, 0.7]for alpha in candidates:    merged_paths[alpha] = merge_with_coefficient(alpha, model_a, model_b)    responses[alpha]    = generate_responses(merged_paths[alpha], eval_prompts)
# Score each alpha by similarity to its neighbors (alpha ± 0.1)best_alpha = max(candidates, key=lambda a: generation_consistency(a, responses))

# Pseudocode — see references/coefficient-tuning.md for full implementationcandidates = [0.3, 0.4, 0.5, 0.6, 0.7]for alpha in candidates:    merged_paths[alpha] = merge_with_coefficient(alpha, model_a, model_b)    responses[alpha]    = generate_responses(merged_paths[alpha], eval_prompts)
# Score each alpha by similarity to its neighbors (alpha ± 0.1)best_alpha = max(candidates, key=lambda a: generation_consistency(a, responses))

See references/coefficient-tuning.md [blocked] for the full algorithm, similarity metrics, multi-coefficient search, and end-to-end pipeline.

3. Method Selection

python

# Choose merge method based on use case:
# 2 models, smooth blend → SLERPmerge_method = "slerp"
# 3+ models, simple average → Linearmerge_method = "linear"
# Multiple task-specific models → Task Arithmetic or TIESmerge_method = "ties"
# Want to reduce redundancy → DAREmerge_method = "dare_ties"

# Choose merge method based on use case:
# 2 models, smooth blend → SLERPmerge_method = "slerp"
# 3+ models, simple average → Linearmerge_method = "linear"
# Multiple task-specific models → Task Arithmetic or TIESmerge_method = "ties"
# Want to reduce redundancy → DAREmerge_method = "dare_ties"

4. Density Tuning (TIES/DARE)

yaml

# Start conservative (keep more parameters)parameters:  density: 0.8  # Keep 80%
# If performance good, increase sparsityparameters:  density: 0.5  # Keep 50%
# If performance degrades, reduce sparsityparameters:  density: 0.9  # Keep 90%

# Start conservative (keep more parameters)parameters:  density: 0.8  # Keep 80%
# If performance good, increase sparsityparameters:  density: 0.5  # Keep 50%
# If performance degrades, reduce sparsityparameters:  density: 0.9  # Keep 90%

5. Layer-specific Merging

Preserve the base model's first/last layers (often best left untouched) and merge only the middle via merge_method: passthrough with slices — see the Layer-wise Merging pattern above.

Evaluation & Testing

Benchmark Merged Models

python

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Test on various taskstest_prompts = {    "math": "Calculate: 25 * 17 =",    "code": "Write a Python function to reverse a string:",    "chat": "What is the capital of France?",}
for task, prompt in test_prompts.items():    inputs = tokenizer(prompt, return_tensors="pt")    outputs = model.generate(**inputs, max_length=100)    print(f"{task}: {tokenizer.decode(outputs[0])}")

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Test on various taskstest_prompts = {    "math": "Calculate: 25 * 17 =",    "code": "Write a Python function to reverse a string:",    "chat": "What is the capital of France?",}
for task, prompt in test_prompts.items():    inputs = tokenizer(prompt, return_tensors="pt")    outputs = model.generate(**inputs, max_length=100)    print(f"{task}: {tokenizer.decode(outputs[0])}")

Common Benchmarks

Open LLM Leaderboard: General capabilities
MT-Bench: Multi-turn conversation
MMLU: Multitask accuracy
HumanEval: Code generation
GSM8K: Math reasoning

Production Deployment

Save and Upload

python

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Upload to HuggingFace Hubmodel.push_to_hub("username/my-merged-model")tokenizer.push_to_hub("username/my-merged-model")

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load merged modelmodel = AutoModelForCausalLM.from_pretrained("./merged-model")tokenizer = AutoTokenizer.from_pretrained("./merged-model")
# Upload to HuggingFace Hubmodel.push_to_hub("username/my-merged-model")tokenizer.push_to_hub("username/my-merged-model")

Quantize Merged Model

bash

# Quantize with GGUFpython convert.py ./merged-model --outtype f16 --outfile merged-model.gguf
# Quantize with GPTQpython quantize_gptq.py ./merged-model --bits 4 --group_size 128

# Quantize with GGUFpython convert.py ./merged-model --outtype f16 --outfile merged-model.gguf
# Quantize with GPTQpython quantize_gptq.py ./merged-model --bits 4 --group_size 128

Common Pitfalls

Mismatched architectures — only merge models that share the same architecture (e.g., don't mix Llama and Mistral).
Over-weighting one model (e.g., 0.95 / 0.05) — keep weights balanced, typically in the 0.3–0.7 range.
Skipping evaluation — always benchmark a merged model before deploying (see the Evaluation & Testing section above).

Resources

mergekit GitHub: https://github.com/arcee-ai/mergekit
HuggingFace Tutorial: https://huggingface.co/blog/mlabonne/merge-models
LazyMergekit: Automated merging notebook
TIES Paper: https://arxiv.org/abs/2306.01708
DARE Paper: https://arxiv.org/abs/2311.03099

來源與署名

來源：orchestra-research/ai-research-skills位於19-emerging-techniques/model-merging提交773a529

授權條款： MIT

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架

Model Merging

Model Merging: Combining Pre-trained Models

When to Use This Skill

Installation

Quick Start

Simple Linear Merge

SLERP Merge (Best for 2 Models)

Core Concepts

1. Merge Methods

2. Configuration Structure

Merge Methods Guide

Linear Merge

SLERP Merge

Task Arithmetic

TIES-Merging

DARE Merge

Advanced Patterns

Layer-wise Merging

MoE from Merged Models

Tokenizer Merging

Best Practices

1. Model Compatibility

2. Weight Selection

3. Method Selection

4. Density Tuning (TIES/DARE)

5. Layer-specific Merging

Evaluation & Testing

Benchmark Merged Models

Common Benchmarks

Production Deployment

Save and Upload

Quantize Merged Model

Common Pitfalls

Resources

See Also

來源與署名

更多來自 orchestra-research/ai-research-skills 的技能

Speculative Decoding

Systems Paper Writing

Moe Training

Presenting Conference Talks

Model Pruning

Outlines

更多AI & Agents技能

Ppt Template Creator

Skill Development

Plugin Structure

Command Development

Claude Md Improver

Find Skills