Moe Training
orchestra-research/ai-research-skills/19-emerging-techniques/moe-trainingby orchestra-research773a52944ba4MIT13K starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated 3 months ago
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.
Only the file list is public. File contents are available once the skill is installed in a workspace.
| Path | Size | Type |
|---|---|---|
| references/architectures.md | 13.7 KB | text/markdown |
| references/inference.md | 8 KB | text/markdown |
| references/training.md | 9.5 KB | text/markdown |
| SKILL.md | 14.6 KB | text/markdown |
Source and attribution
Source:orchestra-research/ai-research-skillsin19-emerging-techniques/moe-trainingat commit773a529
License: MIT
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from orchestra-research/ai-research-skills
Speculative Decoding
orchestra-research
Guides accelerating LLM inference with speculative decoding, Medusa multi-head prediction, and lookahead decoding.
Systems Paper Writing
orchestra-research
Guides paragraph-level structure and page budgets for writing systems conference papers such as OSDI, SOSP, ASPLOS and NSDI.
Presenting Conference Talks
orchestra-research
Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.
Model Pruning
orchestra-research
Guides LLM pruning with Wanda, SparseGPT, magnitude and N:M methods to cut model size and speed inference.
Outlines
orchestra-research
Guides use of the Outlines library for structured, schema-constrained text generation with local models.
Ml Paper Writing
orchestra-research
Guides writing publication-ready ML/AI conference papers, from research repo to camera-ready LaTeX submission.
More in AI & Agents
Ppt Template Creator
anthropics
Turns a user's PowerPoint template into a reusable skill that generates branded presentations.
Skill Development
anthropics
Guides creation of Claude Code plugin skills, covering structure, descriptions, progressive disclosure and validation.
Plugin Structure
anthropics
Guides the structure, manifest, and component layout of Claude Code plugins.
Command Development
anthropics
Guides creation of Claude Code slash commands, covering structure, YAML frontmatter, arguments and plugin features.
Claude Md Improver
anthropics
Audits CLAUDE.md files in a repository, scores their quality, and applies approved targeted improvements.
Find Skills
vercel-labs
Guides discovery, quality checking and installation of agent skills via the npx skills CLI.