Gptq
orchestra-research/ai-research-skills/10-optimization/gptqby orchestra-research773a52944ba4MIT13K starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated 3 months ago
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
- 773a52944ba4Currentcommit 773a529Published Oct 8, 2026
Source and attribution
Source:orchestra-research/ai-research-skillsin10-optimization/gptqat commit773a529
License: MIT
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from orchestra-research/ai-research-skills
Speculative Decoding
orchestra-research
Guides accelerating LLM inference with speculative decoding, Medusa multi-head prediction, and lookahead decoding.
Systems Paper Writing
orchestra-research
Guides paragraph-level structure and page budgets for writing systems conference papers such as OSDI, SOSP, ASPLOS and NSDI.
Moe Training
orchestra-research
Guides training and inference of Mixture of Experts models with DeepSpeed or HuggingFace, covering routing, load balancing and expert parallelism.
Presenting Conference Talks
orchestra-research
Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.
Model Pruning
orchestra-research
Guides LLM pruning with Wanda, SparseGPT, magnitude and N:M methods to cut model size and speed inference.
Outlines
orchestra-research
Guides use of the Outlines library for structured, schema-constrained text generation with local models.
More in Software Development
Playground
anthropics
Builds self-contained interactive HTML playgrounds with controls, live preview, and copyable prompt output.
M5 Onboard
anthropics
Provisions M5Stack ESP32 boards by detecting them on USB, flashing UIFlow 2.0 firmware, and installing a MicroPython app bundle.
Cardputer Buddy
anthropics
Guides iterating on a Cardputer-Adv MicroPython app bundle: adding apps, pushing files over USB-serial, tailing logs, and running REPL commands.
Web Design Guidelines
vercel-labs
Reviews UI code against Web Interface Guidelines fetched from a remote source and reports findings as file:line entries.
Vercel React Native Skills
vercel-labs
React Native and Expo best-practice rules covering list performance, animation, navigation, UI patterns, state, and monorepo setup.
Vercel React Best Practices
vercel-labs
Vercel's React and Next.js performance guidelines, organized as 70 rules for writing, reviewing, and refactoring code.