Gke Ai Troubleshooting Tpu Mxla Hang
google/skills/skills/cloud/gke-ai-troubleshooting-tpu-mxla-hangby googlec6c7e67107f02e3423df8481272f296bbe61902aNo license21K starsListed Oct 9, 2026Updated Oct 9, 2026Repository updated today
Diagnose GKE Cloud TPU multi-slice training hangs (Megascale `HANG_DETECTED` logs and hang monitored events) using the ML Diagnostics `Megascale XLA (MXLA) Hang Analyzer` (`gcloud alpha mldiagnostics monitored-events`) and 1-minute Cloud Monitoring multi-slice latency metrics (`kubernetes.io/container/multislice/*`). Distinguishes XLA compiler/HLO launch divergence and host data-input stalls from TPU chip, SparseCore, ICI, or network fabric faults. Use when multi-slice TPU training jobs freeze without progressing steps, emit `HANG_DETECTED`, or stall in collective operations. Don't use for gradual step-time throughput drops without hangs (use gke-ai-troubleshooting-tpu-performance-degradation) or pod preemption/eviction restarts (use gke-ai-troubleshooting-jobset-interruption).
Add to a SourceWeft workspace
- Open the skill in your dashboard and add it to a workspace.
- Enable it for the chats that should use it.
This skill is instructions only: it ships no scripts to execute.
Add to SourceWeftYou will be asked to sign in first, then taken straight to this skill.
Ask your agent to install it
Paste this prompt into Claude Code, Codex, Cursor or another agent that can run commands — or into SourceWeft chat. The agent reads this skill's install guide, shows you its source, license and scripts, and installs it with the SourceWeft CLI once you agree.
Read https://sourceweft.com/skills/gh-google-skills-gke-ai-troubleshooting-tpu-mxla-hang-skills-cloud-gke-ai-troubleshooting-tpu-mxla-hang-3c11dd2fe212ffe3/install.md and install the skill it describes. Before installing, show me its source, license and whether it ships scripts, and wait for my OK. Ask me before changing anything else on my machine.Install it yourself from a terminal
For Claude Code, Codex, Cursor and other local agents. The SourceWeft CLI fetches the skill from its source repository at the commit scanned here, and verifies every file against the hashes recorded when the skill was scanned. If anything differs, nothing is written.
npx @sourceweft/cli skills install @google/gke-ai-troubleshooting-tpu-mxla-hangAdd --agent claude-code, codex, cursor or universal to choose which agent gets it (Claude Code by default).
Upstream installer — not verified by SourceWeft
The open-source skills installer fetches the same pinned commit, but does not check the files against the hashes SourceWeft recorded.
npx skills add https://github.com/google/skills/tree/c6c7e67107f02e3423df8481272f296bbe61902a/skills/cloud/gke-ai-troubleshooting-tpu-mxla-hangSource and attribution
Source:google/skillsinskills/cloud/gke-ai-troubleshooting-tpu-mxla-hangat commitc6c7e67
License: No license
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from google/skills
Sign In With Google Web
Guidance for implementing and securing Sign In With Google (GIS) across web architectures.
Gke Ai Troubleshooting Tpu Performance Degradation
Diagnoses GKE Cloud TPU training throughput drops using ML Diagnostics events and Cloud Monitoring metrics.
Google Cloud Scc Remediation
Guides remediation of Google Cloud Security Command Center findings with consent-gated plans and reference playbooks.
Google Cloud Recipe Onboarding
Guides a developer through first-time Google Cloud onboarding: authentication, project setup, billing linkage and validation.
Google Cloud Recipe Auth
Guides authentication and authorization to Google Cloud services and APIs for users, service accounts, and workloads.
Gcloud
Provides safety guardrails, syntax validation, and data-reduction rules for running gcloud CLI commands on Google Cloud.
More in DevOps & Cloud
Update Screenshots
microsoft
Updates committed CI screenshot hash baselines after the Screenshots & Tests check fails on a pull request.
Playwright Devops
microsoft
DevOps workflows for Playwright: analyze GitHub Actions failures for the last commit on main and fetch failed job logs.
M5 Onboard
anthropics
Provisions M5Stack ESP32 boards by detecting them on USB, flashing UIFlow 2.0 firmware, and installing a MicroPython app bundle.
Runbook
anthropics
Creates or updates step-by-step operational runbooks for recurring tasks, including troubleshooting, rollback and escalation.
Incident Response
anthropics
Guides an incident response workflow: severity triage, status updates, mitigation tracking, and blameless postmortems.
Deploy Checklist
anthropics
Generates a pre-deployment readiness checklist covering pre-deploy, deploy, post-deploy and rollback triggers.