Tensorboard

orchestra-research/ai-research-skills/13-mlops/tensorboard

作者 orchestra-research773a52944ba4MIT13K 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫3 個月前更新

Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit

僅含說明Data & AnalyticsSoftware Development
AI 產生的概覽

使用 TensorBoard 記錄並視覺化機器學習訓練指標、模型圖、嵌入向量與效能剖析。

功能
此技能提供使用 TensorBoard 追蹤機器學習實驗的說明與程式碼範例。內容涵蓋記錄純量、影像、直方圖、文字、嵌入向量與 PR 曲線，視覺化模型圖，比較多次執行，調整超參數，以及在 PyTorch 與 TensorFlow/Keras 中進行效能剖析。它會產生可在 TensorBoard 儀表板中檢視的日誌目錄。
適用情境
適用於訓練或偵錯機器學習模型時，需要追蹤損失與準確率隨時間的變化、比較不同實驗、檢查權重與梯度分布，或找出效能瓶頸的情境。適合使用 PyTorch 與 TensorFlow/Keras 並希望以視覺化方式追蹤實驗的工作流程。
執行需求
需要 tensorboard 套件，框架相關範例還需要 torch 或 tensorflow。日誌會寫入本機目錄，並透過 TensorBoard 網頁介面檢視，通常位於 localhost:6006。此技能不含指令碼，只有說明文件與參考文件。

TensorBoard: Visualization Toolkit for ML

When to Use This Skill

Use TensorBoard when you need to:

  • Visualize training metrics like loss and accuracy over time
  • Debug models with histograms and distributions
  • Compare experiments across multiple runs
  • Visualize model graphs and architecture
  • Project embeddings to lower dimensions (t-SNE, PCA)
  • Track hyperparameter experiments
  • Profile performance and identify bottlenecks
  • Visualize images and text during training

Users: 20M+ downloads/year | GitHub Stars: 27k+ | License: Apache 2.0

Installation

bash
# Install TensorBoardpip install tensorboard
# PyTorch integrationpip install torch torchvision tensorboard
# TensorFlow integration (TensorBoard included)pip install tensorflow
# Launch TensorBoardtensorboard --logdir=runs# Access at http://localhost:6006

Quick Start

PyTorch

python
from torch.utils.tensorboard import SummaryWriter
# Create writerwriter = SummaryWriter('runs/experiment_1')
# Training loopfor epoch in range(10):    train_loss = train_epoch()    val_acc = validate()
    # Log metrics    writer.add_scalar('Loss/train', train_loss, epoch)    writer.add_scalar('Accuracy/val', val_acc, epoch)
# Close writerwriter.close()
# Launch: tensorboard --logdir=runs

TensorFlow/Keras

python
import tensorflow as tf
# Create callbacktensorboard_callback = tf.keras.callbacks.TensorBoard(    log_dir='logs/fit',    histogram_freq=1)
# Train modelmodel.fit(    x_train, y_train,    epochs=10,    validation_data=(x_val, y_val),    callbacks=[tensorboard_callback])
# Launch: tensorboard --logdir=logs

Core Concepts

1. SummaryWriter (PyTorch)

python
from torch.utils.tensorboard import SummaryWriter
# Default directory: runs/CURRENT_DATETIMEwriter = SummaryWriter()
# Custom directorywriter = SummaryWriter('runs/experiment_1')
# Custom comment (appended to default directory)writer = SummaryWriter(comment='baseline')
# Log datawriter.add_scalar('Loss/train', 0.5, step=0)writer.add_scalar('Loss/train', 0.3, step=1)
# Flush and closewriter.flush()writer.close()

2. Logging Scalars

python
# PyTorchfrom torch.utils.tensorboard import SummaryWriterwriter = SummaryWriter()
for epoch in range(100):    train_loss = train()    val_loss = validate()
    # Log individual metrics    writer.add_scalar('Loss/train', train_loss, epoch)    writer.add_scalar('Loss/val', val_loss, epoch)    writer.add_scalar('Accuracy/train', train_acc, epoch)    writer.add_scalar('Accuracy/val', val_acc, epoch)
    # Learning rate    lr = optimizer.param_groups[0]['lr']    writer.add_scalar('Learning_rate', lr, epoch)
writer.close()
python
# TensorFlowimport tensorflow as tf
train_summary_writer = tf.summary.create_file_writer('logs/train')val_summary_writer = tf.summary.create_file_writer('logs/val')
for epoch in range(100):    with train_summary_writer.as_default():        tf.summary.scalar('loss', train_loss, step=epoch)        tf.summary.scalar('accuracy', train_acc, step=epoch)
    with val_summary_writer.as_default():        tf.summary.scalar('loss', val_loss, step=epoch)        tf.summary.scalar('accuracy', val_acc, step=epoch)

3. Logging Multiple Scalars

python
# PyTorch: Group related metricswriter.add_scalars('Loss', {    'train': train_loss,    'validation': val_loss,    'test': test_loss}, epoch)
writer.add_scalars('Metrics', {    'accuracy': accuracy,    'precision': precision,    'recall': recall,    'f1': f1_score}, epoch)

4. Logging Images

python
# PyTorchimport torchfrom torchvision.utils import make_grid
# Single imagewriter.add_image('Input/sample', img_tensor, epoch)
# Multiple images as gridimg_grid = make_grid(images[:64], nrow=8)writer.add_image('Batch/inputs', img_grid, epoch)
# Predictions visualizationpred_grid = make_grid(predictions[:16], nrow=4)writer.add_image('Predictions', pred_grid, epoch)
python
# TensorFlowimport tensorflow as tf
with file_writer.as_default():    # Encode images as PNG    tf.summary.image('Training samples', images, step=epoch, max_outputs=25)

5. Logging Histograms

python
# PyTorch: Track weight distributionsfor name, param in model.named_parameters():    writer.add_histogram(name, param, epoch)
    # Track gradients    if param.grad is not None:        writer.add_histogram(f'{name}.grad', param.grad, epoch)
# Track activationswriter.add_histogram('Activations/relu1', activations, epoch)
python
# TensorFlowwith file_writer.as_default():    tf.summary.histogram('weights/layer1', layer1.kernel, step=epoch)    tf.summary.histogram('activations/relu1', activations, step=epoch)

6. Logging Model Graph

python
# PyTorchimport torch
model = MyModel()dummy_input = torch.randn(1, 3, 224, 224)
writer.add_graph(model, dummy_input)writer.close()
python
# TensorFlow (automatic with Keras)tensorboard_callback = tf.keras.callbacks.TensorBoard(    log_dir='logs',    write_graph=True)
model.fit(x, y, callbacks=[tensorboard_callback])

Advanced Features

Embedding Projector

Visualize high-dimensional data (embeddings, features) in 2D/3D.

python
import torchfrom torch.utils.tensorboard import SummaryWriter
# Get embeddings (e.g., word embeddings, image features)embeddings = model.get_embeddings(data)  # Shape: (N, embedding_dim)
# Metadata (labels for each point)metadata = ['class_1', 'class_2', 'class_1', ...]
# Images (optional, for image embeddings)label_images = torch.stack([img1, img2, img3, ...])
# Log to TensorBoardwriter.add_embedding(    embeddings,    metadata=metadata,    label_img=label_images,    global_step=epoch)

In TensorBoard:

  • Navigate to "Projector" tab
  • Choose PCA, t-SNE, or UMAP visualization
  • Search, filter, and explore clusters

Hyperparameter Tuning

python
from torch.utils.tensorboard import SummaryWriter
# Try different hyperparametersfor lr in [0.001, 0.01, 0.1]:    for batch_size in [16, 32, 64]:        # Create unique run directory        writer = SummaryWriter(f'runs/lr{lr}_bs{batch_size}')
        # Log hyperparameters        writer.add_hparams(            {'lr': lr, 'batch_size': batch_size},            {'hparam/accuracy': final_acc, 'hparam/loss': final_loss}        )
        # Train and log        for epoch in range(10):            loss = train(lr, batch_size)            writer.add_scalar('Loss/train', loss, epoch)
        writer.close()
# Compare in TensorBoard's "HParams" tab

Text Logging

python
# PyTorch: Log text (e.g., model predictions, summaries)writer.add_text('Predictions', f'Epoch {epoch}: {predictions}', epoch)writer.add_text('Config', str(config), 0)
# Log markdown tablesmarkdown_table = """| Metric | Value ||--------|-------|| Accuracy | 0.95 || F1 Score | 0.93 |"""writer.add_text('Results', markdown_table, epoch)

PR Curves

Precision-Recall curves for classification.

python
from torch.utils.tensorboard import SummaryWriter
# Get predictions and labelspredictions = model(test_data)  # Shape: (N, num_classes)labels = test_labels  # Shape: (N,)
# Log PR curve for each classfor i in range(num_classes):    writer.add_pr_curve(        f'PR_curve/class_{i}',        labels == i,        predictions[:, i],        global_step=epoch    )

Integration Examples

PyTorch Training Loop

python
import torchimport torch.nn as nnfrom torch.utils.tensorboard import SummaryWriter
# Setupwriter = SummaryWriter('runs/resnet_experiment')model = ResNet50()optimizer = torch.optim.Adam(model.parameters(), lr=0.001)criterion = nn.CrossEntropyLoss()
# Log model graphdummy_input = torch.randn(1, 3, 224, 224)writer.add_graph(model, dummy_input)
# Training loopfor epoch in range(50):    model.train()    train_loss = 0.0    train_correct = 0
    for batch_idx, (data, target) in enumerate(train_loader):        optimizer.zero_grad()        output = model(data)        loss = criterion(output, target)        loss.backward()        optimizer.step()
        train_loss += loss.item()        pred = output.argmax(dim=1)        train_correct += pred.eq(target).sum().item()
        # Log batch metrics (every 100 batches)        if batch_idx % 100 == 0:            global_step = epoch * len(train_loader) + batch_idx            writer.add_scalar('Loss/train_batch', loss.item(), global_step)
    # Epoch metrics    train_loss /= len(train_loader)    train_acc = train_correct / len(train_loader.dataset)
    # Validation    model.eval()    val_loss = 0.0    val_correct = 0
    with torch.no_grad():        for data, target in val_loader:            output = model(data)            val_loss += criterion(output, target).item()            pred = output.argmax(dim=1)            val_correct += pred.eq(target).sum().item()
    val_loss /= len(val_loader)    val_acc = val_correct / len(val_loader.dataset)
    # Log epoch metrics    writer.add_scalars('Loss', {'train': train_loss, 'val': val_loss}, epoch)    writer.add_scalars('Accuracy', {'train': train_acc, 'val': val_acc}, epoch)
    # Log learning rate    writer.add_scalar('Learning_rate', optimizer.param_groups[0]['lr'], epoch)
    # Log histograms (every 5 epochs)    if epoch % 5 == 0:        for name, param in model.named_parameters():            writer.add_histogram(name, param, epoch)
    # Log sample predictions    if epoch % 10 == 0:        sample_images = data[:8]        writer.add_image('Sample_inputs', make_grid(sample_images), epoch)
writer.close()

TensorFlow/Keras Training

python
import tensorflow as tf
# Define modelmodel = tf.keras.models.Sequential([    tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),    tf.keras.layers.MaxPooling2D(),    tf.keras.layers.Flatten(),    tf.keras.layers.Dense(128, activation='relu'),    tf.keras.layers.Dense(10, activation='softmax')])
model.compile(    optimizer='adam',    loss='sparse_categorical_crossentropy',    metrics=['accuracy'])
# TensorBoard callbacktensorboard_callback = tf.keras.callbacks.TensorBoard(    log_dir='logs/fit',    histogram_freq=1,          # Log histograms every epoch    write_graph=True,          # Visualize model graph    write_images=True,         # Visualize weights as images    update_freq='epoch',       # Log metrics every epoch    profile_batch='500,520',   # Profile batches 500-520    embeddings_freq=1          # Log embeddings every epoch)
# Trainmodel.fit(    x_train, y_train,    epochs=10,    validation_data=(x_val, y_val),    callbacks=[tensorboard_callback])

Comparing Experiments

Multiple Runs

bash
# Run experiments with different configspython train.py --lr 0.001 --logdir runs/exp1python train.py --lr 0.01 --logdir runs/exp2python train.py --lr 0.1 --logdir runs/exp3
# View all runs togethertensorboard --logdir=runs

In TensorBoard:

  • All runs appear in the same dashboard
  • Toggle runs on/off for comparison
  • Use regex to filter run names
  • Overlay charts to compare metrics

Organizing Experiments

python
# Hierarchical organizationruns/├── baseline/│   ├── run_1/│   └── run_2/├── improved/│   ├── run_1/│   └── run_2/└── final/    └── run_1/
# Log with hierarchywriter = SummaryWriter('runs/baseline/run_1')

Best Practices

1. Use Descriptive Run Names

python
# ✅ Good: Descriptive namesfrom datetime import datetimetimestamp = datetime.now().strftime('%Y%m%d_%H%M%S')writer = SummaryWriter(f'runs/resnet50_lr0.001_bs32_{timestamp}')
# ❌ Bad: Auto-generated nameswriter = SummaryWriter()  # Creates runs/Jan01_12-34-56_hostname

2. Group Related Metrics

python
# ✅ Good: Grouped metricswriter.add_scalar('Loss/train', train_loss, step)writer.add_scalar('Loss/val', val_loss, step)writer.add_scalar('Accuracy/train', train_acc, step)writer.add_scalar('Accuracy/val', val_acc, step)
# ❌ Bad: Flat namespacewriter.add_scalar('train_loss', train_loss, step)writer.add_scalar('val_loss', val_loss, step)

3. Log Regularly but Not Too Often

python
# ✅ Good: Log epoch metrics always, batch metrics occasionallyfor epoch in range(100):    for batch_idx, (data, target) in enumerate(train_loader):        loss = train_step(data, target)
        # Log every 100 batches        if batch_idx % 100 == 0:            writer.add_scalar('Loss/batch', loss, global_step)
    # Always log epoch metrics    writer.add_scalar('Loss/epoch', epoch_loss, epoch)
# ❌ Bad: Log every batch (creates huge log files)for batch in train_loader:    writer.add_scalar('Loss', loss, step)  # Too frequent

4. Close Writer When Done

python
# ✅ Good: Use context managerwith SummaryWriter('runs/exp1') as writer:    for epoch in range(10):        writer.add_scalar('Loss', loss, epoch)# Automatically closes
# Or manuallywriter = SummaryWriter('runs/exp1')# ... logging ...writer.close()

5. Use Separate Writers for Train/Val

python
# ✅ Good: Separate log directoriestrain_writer = SummaryWriter('runs/exp1/train')val_writer = SummaryWriter('runs/exp1/val')
train_writer.add_scalar('loss', train_loss, epoch)val_writer.add_scalar('loss', val_loss, epoch)

Performance Profiling

TensorFlow Profiler

python
# Enable profilingtensorboard_callback = tf.keras.callbacks.TensorBoard(    log_dir='logs',    profile_batch='10,20'  # Profile batches 10-20)
model.fit(x, y, callbacks=[tensorboard_callback])
# View in TensorBoard Profile tab# Shows: GPU utilization, kernel stats, memory usage, bottlenecks

PyTorch Profiler

python
import torch.profiler as profiler
with profiler.profile(    activities=[        profiler.ProfilerActivity.CPU,        profiler.ProfilerActivity.CUDA    ],    on_trace_ready=torch.profiler.tensorboard_trace_handler('./runs/profiler'),    record_shapes=True,    with_stack=True) as prof:    for batch in train_loader:        loss = train_step(batch)        prof.step()
# View in TensorBoard Profile tab

Resources

See Also

  • references/visualization.md - Comprehensive visualization guide
  • references/profiling.md - Performance profiling patterns
  • references/integrations.md - Framework-specific integration examples

來源與署名

來源：orchestra-research/ai-research-skills位於13-mlops/tensorboard提交773a529

授權條款： MIT

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架

更多來自 orchestra-research/ai-research-skills 的技能

Speculative Decoding

orchestra-research

指導使用推測解碼、Medusa 多頭預測與前瞻解碼來加速大型語言模型推論。

AI & Agents13K3 個月前更新

Systems Paper Writing

orchestra-research

為 OSDI、SOSP、ASPLOS、NSDI 等系統會議論文寫作提供段落層級結構與篇幅分配指引。

Writing & Content13K3 個月前更新

Moe Training

orchestra-research

指導使用 DeepSpeed 或 HuggingFace 訓練與推論混合專家模型，涵蓋路由、負載平衡與專家平行。

AI & Agents13K3 個月前更新

Presenting Conference Talks

orchestra-research

Generates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes and talk script. Use when preparing oral talks, spotlight presentations, or invited talks for ML and systems conferences.

待分類13K3 個月前更新

Model Pruning

orchestra-research

指導使用 Wanda、SparseGPT、幅度剪枝與 N:M 方法壓縮 LLM，縮小模型體積並加速推論。

AI & Agents13K3 個月前更新

Outlines

orchestra-research

指導使用 Outlines 函式庫，透過本機模型進行受結構約束的文字生成。

AI & Agents13K3 個月前更新

更多Data & Analytics技能

Swap Curve Strategy

anthropics

精選

運用 MCP 定價工具分析利率交換曲線、交換利差、實質利率與曲線交易機會。

Data & Analytics39K2 週前更新

Unit Economics

anthropics

精選

為私募股權標的分析單位經濟：ARR 世代、LTV/CAC、淨留存、回收期與利潤瀑布。

Data & Analytics39K2 週前更新

Option Vol Analysis

anthropics

精選

結合波動率曲面、含希臘字母的選擇權定價與歷史價格，比較隱含波動率與已實現波動率。

Data & Analytics39K2 週前更新

Macro Rates Monitor

anthropics

精選

運用 MCP 的 GDP、通膨、殖利率曲線、損益兩平通膨與交換利率資料，建立總體與利率儀表板。

Data & Analytics39K2 週前更新

Clean Data Xls

anthropics

精選

清理雜亂的試算表資料：修剪空白、統一大寫、轉換文字數字、標準化日期並移除重複項。

Data & Analytics39K2 週前更新

Fixed Income Portfolio

anthropics

精選

透過債券定價、彙總風險指標、預測現金流量並執行利率情境，審查固定收益投資組合。

Data & Analytics39K2 週前更新