Buildkite Agent Runtime

Buildkite Agent Runtime

The buildkite-agent binary provides subcommands for interacting with Buildkite from within running job steps — creating annotations, uploading artifacts, sharing state between jobs, generating dynamic pipelines, requesting OIDC tokens, and more. This skill covers the command syntax, flags, and patterns for every in-job subcommand.

Quick Start

A step that runs tests, annotates failures, uploads coverage, and stores a result flag for downstream jobs:

steps:  - label: ":test_tube: Tests"    command: |      if ! make test 2>&1 | tee test-output.txt; then        buildkite-agent annotate --style "error" --context "test-failures" < test-output.txt        buildkite-agent meta-data set "tests-passed" "false"        exit 1      fi      buildkite-agent annotate "All tests passed :white_check_mark:" --style "success" --context "test-results"      buildkite-agent artifact upload "coverage/**/*"      buildkite-agent meta-data set "tests-passed" "true"

A downstream step reading that state:

  - label: ":rocket: Deploy"    command: |      PASSED=$(buildkite-agent meta-data get "tests-passed")      if [[ "$PASSED" != "true" ]]; then        echo "Tests did not pass, skipping deploy"        exit 0      fi      scripts/deploy.sh    depends_on: "test-step"

Annotations

Surface build results directly on the build page. Annotations support Markdown and HTML.

Creating annotations

# Simple text annotationbuildkite-agent annotate "Deploy completed successfully" --style "success" --context "deploy"
# Pipe from a filebuildkite-agent annotate --style "error" --context "test-failures" < test-output.md

Key flags

Replacing vs appending

• Same --context without --append replaces the annotation; with --append appends below existing content.
• Always use a stable --context value so reruns update the same annotation instead of creating duplicates.

For pipeline-level notify: configuration, see the buildkite-pipelines skill.

Artifacts

Upload files as build artifacts, download them in later steps or other builds, and search by glob.

Upload

# Upload a single filebuildkite-agent artifact upload "pkg/release.tar.gz"
# Upload with glob patternbuildkite-agent artifact upload "dist/**/*"

Download

# Download to current directorybuildkite-agent artifact download "pkg/release.tar.gz" .
# Download from a specific stepbuildkite-agent artifact download "dist/*" . --step "build-step"

Search

# List matching artifactsbuildkite-agent artifact search "pkg/*.tar.gz" --build "$BUILDKITE_BUILD_ID"

For complete flag tables, see references/flag-reference.md.

For the declarative artifact_paths: YAML key, see the buildkite-pipelines skill. For bk artifact CLI commands, see the buildkite-cli skill.

Meta-data

A build-wide key-value store for sharing state between jobs. Set a value in one job, read it in any other job in the same build.

Set

buildkite-agent meta-data set "release-version" "1.4.2"

Get

VERSION=$(buildkite-agent meta-data get "release-version")

Use --default to return a fallback value instead of a non-zero exit when the key is missing: buildkite-agent meta-data get "deploy-env" --default "staging".

Check existence

# Returns exit code 0 if exists, 100 if notif buildkite-agent meta-data exists "release-version"; then  echo "Version already set"fi

Common patterns

Block step field values are stored automatically as meta-data. Retrieve them by field key:

# After a block step with fields: [{key: "release-name", text: "Release Name"}]RELEASE_NAME=$(buildkite-agent meta-data get "release-name")

Raw webhook payloads are available in webhook-triggered builds, while the webhook data remains cached. Read the special buildkite:webhook key when automation needs fields that are not promoted to first-class environment variables, such as the original pull request comment body.

WEBHOOK="$(buildkite-agent meta-data get "buildkite:webhook")"COMMENT_BODY="$(jq -r '.comment.body' <<< "$WEBHOOK")"

Pipeline Upload

Dynamically add steps to a running build. The core mechanism behind dynamic pipelines — generate YAML at runtime and upload it.

Basic usage

# Upload a specific filebuildkite-agent pipeline upload .buildkite/deploy-steps.yml
# Pipe generated YAML from stdin./scripts/generate-pipeline.sh | buildkite-agent pipeline upload

Replace mode

By default, uploaded steps are appended after the current step. Use --replace to replace the entire remaining pipeline:

# Replace all remaining steps with the uploaded onesbuildkite-agent pipeline upload --replace .buildkite/new-pipeline.yml

Key flags

For pipeline YAML syntax and step types, see the buildkite-pipelines skill.

OIDC Tokens

Request short-lived OpenID Connect tokens from within a job step for authenticating to external services (cloud providers, package registries) without static credentials.

Basic token request

# Request a token for a specific audienceTOKEN=$(buildkite-agent oidc request-token --audience "https://packages.buildkite.com/my-org/my-registry")

Cloud provider authentication

# AWS — request token with STS audienceTOKEN=$(buildkite-agent oidc request-token --audience "sts.amazonaws.com")

Key flags

For end-to-end OIDC auth flows, cloud provider setup, and token claim details, see the buildkite-secure-delivery skill.

Step Management

Read or modify step attributes at runtime. Useful for conditional logic within steps and build automation.

Get step attributes

# Get a step's label (--step is required)LABEL=$(buildkite-agent step get "label" --step "$BUILDKITE_STEP_KEY")
# Get another step's stateSTATE=$(buildkite-agent step get "state" --step "deploy-step")
# Get the entire step as JSONbuildkite-agent step get --step "test-step" --format json

Update step attributes

# Update a step's label dynamicallybuildkite-agent step update "label" ":rocket: Deploying v${VERSION}" --step "$BUILDKITE_STEP_KEY"
# Append to an existing labelbuildkite-agent step update "label" " (retried)" --step "build-step" --append

Cancel a step

# Cancel all unfinished jobs for a stepbuildkite-agent step cancel --step "optional-step"
# Force-cancel a running stepbuildkite-agent step cancel --step "stuck-step" --force

Key flags

Distributed Locks

Coordinate parallel jobs within a build using distributed mutex locks. Prevents race conditions when multiple jobs access shared resources.

Acquire / release pattern

#!/bin/bashset -euo pipefail
# Acquire lock — blocks until available, returns a tokentoken=$(buildkite-agent lock acquire "database-migration")trap 'buildkite-agent lock release "database-migration" "${token}"' EXIT
# Critical section — only one job runs this at a timebundle exec rails db:migrate

Do / done pattern (one-time setup)

Run a setup task exactly once across all parallel jobs:

#!/bin/bashecho "+++ Setting up shared test environment"
if [[ $(buildkite-agent lock do "test-env-setup") == "do" ]]; then  echo "Downloading test assets..."  curl -o /tmp/test-data.zip https://releases.example.com/data.zip  unzip /tmp/test-data.zip -d /tmp/shared-test-files/  buildkite-agent lock done "test-env-setup"else  echo "Assets already prepared by another job"fi
# All jobs continue hererun-tests.sh

Key flags

Environment

Inspect and modify the job's environment variables. Primarily useful for debugging lifecycle hooks and understanding what environment changes hooks made.

# Dump all environment variables as JSONbuildkite-agent env dump | jq .
# Get a specific variablebuildkite-agent env get "BUILDKITE_BRANCH"
# Set variables for subsequent hooks and the command (KEY=value format)buildkite-agent env set DEPLOY_TARGET=production "APP_NAME=My App"

Key flags

Debugging hooks

The env dump command is particularly useful in lifecycle hooks to see what prior hooks changed:

#!/bin/bash# .buildkite/hooks/pre-commandecho "--- Environment after environment hook:"buildkite-agent env dump | jq 'keys'

For agent lifecycle hooks and buildkite-agent.cfg configuration, see the agent hooks and agent configuration documentation.

Secrets

Retrieve cluster secrets at runtime from within job steps. Secrets retrieved this way are automatically added to the log redactor.

Basic usage

# Get a secret valueSECRET_VAR=$(buildkite-agent secret get "deploy-key")
# Pass directly to a toolcli-tool --token "$(buildkite-agent secret get "api-token")"

Key flags

Multiple secret keys can be requested at once: buildkite-agent secret get KEY1 KEY2 KEY3.

By default, secret get automatically registers retrieved values with the log redactor, masking them as [REDACTED] in subsequent output.

For setting up cluster secrets, see the Buildkite Secrets documentation. For the declarative secrets: pipeline YAML key, see the buildkite-pipelines skill.

Log Redaction

Add values to the build log redactor at runtime so they are masked in all subsequent output. Use this for dynamically-retrieved secrets that were not declared via secrets: or buildkite-agent secret get.

Basic usage

# Fetch a token from an external sourceDYNAMIC_TOKEN=$(curl -s https://vault.example.com/token)
# Register it with the redactor before using itecho "$DYNAMIC_TOKEN" | buildkite-agent redactor add
# Now any log output containing the token value shows [REDACTED]echo "Using token: $DYNAMIC_TOKEN"# Output: Using token: [REDACTED]

Multiple values

# Redact multiple valuesecho "$SECRET1" | buildkite-agent redactor addecho "$SECRET2" | buildkite-agent redactor add

When to use redactor vs secret get

Tool Signing

Sign pipeline YAML so that agents can verify step integrity before execution. The tool sign command takes a pipeline file (not individual steps) and annotates it with signatures.

Sign a pipeline from a file

# Sign a pipeline YAML file using a local JWKS keybuildkite-agent tool sign pipeline.yml \  --jwks-file /path/to/private-key.json \  --repo "[email protected]:org/repo.git"

Sign via the GraphQL API

# Retrieve, sign, and update a pipeline via the Buildkite GraphQL APIbuildkite-agent tool sign \  --graphql-token "$BUILDKITE_GRAPHQL_TOKEN" \  --organization-slug my-org \  --pipeline-slug my-pipeline \  --jwks-file /path/to/private-key.json \  --update

Generate a signing key pair

# Generate a new JWS key pair (private + public JWKS files)buildkite-agent tool keygen

Key flags

Note: There is no tool verify command. Signature verification is handled internally by the agent when it receives a job.

For pipeline signing configuration and rollout strategy, see the buildkite-secure-delivery skill.

Common Mistakes

Additional Resources

Reference Files

• references/flag-reference.md — Complete flag tables for all subcommands including upload, download, search, shasum, annotate, meta-data, pipeline upload, oidc, step, lock, env, secret, redactor, and tool
• references/patterns-and-recipes.md — Advanced multi-subcommand patterns: test failure annotation pipelines, cross-job state machines, OIDC-authenticated Docker push, parallel job coordination with locks, environment debugging

Further Reading

• Buildkite Docs for LLMs
• Agent CLI — annotate
• Agent CLI — artifact
• Agent CLI — meta-data
• Agent CLI — pipeline
• Agent CLI — OIDC
• Agent CLI — lock
• Managing secrets with Buildkite secrets
• Dynamic pipelines