Molt Fetch

molt fetch

Bulk data migration from source databases (PostgreSQL, MySQL, Oracle, MSSQL) to CockroachDB.

Basic Structure

molt fetch \  --source "<source-conn>" \  --target "<crdb-conn>" \  --bucket-path "s3://bucket/prefix"   # or --direct-copy or --local-path  [options]

Storage Backends (pick one)

Cloud auth: pass --use-implicit-auth for IAM/ADC/managed identity, or set AWS_ACCESS_KEY_ID/GOOGLE_APPLICATION_CREDENTIALS env vars.

Table Handling (--table-handling)

Import Mode

IMPORT INTO (default): Table goes OFFLINE during load. Highest throughput.

COPY FROM (--use-copy): Table stays ONLINE. Use with --direct-copy. Cannot use compression.

# Zero-downtime loadmolt fetch --source "..." --target "..." --direct-copy --use-copy

Key Flags

# Filtering--table-filter "customers|orders"      # POSIX regex for tables to include--table-exclusion-filter "temp_.*"     # exclude pattern--schema-filter "public"               # PostgreSQL only
# Performance--table-concurrency 4                  # parallel tables (default: 4)--export-concurrency 4                 # export threads (default: 4)--row-batch-size 100000                # rows per SELECT (default: 100k)
# Schema--type-map-file "types.json"           # custom type mappings--transformations-file "transforms.json"  # column exclusions, table aliases
# Logging--log-file "migration.log"             # or "stdout"--logging debug                        # info (default), debug, trace--metrics-listen-addr "0.0.0.0:3030"  # Prometheus scrape endpoint

Source-Specific Prerequisites

MySQL: GTID mode required (gtid_mode=ON, enforce_gtid_consistency=ON). ONLY_FULL_GROUP_BY must be off. Or use --ignore-replication-check.

Oracle: Binary must be built with CGO_ENABLED=1 -tags="cgo source_all". Oracle Instant Client in LD_LIBRARY_PATH.

PostgreSQL: Replication privileges needed, or --ignore-replication-check.

Common Workflows

1. Full migration with schema creation

drop-on-target-and-recreate drops any existing target tables first; run this only against a fresh or empty target.

molt fetch \  --source "postgresql://<user>:<password>@pg:5432/db" \  --target "postgresql://root@crdb:26257/db" \  --bucket-path "s3://mybucket/migration" \  --table-handling drop-on-target-and-recreate \  --table-filter "customers|orders|payments" \  --log-file migration.log

2. Resume after failure

# List available continuation tokensmolt fetch tokens --fetch-id "abc-123" --target "postgresql://root@crdb:26257/db"
# Resume all failed tablesmolt fetch \  --source "..." --target "..." \  --bucket-path "s3://mybucket/migration" \  --fetch-id "abc-123" \  --non-interactive

Error Recovery

Gotchas

• COPY mode: cannot use --compression gzip; must use --compression none (or omit, default is none with copy)
• Table is offline during IMPORT INTO — use --use-copy for zero downtime
• Schema changes between runs require starting from scratch
• --fetch-id continuation tokens live in the target's exceptions table
• For MySQL, --ignore-replication-check skips GTID validation but replication-dependent features won't work
• After fetch, run molt verify to confirm data integrity

See flags reference [blocked] for the full flag list.