Databricks Spark Structured Streaming
by databrickse77e37e8a4daNo license345 starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated today
Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, working with Kafka ingestion, implementing Real-Time Mode (RTM), configuring triggers (processingTime, availableNow), handling stateful operations with watermarks, optimizing checkpoints, performing stream-stream or stream-static joins, writing to multiple sinks, or tuning streaming cost and performance.
Only the file list is public. File contents are available once the skill is installed in a workspace.
| Path | Size | Type |
|---|---|---|
| agents/openai.yaml | 414 B | application/yaml |
| assets/databricks.png | 15 KB | image/png |
| assets/databricks.svg | 582 B | text/plain |
| references/checkpoint-best-practices.md | 8.4 KB | text/markdown |
| references/kafka-streaming.md | 16 KB | text/markdown |
| references/lakebase-sink-python.md | 20.6 KB | text/markdown |
| references/merge-operations.md | 10.7 KB | text/markdown |
| references/multi-sink-writes.md | 12.6 KB | text/markdown |
| references/real-time-mode.md | 18.8 KB | text/markdown |
| references/stateful-operations.md | 11.7 KB | text/markdown |
| references/streaming-best-practices.md | 9.4 KB | text/markdown |
| references/stream-static-joins.md | 13.9 KB | text/markdown |
| references/stream-stream-joins.md | 16.5 KB | text/markdown |
| references/trigger-and-cost-optimization.md | 13.6 KB | text/markdown |
| SKILL.md | 3.7 KB | text/markdown |
Source and attribution
Source:databricks/databricks-agent-skillsinplugins/databricks/claude/skills/databricks-spark-structured-streamingat commite77e37e
License: No license
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from databricks/databricks-agent-skills
Databricks Unstructured Pdf Generation
databricks
Builds synthetic PDF documents plus paired test questions on Databricks for RAG and Knowledge Assistant retrieval evaluation.
Databricks Setup Local
databricks
Guides previewing, provisioning, or diagnosing a uv-managed local Python .venv for Databricks via the CLI setup-local command.
Databricks Dbsql
databricks
Reference guidance for advanced Databricks SQL features, including SQL scripting, materialized views, geospatial functions, and AI functions.
Databricks Data Discovery
databricks
Routes Databricks data discovery, natural-language data questions, and SQL generation to Genie One via the Databricks CLI.
Databricks Dabs
databricks
Guides creating, configuring, validating, deploying and running Databricks Declarative Automation Bundles (DABs).
Databricks App Design
databricks
Designs the UX of custom-code Databricks Apps data screens and maps them to concrete AppKit components.
More in Data & Analytics
Spanner Basics
Guides Google Cloud Spanner administration, schema design, querying and performance diagnosis.
Gke Cost Analysis
Answers natural-language questions about GKE cluster and workload costs using BigQuery billing exports and live cluster metrics.
Datalineage Bigquery Asset Impact Analysis
Guides an agent through downstream impact (blast radius) analysis for a BigQuery table or view using Data Lineage.
Bigquery Troubleshooting
Diagnoses failing, slow, or unexpectedly expensive BigQuery jobs through structured root-cause workflows.
Bigquery Optimization
Guides BigQuery cost and performance optimization across capacity editions, storage layout, and SQL queries.
Bigquery Bigframes
Guides writing Python code with BigQuery DataFrames (BigFrames) for data processing, analysis, and machine learning.