Databricks Spark Structured Streaming
by databrickse77e37e8a4daNo license345 starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated today
Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, working with Kafka ingestion, implementing Real-Time Mode (RTM), configuring triggers (processingTime, availableNow), handling stateful operations with watermarks, optimizing checkpoints, performing stream-stream or stream-static joins, writing to multiple sinks, or tuning streaming cost and performance.
Add to a SourceWeft workspace
- Open the skill in your dashboard and add it to a workspace.
- Enable it for the chats that should use it.
This skill is instructions only: it ships no scripts to execute.
Add to SourceWeftYou will be asked to sign in first, then taken straight to this skill.
Ask your agent to install it
Paste this prompt into Claude Code, Codex, Cursor or another agent that can run commands — or into SourceWeft chat. The agent reads this skill's install guide, shows you its source, license and scripts, and installs it with the SourceWeft CLI once you agree.
Read https://sourceweft.com/skills/gh-databricks-databricks-agent-skills-databricks-spark-structured-streaming/install.md and install the skill it describes. Before installing, show me its source, license and whether it ships scripts, and wait for my OK. Ask me before changing anything else on my machine.Install it yourself from a terminal
For Claude Code, Codex, Cursor and other local agents. The SourceWeft CLI fetches the skill from its source repository at the commit scanned here, and verifies every file against the hashes recorded when the skill was scanned. If anything differs, nothing is written.
npx @sourceweft/cli skills install gh-databricks-databricks-agent-skills-databricks-spark-structured-streamingAdd --agent claude-code, codex, cursor or universal to choose which agent gets it (Claude Code by default).
Upstream installer — not verified by SourceWeft
The open-source skills installer fetches the same pinned commit, but does not check the files against the hashes SourceWeft recorded.
npx skills add https://github.com/databricks/databricks-agent-skills/tree/e77e37e8a4dabbe2662b680c180bb72a05eca48d/plugins/databricks/claude/skills/databricks-spark-structured-streamingSource and attribution
Source:databricks/databricks-agent-skillsinplugins/databricks/claude/skills/databricks-spark-structured-streamingat commite77e37e
License: No license
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from databricks/databricks-agent-skills
Databricks Unstructured Pdf Generation
databricks
Builds synthetic PDF documents plus paired test questions on Databricks for RAG and Knowledge Assistant retrieval evaluation.
Databricks Setup Local
databricks
Guides previewing, provisioning, or diagnosing a uv-managed local Python .venv for Databricks via the CLI setup-local command.
Databricks Dbsql
databricks
Reference guidance for advanced Databricks SQL features, including SQL scripting, materialized views, geospatial functions, and AI functions.
Databricks Data Discovery
databricks
Routes Databricks data discovery, natural-language data questions, and SQL generation to Genie One via the Databricks CLI.
Databricks Dabs
databricks
Guides creating, configuring, validating, deploying and running Databricks Declarative Automation Bundles (DABs).
Databricks App Design
databricks
Designs the UX of custom-code Databricks Apps data screens and maps them to concrete AppKit components.
More in Data & Analytics
Spanner Basics
Guides Google Cloud Spanner administration, schema design, querying and performance diagnosis.
Gke Cost Analysis
Answers natural-language questions about GKE cluster and workload costs using BigQuery billing exports and live cluster metrics.
Datalineage Bigquery Asset Impact Analysis
Guides an agent through downstream impact (blast radius) analysis for a BigQuery table or view using Data Lineage.
Bigquery Troubleshooting
Diagnoses failing, slow, or unexpectedly expensive BigQuery jobs through structured root-cause workflows.
Bigquery Optimization
Guides BigQuery cost and performance optimization across capacity editions, storage layout, and SQL queries.
Bigquery Bigframes
Guides writing Python code with BigQuery DataFrames (BigFrames) for data processing, analysis, and machine learning.