Managing Amazon Kinesis Data Streams

by aws188af2f810ceNo license2.8K starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated today

Operates Amazon Kinesis Data Streams (KDS). Covers streaming tables and Amazon S3 delivery - serverless, fully managed delivery from a KDS stream to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets. Includes channel setup, IAM, schemas, output key templates, CloudWatch metrics and alarms, DLQ handling, quotas, and troubleshooting. For all other KDS topics and questions, search AWS documentation and blogs instead. Triggers: Kinesis Data Streams, KDS, streaming tables, stream to S3, stream to Iceberg, stream to S3 Tables, KDS delivery, KDS channel, CreateChannel, data channel, data freshness, dead-letter queue, KDS lakehouse, serverless Kinesis delivery, Firehose alternative for KDS, zero-ops Kinesis to S3. DO NOT USE for Kinesis Data Firehose, Kinesis Video Streams, or Managed Service for Apache Flink — use dedicated skills or search documentation instead.

Instructions onlyDevOps & Cloud
AI-generated overview

Guides configuring Amazon Kinesis Data Streams channels that deliver stream data to S3 Tables or S3 buckets.

What it does
Provides domain guidance for Amazon Kinesis Data Streams streaming tables and Amazon S3 delivery, where a channel on a stream delivers records to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets. It routes the user to reference documents covering channel setup, IAM, schemas, output key templates, monitoring, quotas and troubleshooting. It also lists prerequisites and hard limits, such as ON_DEMAND streams, append-only delivery and same-Region destinations.
When to use it
Use it when setting up or operating KDS delivery to S3 or S3 Tables, building a lakehouse from Kinesis, or diagnosing a channel that is ACTIVE but not delivering data. It is not intended for Kinesis Data Firehose, Kinesis Video Streams or Managed Service for Apache Flink.
Requirements
Instructions only; no scripts are shipped. AWS operations are recommended through the AWS MCP server, or the AWS CLI or shell commands when that server is unavailable. Requires an ON_DEMAND Kinesis stream, an IAM service execution role trusted by kinesis.amazonaws.com, and a destination in the same Region.

Amazon Kinesis Data Streams

Overview

Domain expertise for Amazon Kinesis Data Streams. This skill covers streaming tables and Amazon S3 delivery, which provide serverless, fully managed delivery from a KDS stream directly to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets — no consumers, no Firehose, no Flink required for append-only delivery.

Delivery is configured as a channel on a stream via CreateChannel. Channels consume no shard capacity and no enhanced fan-out slots, so they do not compete with existing consumers.

The AWS MCP server is recommended for executing AWS operations. When the MCP server is not available, use the AWS CLI or shell commands instead.

Which Workflow Do You Need?

Customer IntentReference
Deliver stream data to Apache Iceberg tables on S3 Tables — setup, IAM, Glue schema, type mapping, partitioning, channel lifecyclereferences/streaming-tables.md [blocked]
Deliver stream data to a general-purpose S3 bucket as objects — setup, IAM, compression, storage class, output key templates, channel lifecyclereferences/general-purpose-delivery-to-s3.md [blocked]
Build a lakehouse / data lake from Kinesis; make streaming data queryable in Athenareferences/streaming-tables.md [blocked]
Zero-ops, serverless alternative to Firehose or Kinesis consumers for KDS → S3 deliveryreferences/general-purpose-delivery-to-s3.md [blocked]
CloudWatch metrics and alarms, delivery logging, CloudTrail, encryption, channel states, quotas, naming rules, what can be updatedreferences/monitoring-security-and-limits.md [blocked]
Channel is ACTIVE but no data arrives; records landing in the DLQ; rising data freshnessStart with references/monitoring-security-and-limits.md [blocked], then the destination-specific troubleshooting table
Deliver to both S3 and S3 Tables from one streamSupported — create one channel per destination

Prerequisites

Both destinations require an ON_DEMAND stream — provisioned-mode streams are not supported. Each channel also needs an IAM service execution role trusted by kinesis.amazonaws.com, and a destination in the same Region as the stream.

Hard Limits to Check First

These rule out the feature entirely, so confirm them before designing a solution:

  • Append-only — no CDC, upserts, or deletes
  • No schema evolution — a schema change means deleting and recreating the channel
  • No backfill — only records produced after the channel is ACTIVE are delivered
  • No transformations — records are delivered as-is
  • New table per channel — cannot deliver into an existing Iceberg table
  • Data freshness is 300–900 seconds — not sub-minute
  • Same Region only — cross-account is supported, cross-Region is not

If any of these block the customer, recommend Managed Service for Apache Flink instead.

Additional Resources

Source and attribution

Source:aws/agent-toolkit-for-awsinskills/specialized-skills/analytics-skills/managing-amazon-kinesis-data-streamsat commit188af2f

License: No license

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

More from aws/agent-toolkit-for-aws

Rds Oss

aws

Advises on Amazon RDS MySQL, MariaDB and PostgreSQL instance creation, upgrades, commitment pricing, RDS Proxy and Blue/Green deployments.

Includes scripts
DevOps & Cloud2.8Kupdated today

Exporting Rds To S3

aws

Guides exporting Amazon RDS or Aurora snapshots to Amazon S3 as Parquet, covering IAM, KMS, monitoring and verification.

DevOps & Cloud2.8Kupdated today

Creating Amazon Aurora Db Cluster With Instances

aws

Guides creation of an Amazon Aurora cluster with instances, including Secrets Manager password handling.

DevOps & Cloud2.8Kupdated today

Debugging Lambda Timeouts

aws

Diagnoses AWS Lambda timeout failures by analyzing configuration, CloudWatch logs and metrics, networking, cold starts, and dependencies.

DevOps & Cloud2.8Kupdated today

Waf

aws

Routes AWS WAF configuration tasks to per-task reference procedures for filtering web traffic.

Security2.8Kupdated today

Creating Api Gateway Stage

aws

Creates and configures AWS API Gateway stages with logging, tracing, throttling, WAF and IAM roles.

DevOps & Cloud2.8Kupdated today

More in DevOps & Cloud

Playwright Devops

microsoft

Featured

DevOps workflows for Playwright: analyze GitHub Actions failures for the last commit on main and fetch failed job logs.

Includes scripts
DevOps & Cloud97Kupdated today

M5 Onboard

anthropics

Featured

Provisions M5Stack ESP32 boards by detecting them on USB, flashing UIFlow 2.0 firmware, and installing a MicroPython app bundle.

DevOps & Cloud37Kupdated today

Yeet

openai

Featured

Stages, commits, pushes, and opens or updates a GitHub pull request in one flow using the GitHub CLI.

DevOps & Cloud27Kupdated 4 weeks ago

Runbook

anthropics

Featured

Creates or updates step-by-step operational runbooks for recurring tasks, including troubleshooting, rollback and escalation.

DevOps & Cloud27Kupdated today

Incident Response

anthropics

Featured

Guides an incident response workflow: severity triage, status updates, mitigation tracking, and blameless postmortems.

DevOps & Cloud27Kupdated today

Deploy Checklist

anthropics

Featured

Generates a pre-deployment readiness checklist covering pre-deploy, deploy, post-deploy and rollback triggers.

DevOps & Cloud27Kupdated today
Managing Amazon Kinesis Data Streams Agent Skill | SourceWeft