Managing Amazon Kinesis Data Streams

aws/agent-toolkit-for-aws/skills/specialized-skills/analytics-skills/managing-amazon-kinesis-data-streams

作者 aws188af2f810ce無授權條款2.8K 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫今天更新

Operates Amazon Kinesis Data Streams (KDS). Covers streaming tables and Amazon S3 delivery - serverless, fully managed delivery from a KDS stream to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets. Includes channel setup, IAM, schemas, output key templates, CloudWatch metrics and alarms, DLQ handling, quotas, and troubleshooting. For all other KDS topics and questions, search AWS documentation and blogs instead. Triggers: Kinesis Data Streams, KDS, streaming tables, stream to S3, stream to Iceberg, stream to S3 Tables, KDS delivery, KDS channel, CreateChannel, data channel, data freshness, dead-letter queue, KDS lakehouse, serverless Kinesis delivery, Firehose alternative for KDS, zero-ops Kinesis to S3. DO NOT USE for Kinesis Data Firehose, Kinesis Video Streams, or Managed Service for Apache Flink — use dedicated skills or search documentation instead.

僅含說明DevOps & Cloud
AI 產生的概覽

指導設定 Amazon Kinesis Data Streams 通道,將串流資料傳送到 S3 Tables 或 S3 儲存貯體。

功能
為 Amazon Kinesis Data Streams 的串流資料表與 Amazon S3 傳送提供領域指引,也就是透過串流上的通道將記錄傳送到 S3 Tables 上的 Apache Iceberg 資料表或一般用途 S3 儲存貯體。它會將使用者導向參考文件,內容涵蓋通道設定、IAM、結構描述、輸出鍵範本、監控、配額與疑難排解。它也列出先決條件與硬性限制,例如 ON_DEMAND 串流、僅附加傳送,以及同區域的目的地。
適用情境
適用於設定或維運 KDS 到 S3 或 S3 Tables 的傳送、以 Kinesis 建置湖倉,或診斷處於 ACTIVE 狀態但未傳送資料的通道。不適用於 Kinesis Data Firehose、Kinesis Video Streams 或 Managed Service for Apache Flink。
執行需求
僅為說明文件,未隨附指令碼。建議透過 AWS MCP 伺服器執行 AWS 操作;該伺服器無法使用時,可改用 AWS CLI 或 shell 指令。需要 ON_DEMAND 模式的 Kinesis 串流、受 kinesis.amazonaws.com 信任的 IAM 服務執行角色,以及位於相同區域的目的地。

Amazon Kinesis Data Streams

Overview

Domain expertise for Amazon Kinesis Data Streams. This skill covers streaming tables and Amazon S3 delivery, which provide serverless, fully managed delivery from a KDS stream directly to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets — no consumers, no Firehose, no Flink required for append-only delivery.

Delivery is configured as a channel on a stream via CreateChannel. Channels consume no shard capacity and no enhanced fan-out slots, so they do not compete with existing consumers.

The AWS MCP server is recommended for executing AWS operations. When the MCP server is not available, use the AWS CLI or shell commands instead.

Which Workflow Do You Need?

Customer IntentReference
Deliver stream data to Apache Iceberg tables on S3 Tables — setup, IAM, Glue schema, type mapping, partitioning, channel lifecyclereferences/streaming-tables.md [blocked]
Deliver stream data to a general-purpose S3 bucket as objects — setup, IAM, compression, storage class, output key templates, channel lifecyclereferences/general-purpose-delivery-to-s3.md [blocked]
Build a lakehouse / data lake from Kinesis; make streaming data queryable in Athenareferences/streaming-tables.md [blocked]
Zero-ops, serverless alternative to Firehose or Kinesis consumers for KDS → S3 deliveryreferences/general-purpose-delivery-to-s3.md [blocked]
CloudWatch metrics and alarms, delivery logging, CloudTrail, encryption, channel states, quotas, naming rules, what can be updatedreferences/monitoring-security-and-limits.md [blocked]
Channel is ACTIVE but no data arrives; records landing in the DLQ; rising data freshnessStart with references/monitoring-security-and-limits.md [blocked], then the destination-specific troubleshooting table
Deliver to both S3 and S3 Tables from one streamSupported — create one channel per destination

Prerequisites

Both destinations require an ON_DEMAND stream — provisioned-mode streams are not supported. Each channel also needs an IAM service execution role trusted by kinesis.amazonaws.com, and a destination in the same Region as the stream.

Hard Limits to Check First

These rule out the feature entirely, so confirm them before designing a solution:

  • Append-only — no CDC, upserts, or deletes
  • No schema evolution — a schema change means deleting and recreating the channel
  • No backfill — only records produced after the channel is ACTIVE are delivered
  • No transformations — records are delivered as-is
  • New table per channel — cannot deliver into an existing Iceberg table
  • Data freshness is 300–900 seconds — not sub-minute
  • Same Region only — cross-account is supported, cross-Region is not

If any of these block the customer, recommend Managed Service for Apache Flink instead.

Additional Resources

來源與署名

來源:aws/agent-toolkit-for-aws位於skills/specialized-skills/analytics-skills/managing-amazon-kinesis-data-streams提交188af2f

授權條款: 無授權條款

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架