Aidp Spark Optimization
作者 oracle-samples90b42d6c24d4无许可证收录于 2026年10月8日更新于 2026年10月8日
Use when a Spark job/notebook is slow, missing an SLA, spilling, OOMing, generating too many small files, or shuffling/skewing heavily; when reviewing Spark/PySpark code or a Spark UI for performance; or before running a large Spark workload. Covers open-source Apache Spark 3.5.0 (+ Delta Lake) tuning -- partitions, shuffle, joins, skew, file layout, memory, codegen, caching, AQE, compression, and configuration.
仅公开文件列表。将技能安装到工作区后即可查看文件内容。
| 路径 | 大小 | 类型 |
|---|---|---|
| references/01-partitioning.md | 21.8 KB | text/markdown |
| references/02-joins.md | 24.8 KB | text/markdown |
| references/03-file-layout-io.md | 22 KB | text/markdown |
| references/04-memory-and-spill.md | 16 KB | text/markdown |
| references/05-codegen.md | 15.4 KB | text/markdown |
| references/06-caching-materialization.md | 6.8 KB | text/markdown |
| references/07-aqe.md | 21.1 KB | text/markdown |
| references/08-delta-lake.md | 18 KB | text/markdown |
| references/09-oracle-database.md | 23.1 KB | text/markdown |
| references/aidp-notes.md | 3.9 KB | text/markdown |
| references/case-studies.md | 6.9 KB | text/markdown |
| references/cluster-sizing.md | 6.6 KB | text/markdown |
| references/config-matrix.md | 13.2 KB | text/markdown |
| references/diagnosis.md | 5.3 KB | text/markdown |
| references/quick-reference.md | 6.1 KB | text/markdown |
| SKILL.md | 10.2 KB | text/markdown |
来源与署名
来源：oracle-samples/oracle-aidp-samples位于ai/claude-code-plugins/oracle-ai-data-platform-workbench-engineer-agent/skills/aidp-spark-optimization提交90b42d6
许可证： 无许可证
内容归原作者所有。SourceWeft 从公开仓库中收录这些内容。
更多来自 oracle-samples/oracle-aidp-samples 的技能
Aidp Workspace Admin
oracle-samples
Provision and inspect AIDP DataLake instances and workspaces, including private-network workspaces attached to a customer VCN/subnet. Use when the user wants to create/list/get a workspace or DataLake instance, set up a new (e.g. private) AIDP environment, or replicate a customer setup. Create/delete are guarded — confirm before any provisioning.
Aidp Volumes
oracle-samples
Work with AIDP volumes — list volumes, browse files inside a volume, upload/download via the PAR flow, and create directories. Use when the user mentions volumes, needs to stage large/binary files, or move data in/out of a volume (distinct from the workspace filesystem). Control-plane via the official `aidp` CLI.
Aidp Verified Queries
oracle-samples
维护经过验证的问题到 Spark SQL 配对库，让智能体在生成新 SQL 前优先复用可信查询。
Aidp User Settings
oracle-samples
通过 aidp CLI 或 oci raw-request 备用方式管理 AIDP DataLake 用户设置与偏好。
Aidp Semantic Model
oracle-samples
维护 .aidp/semantic.md 业务语义层，定义指标、连接、同义词和值字典，为自然语言转 SQL 提供依据。
Aidp Roles Access
oracle-samples
管理 AIDP 角色与访问权限：列出角色、增删成员，以及授予或撤销各资源的权限。
更多Data & Analytics技能
Spanner Basics
指导 Google Cloud Spanner 的实例与数据库管理、架构设计、查询和性能诊断。
Gke Cost Analysis
使用 BigQuery 账单导出数据和实时集群指标，回答关于 GKE 集群与工作负载成本的自然语言问题。
Datalineage Bigquery Asset Impact Analysis
指导智能体使用数据血缘对 BigQuery 表或视图进行下游影响（爆炸半径）分析。
Bigquery Troubleshooting
通过结构化根因工作流诊断失败、缓慢或费用异常的 BigQuery 作业。
Bigquery Optimization
指导 BigQuery 在容量版本、存储布局和 SQL 查询方面的成本与性能优化。
Bigquery Bigframes
指导使用 BigQuery DataFrames（BigFrames）编写 Python 代码，用于数据处理、分析和机器学习。