Indexion Segment

作者 trkbt107ad5ad35a668無授權條款2 個星標收錄於 2026年10月8日更新於 2026年10月8日儲存庫2 週前更新

Split text into contextual chunks for RAG/embedding pipelines. Document segmentation and section extraction using window, tfidf, punctuation, or hybrid strategies chosen by intent.

僅含說明AI & Agents
AI 產生的概覽

使用視窗、TF-IDF、標點或混合策略將文字切分為上下文區塊,供 RAG 與嵌入流程使用。

功能
此技能會將文字文件切分為較小的上下文區塊,供檢索增強生成或嵌入流程使用。它提供視窗散度、TF-IDF、標點以及 NCD+TF-IDF 混合策略,並可調整區塊大小、閾值與視窗大小。輸出會寫入指定目錄,檔名前綴可設定。
適用情境
當需要為 RAG 或嵌入流程切分文字,或需要將文件拆分為有意義的章節時使用。它也適合為子文件層級的相似度分析準備文字。
執行需求
需要具備 segment 子命令的 indexion 命令列工具;此技能未隨附指令碼。輸入為文字檔,輸出為目錄。

indexion segment

Split text into contextual segments using divergence-based, TF-IDF, or punctuation strategies.

When to Use

  • User needs to chunk text for RAG or embedding pipelines
  • User wants to split a document into meaningful sections
  • User asks to segment text for processing
  • Preparing text for similarity analysis at sub-document level

Usage

bash
# Default window divergence strategyindexion segment <input-file> <output-dir>
# TF-IDF based segmentationindexion segment --strategy=tfidf <input-file> <output-dir>
# Punctuation-based segmentationindexion segment --strategy=punctuation <input-file> <output-dir>
# Custom segment sizesindexion segment --min-size=200 --max-size=3000 --target-size=800 document.txt output/
# Custom divergence thresholdindexion segment --threshold=0.5 document.txt output/
# Adaptive threshold mode (default)indexion segment --adaptive document.txt output/
# Hybrid NCD+TF-IDF modeindexion segment --hybrid --ncd-weight=0.6 --tfidf-weight=0.4 document.txt output/
# Custom window sizeindexion segment --window-size=5 document.txt output/
# Custom output prefixindexion segment --prefix=chunk document.txt output/

Options

OptionDefaultDescription
--strategy=NAMEwindowStrategy: window, tfidf, punctuation
--min-size=INT100Minimum segment characters
--max-size=INT2000Maximum segment characters
--target-size=INT500Target segment characters
--threshold=FLOAT0.42Divergence threshold
--window-size=INT3Window size
--adaptivetrueAdaptive threshold mode
--hybridfalseNCD+TF-IDF hybrid mode
--ncd-weight=FLOAT0.5NCD weight in hybrid mode
--tfidf-weight=FLOAT0.5TF-IDF weight in hybrid mode
--prefix=NAMEsegmentOutput file prefix

Strategies

StrategyDescription
window (default)Sliding window divergence detection
tfidfTF-IDF based topic change detection
punctuationPunctuation/sentence boundary based

Workflow

  1. Run indexion segment <input-file> <output-dir> to split text with defaults
  2. Adjust --threshold and --target-size to tune segmentation granularity
  3. Use --hybrid mode for better accuracy on mixed-content documents

來源與署名

來源:trkbt10/indexion-skills位於skills/indexion-segment提交7ad5ad3

授權條款: 無授權條款

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架