A/B Test Sample Size Calculator MCP

io.github.theluckystrike由 GitHub 帳號 theluckystrike 發布v1.0.0更新於 Nov 3, 2026

精確的A/B測試樣本量、時長、z檢驗、卡方檢驗、Fisher-Freeman-Halton檢驗以及偷看風險。

AI 翻譯由 AI 模型根據下方的原文描述翻譯，可能有誤。

原文描述

Exact A/B test sample size, duration, z-test, chi-square, Fisher-Freeman-Halton and peeking risk.

發布方自己的服務：Calculator Tools MCP

已驗證Streamable HTTP可網頁執行

概覽

A/B Test Sample Size Calculator 的託管 MCP 服務（ab-test-sample-size-calculator-mcp.lipmichal.workers.dev）；無需登入；Streamable HTTP；5 個工具。

暫無 AI 概覽：以下資訊來自註冊中心條目和服務本身。

端點
ab-test-sample-size-calculator-mcp.lipmichal.workers.dev
最近檢查

安裝

在 SourceWeft 中

  1. 開啟 儀表板中的 A/B Test Sample Size Calculator MCP，將其新增到工作區。
  2. 為需要使用其工具的對話啟用該服務。

Web executable，透過 Streamable HTTP。 遠端服務在工作區中設定後即可從網頁執行環境執行。

交給代理程式安裝

把這段提示詞貼上到 Claude Code、Codex、Cursor 等代理程式，或 SourceWeft 對話中。代理程式會讀取該伺服器的安裝指南，先向你展示發布者、端點和登入方式，經你同意後再把它新增到你的客戶端。金鑰不會寫進任何命令。

閱讀 https://sourceweft.com/mcp/io.github.theluckystrike%2Fab-test-sample-size-calculator-mcp/install.md，並按其中說明把這個 MCP 伺服器接入我的客戶端。修改任何設定之前，先告訴我它的發布者、端點和登入方式，等我確認。不要把 API 金鑰寫進命令或我會提交的檔案；問我金鑰應從哪裡取得。

檢視代理程式所依據的安裝指南

其他 MCP 客戶端

把它新增到你客戶端的 mcpServers 設定中。

{
  "mcpServers": {
    "a-b-test-sample-size-calculator": {
      "type": "http",
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

README

ab-test-sample-size-calculator-mcp

A/B test sample size calculator and significance tests for AI assistants, with exact answers instead of mental arithmetic.

This is a Model Context Protocol (MCP) server. It works out how many users an A/B test needs, how long it must run, whether a result is significant (z-test, chi-square, Fisher exact), the Fisher-Freeman-Halton exact test for larger tables, and how much peeking inflates false positives. It needs no API key and makes no network calls.

Tools

ToolWhat it returns
calculate_sample_sizeUsers per variant for a conversion-rate test: baseline, relative or absolute MDE, alpha, power, one or two-sided, several variants with optional Bonferroni, unequal splits. Also the figure Evan Miller's calculator gives for the same inputs.
estimate_test_durationDays and whole weeks to reach the sample size, from daily visitors, traffic share and number of variants.
test_ab_significanceTwo-proportion z-test, p-value, confidence interval, relative lift, chi-square with and without Yates, Fisher exact p and a sample ratio mismatch check.
fisher_exact_testExact p-value for a 2x2 table (two-sided, greater, less) or any r x c table (Fisher-Freeman-Halton), next to Pearson chi-square and an expected-count check.
check_peeking_riskThe real false positive rate when a test is checked K times, and Pocock and O'Brien-Fleming boundaries for up to 10 looks.

Every result has structured JSON, a one-paragraph summary, as_of, source (the method references) and more (the matching calculator on abwex.com).

What it knows that your model does not

  • Exact inverse normal quantiles and the full sample size formula. Models round z-values, drop the pooled variance term or confuse relative and absolute MDE, and the answer moves by thousands of users.
  • Why calculators disagree. The result shows both the standard pooled formula (as R power.prop.test) and Evan Miller's published formula, so a 37,513 vs 36,314 gap has an explanation.
  • Exact hypergeometric sums. A Fisher-Freeman-Halton p-value for a 4 x 4 table needs millions of tables summed; it cannot be estimated in your head.
  • Sequential testing numbers: checking 5 times at p < 0.05 gives a 14.2% false positive rate, and the boundaries that fix it.

Install

Hosted (Claude.ai custom connector, ChatGPT developer mode)

Paste this URL. No token, no sign-up.

https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Claude Code

claude mcp add --transport http ab-test-calculator https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Or run it locally over stdio:

claude mcp add ab-test-calculator -- npx -y github:theluckystrike/ab-test-sample-size-calculator-mcp

Claude Desktop

Add to claude_desktop_config.json:

json
{  "mcpServers": {    "ab-test-calculator": { "command": "npx", "args": ["-y", "github:theluckystrike/ab-test-sample-size-calculator-mcp"] }  }}

Cursor

Add to ~/.cursor/mcp.json:

json
{  "mcpServers": {    "ab-test-calculator": { "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp" }  }}

VS Code

Add to .vscode/mcp.json:

json
{  "servers": {    "ab-test-calculator": { "type": "http", "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp" }  }}

Node 20 or newer is needed for the local stdio version.

Example prompts

  • "Our checkout converts at 4.2%. How many users per variant to detect a 10% relative lift with 80% power?"
  • "We get 4,000 visitors a day and want to test 3 variants. How many weeks will it take?"
  • "Control had 420 conversions from 10,000, the variant 495 from 10,050. Is that significant?"
  • "Run a Fisher-Freeman-Halton test on [[12, 30, 8], [20, 25, 15]]."
  • "If I check the dashboard every day for 10 days, what is my real false positive rate?"

Data sources

No live data. Methods and the published values the tests check against are in DATA_SOURCES.md: R power.prop.test and fisher.test documentation outputs, Evan Miller's calculator code, scipy and statsmodels reference values (scripts/reference.py), Armitage et al. (1969) and Jennison and Turnbull (2000).

Limitations

  • Sample size covers conversion rates (two proportions). Continuous metrics such as revenue per user need the standard deviation and are not covered.
  • Sample size and duration assume a fixed-horizon test analysed once. For several looks, plan with check_peeking_risk.
  • The Fisher-Freeman-Halton test enumerates tables exactly. Very large or sparse r x c tables can exceed the work limit (lower on the hosted server than locally). The tool then returns the chi-square result with a note instead of an exact p.
  • Peeking boundaries assume equally spaced looks and two-sided tests; boundaries are given for up to 10 looks.
  • The confidence interval for the difference is the Wald interval, which is poor for rates near 0 or 1 with small samples.

More

Interactive versions of these calculators and guides are on abwex.com, including the Fisher-Freeman-Halton calculator and the peeking inflation calculator.

Part of the Zovo MCP collection: https://mcp.zovo.one

Development

npm installnpm test                      # builds, then runs the fixture testspython3 scripts/reference.py  # recomputes the reference values with scipy and statsmodelsnode scripts/smoke-stdio.mjs  # stdio check, one call per toolnpx wrangler deploy           # Cloudflare Worker, Streamable HTTP at /mcp

License

MIT

More data servers from the same author

Each one is free, needs no API key and has a hosted URL you can paste into Claude or ChatGPT.

servertoolshosted URL
cups-to-grams-mcpconvert_ingredient, search_ingredients, scale_recipe, convert_oven_temperaturehttps://cups-to-grams-mcp.lipmichal.workers.dev/mcp
dscr-loan-calculator-mcpcalculate_dscr, max_loan_for_target_dscr, rent_needed_for_target_dscr, compare_scenarios, amortization_summaryhttps://dscr-loan-calculator-mcp.lipmichal.workers.dev/mcp
gold-price-per-gram-mcpget_gold_price_per_gram, get_metal_prices, calculate_jewelry_value, convert_gold_unitshttps://gold-price-per-gram-mcp.lipmichal.workers.dev/mcp
lake-level-mcpsearch_lakes, get_lake_level, get_lake_history, compare_lakeshttps://lake-level-mcp.lipmichal.workers.dev/mcp
minimum-wage-mcpget_minimum_wage, list_upcoming_increases, compare_states, annual_income_at_minimum_wagehttps://minimum-wage-mcp.lipmichal.workers.dev/mcp
no-tax-on-overtime-mcpcalculate_overtime_deduction, calculate_tips_deduction, calculate_senior_deduction, estimate_tax_savings, explain_eligibilityhttps://no-tax-on-overtime-mcp.lipmichal.workers.dev/mcp
paycheck-calculator-mcpcalculate_take_home_pay, compare_states_take_home, salary_to_hourly, hourly_to_salary, get_federal_tax_bracketshttps://paycheck-calculator-mcp.lipmichal.workers.dev/mcp
uk-income-tax-salary-mcpcalculate_uk_take_home, get_uk_tax_bands, compare_salaries, marginal_rate_at_incomehttps://uk-income-tax-salary-mcp.lipmichal.workers.dev/mcp
us-federal-holidays-business-days-mcplist_federal_holidays, is_business_day, add_business_days, count_business_days, invoice_due_datehttps://us-federal-holidays-business-days-mcp.lipmichal.workers.dev/mcp
aca-premium-tax-credit-calculator-mcpget_federal_poverty_level, calculate_premium_tax_credit, check_subsidy_cliff, get_aptc_repayment_caphttps://aca-premium-tax-credit-calculator-mcp.lipmichal.workers.dev/mcp
ap-score-distributions-mcpget_ap_score_distribution, estimate_ap_percentile, compare_ap_exams, list_ap_exams, estimate_ap_score_from_compositehttps://ap-score-distributions-mcp.lipmichal.workers.dev/mcp
car-loan-interest-deduction-calculator-mcpcalculate_car_loan_interest_deduction, calculate_salt_deduction, get_tax_brackets_and_standard_deduction, calculate_child_tax_credithttps://car-loan-interest-deduction-calculator-mcp.lipmichal.workers.dev/mcp
estate-gift-tax-exemption-calculator-mcpget_exemption_amounts, calculate_estate_tax, check_gift, compare_yearshttps://estate-gift-tax-exemption-calculator-mcp.lipmichal.workers.dev/mcp
recipe-nutrition-calculator-mcplookup_ingredient_nutrition, calculate_recipe_nutrition, round_nutrition_label, scale_recipe_nutritionhttps://recipe-nutrition-calculator-mcp.lipmichal.workers.dev/mcp
retirement-contribution-limits-2027-mcpget_contribution_limits, calculate_max_contribution, get_social_security_cola, compare_yearshttps://retirement-contribution-limits-2027-mcp.lipmichal.workers.dev/mcp
section-179-deduction-calculator-mcpcalculate_section_179, calculate_bonus_depreciation, get_vehicle_depreciation_limits, calculate_asset_loan_paymenthttps://section-179-deduction-calculator-mcp.lipmichal.workers.dev/mcp
student-loan-repayment-assistance-plan-calculator-mcpcalculate_rap_payment, compare_rap_vs_standard, get_federal_loan_limits, get_plan_transition_dateshttps://student-loan-repayment-assistance-plan-calculator-mcp.lipmichal.workers.dev/mcp
subdomain-finder-mcpfind_subdomains, resolve_subdomains, get_certificate_history, summarize_attack_surfacehttps://subdomain-finder-mcp.lipmichal.workers.dev/mcp
uk-statutory-sick-pay-maternity-pay-calculator-mcpcalculate_statutory_sick_pay, calculate_statutory_maternity_pay, get_minimum_wage_rates, check_eligibilityhttps://uk-statutory-sick-pay-maternity-pay-calculator-mcp.lipmichal.workers.dev/mcp

The paid back-office collection (invoices, VAT, PDFs, spreadsheets) is at https://mcp.zovo.one

來源：README.md，提交 a7040c5

工具

5

calculate_sample_size

A/B test sample size calculator

Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two can be reconciled.

輸入參數mde必填alphapowersidedmde_typevariantsbonferronibaseline_rate必填allocation_ratio

estimate_test_duration

A/B test duration estimator

How many days an A/B test must run: from a known sample size per variant, or from baseline rate and MDE (computed as in calculate_sample_size), plus daily visitors and the share of traffic in the test. Returns exact days and the recommended whole weeks (to cover weekday effects).

輸入參數mdealphapowermde_typevariantsbaseline_ratetraffic_sharedaily_visitors必填sample_size_per_variant

test_ab_significance

A/B test significance calculator

Is an A/B test result statistically significant? From visitors and conversions per group: two-proportion z-test (pooled), one or two-sided p-value, Wald confidence interval for the difference, relative lift, Pearson chi-square with and without Yates correction (R prop.test), Fisher exact p for the same 2x2 table, and a sample ratio mismatch (SRM) check against the planned split.

輸入參數alphaalternativeexpected_splitcontrol_visitors必填variant_visitors必填control_conversions必填variant_conversions必填

fisher_exact_test

Fisher exact and Fisher-Freeman-Halton test

Exact p-value for a contingency table of counts. 2x2: Fisher exact test (two-sided, greater or less, as R fisher.test). Larger r x c tables (for example 3 variants x converted/not, or 4 x 4 survey tables): Fisher-Freeman-Halton exact test by a network algorithm. Also returns Pearson chi-square and how many cells have expected counts below 5, so you can see when the chi-square approximation is unsafe.

輸入參數table必填alternative

check_peeking_risk

A/B test peeking and sequential testing risk

What peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.

輸入參數alphalooks必填

版本歷史

1
  1. v1.0.0最新Nov 3, 2026

評分

已滿足適用檢查項的 100%

該評分衡量此收錄條目的完整程度，依據註冊中心條目、端點與公開來源計算得出。它不是對伺服器本身的評價，也不表示伺服器安全或目前可用：可連通性是帶日期的檢查結果，評分也不包含安裝量或使用者評價。

  • 近 30 天內可連通必需25 分
  • 已宣告認證方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 圖示10 分
  • 文件15 分
  • 授權條款10 分

其他 Calculator 整合

Calculator Tools MCP

tools.calculator官方發布者

一個遠端 MCP 端點，讓助理從 Cursor、Claude 或 ChatGPT 發布可用的計算機、追蹤器、工具或遊戲。

Streamable HTTP可網頁執行
效率與工作流尚未收錄Oct 25, 2026

Free IFTA Calculator

dev.pages.ifta-calculator官方發布者

唯讀工具，回答美國／加拿大 IFTA 燃油稅問題，並依各轄區里程與加侖數估算季度稅額。

Streamable HTTP可網頁執行
金融1 個工具Oct 29, 2026

Complexity Cost Calculator

由 GitHub 帳號 beamersaas 發布

遠端 MCP 伺服器，用來估算軟體複雜度的成本，並預測 AI 採用於未來幾季如何改變工程成本。

Streamable HTTP可網頁執行驗證方式
資料與分析2 個工具Nov 6, 2026

UK Statutory Sick Pay and Maternity Pay

由 GitHub 帳號 theluckystrike 發布

UK Statutory Sick Pay (April 2026 rules), Maternity Pay and National Living Wage from gov.uk.

Streamable HTTP可網頁執行
4 個工具Nov 3, 2026

Student Loan RAP Calculator

由 GitHub 帳號 theluckystrike 發布

學生貸款RAP還款計算器、2026年聯邦貸款限額、SAVE/PAYE/ICR終止日期（OBBBA）。AI 翻譯

Streamable HTTP可網頁執行
4 個工具Nov 3, 2026