A/B Test Sample Size Calculator MCP

io.github.theluckystrike由 GitHub 账号 theluckystrike 发布v1.0.0更新于 Nov 3, 2026

精确的A/B测试样本量、时长、z检验、卡方检验、Fisher-Freeman-Halton检验以及偷看风险。

AI 翻译由 AI 模型根据下方的原文描述翻译，可能有误。

原文描述

Exact A/B test sample size, duration, z-test, chi-square, Fisher-Freeman-Halton and peeking risk.

发布方自己的服务：Calculator Tools MCP

已验证Streamable HTTP可网页运行

概览

A/B Test Sample Size Calculator 的托管 MCP 服务（ab-test-sample-size-calculator-mcp.lipmichal.workers.dev）；无需登录；Streamable HTTP；5 个工具。

暂无 AI 概览：以下信息来自注册中心条目和服务本身。

端点
ab-test-sample-size-calculator-mcp.lipmichal.workers.dev
最近检查

安装

在 SourceWeft 中

  1. 打开 控制台中的 A/B Test Sample Size Calculator MCP，将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Web executable，通过 Streamable HTTP。 远程服务在工作区中配置后即可从网页运行时运行。

交给智能体安装

把这段提示词粘贴到 Claude Code、Codex、Cursor 等智能体，或 SourceWeft 对话中。智能体会读取该服务器的安装指南，先向你展示发布者、端点和登录方式，经你同意后再把它添加到你的客户端。密钥不会写进任何命令。

阅读 https://sourceweft.com/mcp/io.github.theluckystrike%2Fab-test-sample-size-calculator-mcp/install.md，并按其中说明把这个 MCP 服务器接入我的客户端。修改任何配置之前，先告诉我它的发布者、端点和登录方式，等我确认。不要把 API 密钥写进命令或我会提交的文件；问我密钥应从哪里获取。

查看智能体所依据的安装指南

其他 MCP 客户端

把它添加到你客户端的 mcpServers 配置中。

{
  "mcpServers": {
    "a-b-test-sample-size-calculator": {
      "type": "http",
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

README

ab-test-sample-size-calculator-mcp

A/B test sample size calculator and significance tests for AI assistants, with exact answers instead of mental arithmetic.

This is a Model Context Protocol (MCP) server. It works out how many users an A/B test needs, how long it must run, whether a result is significant (z-test, chi-square, Fisher exact), the Fisher-Freeman-Halton exact test for larger tables, and how much peeking inflates false positives. It needs no API key and makes no network calls.

Tools

ToolWhat it returns
calculate_sample_sizeUsers per variant for a conversion-rate test: baseline, relative or absolute MDE, alpha, power, one or two-sided, several variants with optional Bonferroni, unequal splits. Also the figure Evan Miller's calculator gives for the same inputs.
estimate_test_durationDays and whole weeks to reach the sample size, from daily visitors, traffic share and number of variants.
test_ab_significanceTwo-proportion z-test, p-value, confidence interval, relative lift, chi-square with and without Yates, Fisher exact p and a sample ratio mismatch check.
fisher_exact_testExact p-value for a 2x2 table (two-sided, greater, less) or any r x c table (Fisher-Freeman-Halton), next to Pearson chi-square and an expected-count check.
check_peeking_riskThe real false positive rate when a test is checked K times, and Pocock and O'Brien-Fleming boundaries for up to 10 looks.

Every result has structured JSON, a one-paragraph summary, as_of, source (the method references) and more (the matching calculator on abwex.com).

What it knows that your model does not

  • Exact inverse normal quantiles and the full sample size formula. Models round z-values, drop the pooled variance term or confuse relative and absolute MDE, and the answer moves by thousands of users.
  • Why calculators disagree. The result shows both the standard pooled formula (as R power.prop.test) and Evan Miller's published formula, so a 37,513 vs 36,314 gap has an explanation.
  • Exact hypergeometric sums. A Fisher-Freeman-Halton p-value for a 4 x 4 table needs millions of tables summed; it cannot be estimated in your head.
  • Sequential testing numbers: checking 5 times at p < 0.05 gives a 14.2% false positive rate, and the boundaries that fix it.

Install

Hosted (Claude.ai custom connector, ChatGPT developer mode)

Paste this URL. No token, no sign-up.

https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Claude Code

claude mcp add --transport http ab-test-calculator https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Or run it locally over stdio:

claude mcp add ab-test-calculator -- npx -y github:theluckystrike/ab-test-sample-size-calculator-mcp

Claude Desktop

Add to claude_desktop_config.json:

json
{  "mcpServers": {    "ab-test-calculator": { "command": "npx", "args": ["-y", "github:theluckystrike/ab-test-sample-size-calculator-mcp"] }  }}

Cursor

Add to ~/.cursor/mcp.json:

json
{  "mcpServers": {    "ab-test-calculator": { "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp" }  }}

VS Code

Add to .vscode/mcp.json:

json
{  "servers": {    "ab-test-calculator": { "type": "http", "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp" }  }}

Node 20 or newer is needed for the local stdio version.

Example prompts

  • "Our checkout converts at 4.2%. How many users per variant to detect a 10% relative lift with 80% power?"
  • "We get 4,000 visitors a day and want to test 3 variants. How many weeks will it take?"
  • "Control had 420 conversions from 10,000, the variant 495 from 10,050. Is that significant?"
  • "Run a Fisher-Freeman-Halton test on [[12, 30, 8], [20, 25, 15]]."
  • "If I check the dashboard every day for 10 days, what is my real false positive rate?"

Data sources

No live data. Methods and the published values the tests check against are in DATA_SOURCES.md: R power.prop.test and fisher.test documentation outputs, Evan Miller's calculator code, scipy and statsmodels reference values (scripts/reference.py), Armitage et al. (1969) and Jennison and Turnbull (2000).

Limitations

  • Sample size covers conversion rates (two proportions). Continuous metrics such as revenue per user need the standard deviation and are not covered.
  • Sample size and duration assume a fixed-horizon test analysed once. For several looks, plan with check_peeking_risk.
  • The Fisher-Freeman-Halton test enumerates tables exactly. Very large or sparse r x c tables can exceed the work limit (lower on the hosted server than locally). The tool then returns the chi-square result with a note instead of an exact p.
  • Peeking boundaries assume equally spaced looks and two-sided tests; boundaries are given for up to 10 looks.
  • The confidence interval for the difference is the Wald interval, which is poor for rates near 0 or 1 with small samples.

More

Interactive versions of these calculators and guides are on abwex.com, including the Fisher-Freeman-Halton calculator and the peeking inflation calculator.

Part of the Zovo MCP collection: https://mcp.zovo.one

Development

npm installnpm test                      # builds, then runs the fixture testspython3 scripts/reference.py  # recomputes the reference values with scipy and statsmodelsnode scripts/smoke-stdio.mjs  # stdio check, one call per toolnpx wrangler deploy           # Cloudflare Worker, Streamable HTTP at /mcp

License

MIT

More data servers from the same author

Each one is free, needs no API key and has a hosted URL you can paste into Claude or ChatGPT.

servertoolshosted URL
cups-to-grams-mcpconvert_ingredient, search_ingredients, scale_recipe, convert_oven_temperaturehttps://cups-to-grams-mcp.lipmichal.workers.dev/mcp
dscr-loan-calculator-mcpcalculate_dscr, max_loan_for_target_dscr, rent_needed_for_target_dscr, compare_scenarios, amortization_summaryhttps://dscr-loan-calculator-mcp.lipmichal.workers.dev/mcp
gold-price-per-gram-mcpget_gold_price_per_gram, get_metal_prices, calculate_jewelry_value, convert_gold_unitshttps://gold-price-per-gram-mcp.lipmichal.workers.dev/mcp
lake-level-mcpsearch_lakes, get_lake_level, get_lake_history, compare_lakeshttps://lake-level-mcp.lipmichal.workers.dev/mcp
minimum-wage-mcpget_minimum_wage, list_upcoming_increases, compare_states, annual_income_at_minimum_wagehttps://minimum-wage-mcp.lipmichal.workers.dev/mcp
no-tax-on-overtime-mcpcalculate_overtime_deduction, calculate_tips_deduction, calculate_senior_deduction, estimate_tax_savings, explain_eligibilityhttps://no-tax-on-overtime-mcp.lipmichal.workers.dev/mcp
paycheck-calculator-mcpcalculate_take_home_pay, compare_states_take_home, salary_to_hourly, hourly_to_salary, get_federal_tax_bracketshttps://paycheck-calculator-mcp.lipmichal.workers.dev/mcp
uk-income-tax-salary-mcpcalculate_uk_take_home, get_uk_tax_bands, compare_salaries, marginal_rate_at_incomehttps://uk-income-tax-salary-mcp.lipmichal.workers.dev/mcp
us-federal-holidays-business-days-mcplist_federal_holidays, is_business_day, add_business_days, count_business_days, invoice_due_datehttps://us-federal-holidays-business-days-mcp.lipmichal.workers.dev/mcp
aca-premium-tax-credit-calculator-mcpget_federal_poverty_level, calculate_premium_tax_credit, check_subsidy_cliff, get_aptc_repayment_caphttps://aca-premium-tax-credit-calculator-mcp.lipmichal.workers.dev/mcp
ap-score-distributions-mcpget_ap_score_distribution, estimate_ap_percentile, compare_ap_exams, list_ap_exams, estimate_ap_score_from_compositehttps://ap-score-distributions-mcp.lipmichal.workers.dev/mcp
car-loan-interest-deduction-calculator-mcpcalculate_car_loan_interest_deduction, calculate_salt_deduction, get_tax_brackets_and_standard_deduction, calculate_child_tax_credithttps://car-loan-interest-deduction-calculator-mcp.lipmichal.workers.dev/mcp
estate-gift-tax-exemption-calculator-mcpget_exemption_amounts, calculate_estate_tax, check_gift, compare_yearshttps://estate-gift-tax-exemption-calculator-mcp.lipmichal.workers.dev/mcp
recipe-nutrition-calculator-mcplookup_ingredient_nutrition, calculate_recipe_nutrition, round_nutrition_label, scale_recipe_nutritionhttps://recipe-nutrition-calculator-mcp.lipmichal.workers.dev/mcp
retirement-contribution-limits-2027-mcpget_contribution_limits, calculate_max_contribution, get_social_security_cola, compare_yearshttps://retirement-contribution-limits-2027-mcp.lipmichal.workers.dev/mcp
section-179-deduction-calculator-mcpcalculate_section_179, calculate_bonus_depreciation, get_vehicle_depreciation_limits, calculate_asset_loan_paymenthttps://section-179-deduction-calculator-mcp.lipmichal.workers.dev/mcp
student-loan-repayment-assistance-plan-calculator-mcpcalculate_rap_payment, compare_rap_vs_standard, get_federal_loan_limits, get_plan_transition_dateshttps://student-loan-repayment-assistance-plan-calculator-mcp.lipmichal.workers.dev/mcp
subdomain-finder-mcpfind_subdomains, resolve_subdomains, get_certificate_history, summarize_attack_surfacehttps://subdomain-finder-mcp.lipmichal.workers.dev/mcp
uk-statutory-sick-pay-maternity-pay-calculator-mcpcalculate_statutory_sick_pay, calculate_statutory_maternity_pay, get_minimum_wage_rates, check_eligibilityhttps://uk-statutory-sick-pay-maternity-pay-calculator-mcp.lipmichal.workers.dev/mcp

The paid back-office collection (invoices, VAT, PDFs, spreadsheets) is at https://mcp.zovo.one

来源：README.md，提交 a7040c5

工具

5

calculate_sample_size

A/B test sample size calculator

Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two can be reconciled.

输入参数mde必填alphapowersidedmde_typevariantsbonferronibaseline_rate必填allocation_ratio

estimate_test_duration

A/B test duration estimator

How many days an A/B test must run: from a known sample size per variant, or from baseline rate and MDE (computed as in calculate_sample_size), plus daily visitors and the share of traffic in the test. Returns exact days and the recommended whole weeks (to cover weekday effects).

输入参数mdealphapowermde_typevariantsbaseline_ratetraffic_sharedaily_visitors必填sample_size_per_variant

test_ab_significance

A/B test significance calculator

Is an A/B test result statistically significant? From visitors and conversions per group: two-proportion z-test (pooled), one or two-sided p-value, Wald confidence interval for the difference, relative lift, Pearson chi-square with and without Yates correction (R prop.test), Fisher exact p for the same 2x2 table, and a sample ratio mismatch (SRM) check against the planned split.

输入参数alphaalternativeexpected_splitcontrol_visitors必填variant_visitors必填control_conversions必填variant_conversions必填

fisher_exact_test

Fisher exact and Fisher-Freeman-Halton test

Exact p-value for a contingency table of counts. 2x2: Fisher exact test (two-sided, greater or less, as R fisher.test). Larger r x c tables (for example 3 variants x converted/not, or 4 x 4 survey tables): Fisher-Freeman-Halton exact test by a network algorithm. Also returns Pearson chi-square and how many cells have expected counts below 5, so you can see when the chi-square approximation is unsafe.

输入参数table必填alternative

check_peeking_risk

A/B test peeking and sequential testing risk

What peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.

输入参数alphalooks必填

版本历史

1
  1. v1.0.0最新Nov 3, 2026

评分

已满足适用检查项的 100%

该评分衡量此收录条目的完整程度，依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价，也不表示服务器安全或当前可用：可连通性是带日期的检查结果，评分也不包含安装量或用户评价。

  • 近 30 天内可连通必需25 分
  • 已声明认证方式必需10 分
  • 已列出工具必需15 分
  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分

其他 Calculator 集成

Calculator Tools MCP

tools.calculator官方发布者

一个远程 MCP 端点，让助手从 Cursor、Claude 或 ChatGPT 发布可用的计算器、追踪器、工具或游戏。

Streamable HTTP可网页运行
效率与工作流尚未收录Oct 25, 2026

Free IFTA Calculator

dev.pages.ifta-calculator官方发布者

只读工具，回答美国/加拿大 IFTA 燃油税问题，并依据各辖区里程与加仑数估算季度税额。

Streamable HTTP可网页运行
金融1 个工具Oct 29, 2026

Complexity Cost Calculator

由 GitHub 账号 beamersaas 发布

远程 MCP 服务器，用于估算软件复杂度的成本，并预测 AI 采用在未来几个季度如何改变工程成本。

Streamable HTTP可网页运行鉴权
数据与分析2 个工具Nov 6, 2026

UK Statutory Sick Pay and Maternity Pay

由 GitHub 账号 theluckystrike 发布

UK Statutory Sick Pay (April 2026 rules), Maternity Pay and National Living Wage from gov.uk.

Streamable HTTP可网页运行
4 个工具Nov 3, 2026

Student Loan RAP Calculator

由 GitHub 账号 theluckystrike 发布

学生贷款RAP还款计算器、2026年联邦贷款限额、SAVE/PAYE/ICR终止日期（OBBBA）。AI 翻译

Streamable HTTP可网页运行
4 个工具Nov 3, 2026