How do you calculate statistical significance?

This calculator runs a two-proportion z-test: it compares the control and variant conversion rates, turns the gap into a z-score, and converts that into a p-value. If the p-value is below 0.05, the result is significant at 95% confidence.

Is it a 5% or 95% significance level?

Both describe the same threshold. A p-value cutoff of 0.05 is a 5% significance level, which is the same as 95% confidence, and it's the default this calculator uses.

Is a p-value of 0.001 statistically significant?

Yes. A p-value of 0.001 is far below the 0.05 cutoff, so the difference is very unlikely to be noise. Check the confidence interval too, since a tiny lift can still be significant on a big sample.

Does it work as a sample size calculator?

Partly. When a test isn't significant yet, it estimates the sample size you'd need at 80% power for the effect size you're seeing. It doesn't plan sample size before a test starts.

Can I use it as a split test calculator for emails or landing pages?

Yes. Any two-variant test with a rate works, like email click-through, landing page signups, or checkout conversions. It's built for two variants, not multivariate tests.

Can I save the calculation?

Use the toolbar above the result to copy or download as .md, .txt, .html, or PDF, and nothing is stored unless you sign up for Intempt. To turn results into a write-up, try the AI report generator.

What happens after I sign up for Intempt?

The Experimentation Lead agent designs the test with a hypothesis and a guardrail metric, runs it against the product events already connected on day one, tracks significance live, and drafts the next experiment from what shipped.

All free tools

A brass compass lying open on a sea chart beside a starfish.

Analytics

Free A/B test significance calculator.

Check if your A/B test result is statistically significant. Paste control and variant numbers to get p-value, confidence interval, and the sample size you still need. In Blu, the Experimentation Lead agent fixes sample size, a guardrail metric, and a stopping rule before a test ships, so underpowered ones don't.

How to use the A/B test significance calculator

It takes three steps and every output renders on this page so you can copy or export it right away.

Enter visitors and conversions

Add the visitor and conversion counts for your control and your variant. Manual counts from any tool are fine.

Check significance

The A/B test significance calculator returns the p-value, relative lift, both conversion rates, and a 95% confidence interval on the difference.

Open it inside Intempt

Push the parameters into an experiment inside Intempt so the Experimentation Lead agent runs the test with a real hypothesis, guardrails, and a defined exit rule.

Use in Claude

Blu Agent

Who it's for

Marketing and growth teams deciding whether to call an A/B test
Anyone who wants the significance math without a spreadsheet or a Bayesian black box

What's in the download

SKILL.md - The exact two-proportion z-test math (p-value, CI, sample size, days-to-significance) from the browser tool, plus instructions to check for a connected analytics/CRM MCP first, falling back to pasted numbers.

How to install it

claude.ai or Claude Desktop

Upload as a zip in Capabilities

Paid plan

Open Settings, then Capabilities
Turn on code execution if it isn't already on
Upload the .zip you downloaded

Requires a Pro, Max, Team, or Enterprise plan. Not available on the Free plan.

Claude Code

Drop the folder, it auto-loads

Any plan

Unzip the download
Drop the folder into ~/.claude/skills/ (or .claude/skills/ in a project)
Claude Code finds it automatically

$ ls ~/.claude/skills/
your-new-skill/

What you need

Claude Desktop or claude.ai (Pro, Max, Team, or Enterprise for claude.ai upload)
Claude Code: no plan restriction
An analytics or CRM MCP connector is optional - pasted numbers work without one

Try it with

“Use the ab-test-calculator skill to check if my checkout redesign test is significant”
“Pull my latest experiment numbers and tell me if I can call a winner yet”

Prefer an instant answer? Run it now above.

What you get from the A/B test significance calculator

What it does on this page, and what it does inside Intempt.

Four numbers in, no tracking setup needed

Enter visitors and conversions for the control and the variant
Works with any rate: signups, clicks, demo requests, or bounces
Counts from a spreadsheet or a dashboard export are fine

A test data card for a pricing CTA listing control and variant visitors, conversions, the metric, and daily traffic.

A statistical significance calculator that shows its math

Runs a two-tailed, two-proportion z-test at 95% confidence
Reports relative lift, the p-value, and both conversion rates
Adds the 95% confidence interval on the absolute difference

A two-proportion z-test result marked significant, showing relative lift, p-value, both conversion rates, and the confidence interval.

Ship every experiment with a defined hypothesis

Hypothesis, primary metric, guardrail, and exit rule are all set upfront
Locked in before a single pixel moves
No test ships on gut feel

A/B test setup panel with hypothesis, primary metric, and guardrail defined before launch.

Watch guardrails while the test runs

You watch the primary metric and every guardrail side by side
If a variant lifts one metric while it wrecks another, you'll catch it
The kill switch trips on its own - nobody has to be watching

Automatic alert firing when a guardrail metric is crossed mid-test.

See it work on your own output.

Start free

Questions about theA/B test significance calculator.

01 / 08

Is the A/B test significance calculator free?

Yes. Sign up for the free Intempt plan, then run calculations on this page free 10 times a day, and up to 10 times a day across all the tools. Designing experiments, running them against real users, and reading confidence intervals sit inside the free Intempt plan.

Teams love it

They stopped stitching tools andhired one system.

Acquisition and retention are one connected motion. Personalizing the path before signup only pays off when the same system carries the user through onboarding afterward.

Anonymous traffic was not turning into registered users, and acquisition and retention were run as two separate problems. We ran Experiences on the anonymous visitor flow to put the right content and CTAs in front of each visitor, then handed new signups to automated Journeys for onboarding and deeper engagement.

455,396

users instrumented

Read the full research

Judge experiments on revenue per visitor instead of conversion rate. A variant can win on conversions and still lose on revenue, and repeat buying only shows up when the whole journey is in one view.

High-intent shoppers browsed a large, specialized outdoor catalog, and browsing, product interest and purchase history sat in different places. We unified them into one customer view, then ran real-time personalization, intent-based segmentation and timely follow-ups on top of it, judging experiments on revenue per visitor.

Read the full research

Build the pipeline on observed signals. When follow-ups trigger on intent, the team spends its time on the users who are genuinely engaging.

Static lead lists and timeline-based follow-ups were hiding who was actually engaging. We ran a signal-led outbound pipeline where follow-ups fire on real behavior: outbound replies, onboarding progress and product usage.

Read the full research

Run the experiment live.

Hypothesis and stopping rule fixed before launch, with no analyst to staff. Guardrail metrics come in on the Org plan.

Start for free

More free tools

Other tools teams reach for right after this one.

Analytics

Customer Churn Risk Calculator

Usage and relationship signals in, a weighted risk score and one action out.

Risk score with the math shown

Score my account

Analytics

AI Report Generator

Paste campaign data, get a formatted report back.

Results in seconds

Try it free

Marketing

Customer Journey Generator

ICP in, a 5-stage journey map out.

Results in seconds

Try it free

Marketing

Ad Copy Generator

Headlines and primary text built to convert.

Meta, Google, TikTok ready

Write my ad

Analytics

Keep exploring

Try a free tool

AI Report Generator

Paste campaign data, get a formatted report back.

Customer Churn Risk Calculator

Usage and relationship signals in, a weighted risk score and one action out.

Automate it

Accounts At Churn Risk

Paying accounts the health score has flagged as at risk, with no renewal deal in flight, so your CSMs can step in before they leave.

Accounts No Open Deal

Customer accounts with no open deal and account-level health attributes, plus per-user recent session activity, so AEs can review expansion whitespace.

Customer data integration: how two-way sync keeps one profile

One-way connectors copy fields and drop identity, so one customer turns into four records. Here's how two-way sync keeps one profile, and what to check before a write.

9 best Claude skills for marketing analytics (2026)

9 free Claude Code skills for marketing analytics: KPI blueprint, anomaly alert, benchmark check, cohort tracker, leak finder, lever finder, weekly reporter, promo impact check and margin builder. Every skill states its method.