If you’re running a survey or a test, how many responses do you need for your data to be “statistically valid?”  It’s often a difficult goal to achieve, but without valid data, you can’t trust your test results.

When you’re determining the statistical validity of your data, there are four criteria to consider.

1.  Population: The reach or total number of people to whom you want to apply the data.

2.  Probability or percentage: The percentage of people you expect to respond to your survey or campaign.

3.  Confidence: How confident you need to be that your data is accurate.  Expressed as a percentage, the typical value is 95% or 0.95.

4.  Margin of Error or Confidence Interval: The amount of sway or potential error you will accept.  It’s the “+/-” value you see in media polls.  The smaller the percentage, the larger your sample size will need to be.

For example, if 45% of your survey respondees choose a particular answer and you have a 5% (+/- 5) margin of error, then you can assume that 40%-50% of the entire population will choose the same answer.

Sample Size Calculators

If you’re looking to determine how many participants you need in an A/B test, check out this sample size tool that will tell you how many visitors you need at various conversion rates for different desired confidence levels.

Armed with these basic concepts, you can use the Sample Size Calculator and Confidence Interval Calculator at Survey System.