Sylvics Apply for the beta
Sample size calculator scriptTool
TOOL

Sample size calculator script

🛠️ Sample size calculator script

💡 Use Case

  • The A-B test sample size sheet rewritten as a short script, so that the four assumptions sit in one place, at the top, with their reasons written next to them.
  • Written two weeks after the triage test was designed, when I found I had changed the power on one tab of the sheet and not on the other.
  • It implements 📐 Sample size for comparing two proportions and prints a small table: the sample per group for the difference I care about, and for half and double that difference.
  • The half and double rows are the lesson of 🎓 Lecture 9 - A-B tests and their traps: halve the effect you hope for and the sample you need roughly quadruples.
A practice run, not the real test

Baseline 40 percent, smallest difference worth having 5 points, significance 0.05, power 0.80: about 1,530 per group. Halve the difference to 2.5 points and it is over 6,000.

⚖️ Pros & Cons

Pros

  • Assumptions first. Anyone can read lines 1 to 8 and disagree with me before any arithmetic happens.
  • The three-row table ends the conversation that starts can we not just run it for a week.
  • It is saved with each test, so the sizing is part of the record.

Cons

  • It assumes every dispatcher is independent of every other. Dispatchers share an office and a list.
  • It answers how many, not how long. How long depends on how many dispatchers a customer has, and for some customers the answer is never.

🔄 Alternatives

  • The spreadsheet, for anyone who will not open a script.
  • An online calculator. Fine for the number, useless as a record of what was assumed.