statistical-power
Sample-size and statistical power calculations for planning studies. Use whenever someone asks "how many subjects/samples/replicates do I need", wants an a priori power analysis, a minimum detectable effect (MDE), a power curve, or needs to justify a sample size for a grant, IRB protocol, or pre-registration. Covers closed-form power for t-tests, ANOVA, proportions, correlations, chi-square, and regression, plus simulation-based (Monte Carlo) power for designs with no formula — logistic/Poisson regression, mixed models, cluster-randomized trials, survival, and interactions. Use this skill even when the request only mentions an effect size, alpha, or "80% power" without saying "power analysis" explicitly. For laying out the study (randomization, blocking, factorial/DOE, crossover, sequential designs) use experimental-design; for analyzing data already collected and reporting it use statistical-analysis.
Author
Category
Product DesignInstall
Download and extract to your skills directory
Copy command and send to AI Agent for auto-install:
Sample Size Calculation and Statistical Power Analysis — Power Analysis Tool for Optimized Study Design
Skill Overview
statistical-power provides a complete set of tools for sample size calculation and statistical power analysis. It supports closed-form formulas and Monte Carlo simulation methods—from simple t-tests to complex mixed models—helping researchers determine an appropriate sample size scientifically before data collection.Common Use Cases
1. Determine the Required Sample Size
Perform an a priori power analysis before submitting an IRB application, a grant proposal, or a pre-registration. Calculate the number of participants needed to reach the target power (typically 80% or 90%). This helps avoid invalid studies caused by insufficient sample size, as well as wasted resources due to an excessively large sample.
2. Compute the Minimum Detectable Effect (MDE)
When the sample size is already fixed (e.g., due to budget limits, a time window, or an available participant pool), assess the smallest effect size the study can reliably detect. Answer the key question: Given the resources available, how large an effect can I detect?
3. Power Analysis for Complex Designs
Use Monte Carlo simulation to estimate power for models without closed-form solutions, such as mixed models, logistic/Poisson regression, cluster randomized trials, and survival analysis. Supports handling real-world complexities including attrition, design effects (ICC), and multiple comparisons.
Core Features
1. Closed-Form Sample Size Calculations
A single interface supports sample size, power, and MDE calculations for a wide range of common statistical tests, including t-tests (independent, paired, one-sample), ANOVA, proportion tests, correlation, chi-square tests, and linear regression. It automatically handles unequal allocation ratios, one-sided vs. two-sided tests, and generates visual power curves.
2. Monte Carlo Simulation Power Analysis
A flexible framework covers complex designs without closed-form formulas, including logistic/Poisson regression, mixed-effects models, cluster randomized trials, and survival analysis. It provides power estimates with 95% confidence intervals, includes a built-in sample size search algorithm, and automatically finds the sample size needed to achieve the target power.
3. Sample Size Adjustments and Sensitivity Analysis
Automatically adjusts for attrition rates, cluster effects (design effect DEFF), and multiple comparisons to ensure the calculated sample size matches what is actually needed for the analysis. Generates power curves (power vs. n or power vs. effect size) for sensitivity analysis in grant applications, IRB reviews, and pre-registrations.
Frequently Asked Questions
How many samples does my experiment need?
The required sample size depends on four interlocking factors: effect size, significance level (α, typically 0.05), target power (typically 0.80 or 0.90), and the type of test. Using this skill’s
sample_size() function, input the expected effect size (e.g., Cohen’s d = 0.5) and target power (e.g., 0.80) to obtain the number of participants needed per group. The key is: Do not pick an effect size out of thin air. Use a “smallest important effect” (SESOI) or a shrunken estimate from prior studies, and always run a sensitivity analysis.How do I choose an appropriate effect size?
Effect size is the most critical input in power analysis and also the easiest to get wrong. Recommended priority order:
1) Smallest Important Effect (SESOI) — the minimum effect that would truly change a decision; this is the most defensible choice.
2) Pilot study or prior study estimates — but shrink them, because publication bias can inflate the effect.
3) Cohen’s conventional values (small = 0.2, medium = 0.5, large = 0.8) only as a last resort.
No matter which value you choose, you must report a sensitivity analysis: show how the required sample size varies across a range of plausible effect sizes, rather than reporting a single number.
When should I use Monte Carlo simulation instead of closed-form formulas?
Closed-form formulas are only suitable for a small number of simple tests (t-tests, ANOVA, proportions, correlation, chi-square, simple regression). If your analysis plan involves any of the following, you should use Monte Carlo simulation:
The simulation principle is simple: generate data based on assumed effects, test using the planned analysis method, repeat thousands of times, and power is the percentage of simulations that yield statistically significant results. This skill’s simulate_power() function provides a complete simulation framework.