statistical-analysis

Statistical Analysis - A Complete Statistical Testing Plan for Scientific Data Analysis

Skill Overview

Statistical Analysis provides a systematic statistical analysis solution for scientific data analysis. It covers guidance on selecting hypothesis testing methods, validating statistical assumptions, calculating effect sizes, conducting power analysis, and writing APA-standard reports that meet academic norms. This helps researchers deliver statistical analyses that reviewers cannot fault.

Applicable Scenarios

1. Hypothesis Testing and Between-Group Comparisons

When you need to compare differences between groups, test relationships among variables, or validate research hypotheses, this skill offers a complete guide to choosing testing methods. From t-tests and ANOVA to chi-square tests, the system recommends appropriate statistical methods based on data characteristics. It also automatically checks assumptions such as normality and homogeneity of variances to ensure the validity of results. If assumptions are not met, it provides specific recommendations such as Welch correction and nonparametric alternatives.

2. Regression Analysis and Correlational Studies

For linear regression, logistic regression, or correlation analysis, this skill provides comprehensive diagnostic tools and report templates. This includes residual analysis, multicollinearity detection (VIF), heteroscedasticity testing (Breusch-Pagan), and model fit evaluation, ensuring that the regression model meets statistical assumptions and can be interpreted reasonably. It automatically generates APA-format reports that include regression coefficients, confidence intervals, and effect size (R²).

3. Bayesian Statistics and Small-Sample Analysis

When sample sizes are small, prior information needs to be incorporated, or direct probability statements are desired, this skill provides a Bayesian statistical analysis plan. It supports Bayesian t-tests, Bayes factor calculations, and hierarchical model building with PyMC. It can quantify evidence supporting or favoring the null hypothesis and produce a complete report including posterior distributions and credible intervals.

Core Features

1. Intelligent Test Selection and Assumption Validation

Based on data type, number of groups, and study design type (independent/paired), the system automatically recommends suitable statistical tests. The built-in comprehensive_assumption_check function performs one-click anomaly detection, normality testing (Shapiro-Wilk), and homogeneity of variances testing (Levene's test), and generates visualizations (Q-Q plots, boxplots). When assumptions are violated, it automatically recommends alternative approaches such as Welch’s t-test, Mann-Whitney U test, or data transformation methods.

2. Effect Size Calculation and Power Analysis

Going beyond the limitations of p-values, it provides comprehensive effect size calculation and interpretation. It automatically computes effect size indicators such as Cohen’s d, partial eta squared (η²_p), and correlation coefficient r, along with their confidence intervals. It supports prior power analysis to determine the required sample size and post hoc sensitivity analysis to assess the smallest effect the study can detect. It offers effect size interpretation guidance based on benchmarks (small/medium/large effects) while emphasizing interpretation of practical meaning in the research context.

3. APA-Format Reporting and Academic Writing Standards

Compliant with APA standards and top-journal requirements, it automatically generates complete report templates including descriptive statistics, test statistics, exact p-values, effect sizes, and confidence intervals. It covers common report formats for t-tests, ANOVA, regression analyses, Bayesian analyses, and more. Built-in best practices for statistical integrity guide users to distinguish confirmatory versus exploratory analyses, avoid p-value manipulation, properly handle multiple comparisons, and address missing data correctly.

Frequently Asked Questions

How do I choose an appropriate statistical test?

Choosing a statistical test requires considering data type (continuous/categorical), number of groups (two groups/multiple groups), study design type (independent/paired), and distribution characteristics. This skill provides a decision tree and a reference document test_selection_guide.md to help you select suitable tests based on your research design. Simple rule of thumb: for continuous data, two independent groups use a t-test; multiple groups use ANOVA; paired designs use a paired t-test or repeated-measures ANOVA; when normality assumptions are not met, use nonparametric tests such as Mann-Whitney U or Kruskal-Wallis.

What should I do if statistical assumptions are not met?

When assumptions such as normality, homogeneity of variances, or linearity are violated, you should not simply use the original test results. If violations are mild and sample sizes are large (n > 30), parametric tests are somewhat robust. For moderate violations, nonparametric alternatives are recommended. For severe violations, consider data transformation or robust methods such as Welch correction or Brown-Forsythe. The skill’s assumption_checks.py will automatically detect assumption violations and provide specific recommendations, including whether data transformation is needed, which alternative tests to use, or what correction methods to apply.

When should Bayesian statistical methods be used?

Bayesian methods are particularly useful in these situations: when sample sizes are small (e.g., n < 30) and stronger inference is needed; when you want direct probability statements (e.g., “95% probability that the effect is positive”) rather than confidence intervals; when you need to quantify evidence supporting the null hypothesis; when prior information is available to use; when adopting sequential data analysis designs; and when models are complex (e.g., hierarchical structures, missing data). This skill integrates PyMC and ArviZ, supports complete analysis workflows from simple Bayesian t-tests to complex hierarchical models, automatically checks convergence (R-hat, ESS), and generates posterior predictive checks.