Introduction Link to heading

Misunderstandings around statistical tests are quite common, so we propose below an actuarial explanation. We’ll walk through a concrete example (independence testing) step by step.

This article explains how to choose the significance level α based on test accuracy , and how to interpret results using the concept of the acceptance region .

Step 1: Formulating Hypotheses Link to heading

  • Null Hypothesis (H₀): the data series are independent
  • Theoretical Alternative Hypothesis (H₁): the series are dependent

H₁ reflects the actual hypothesis of interest, but it is often difficult to state precisely or test directly. Therefore, we may introduce a more practical alternative hypothesis:

  • Practical Alternative Hypothesis (H₂): the series exhibit a concrete, observable form of dependence (e.g., Pearson correlation ρ ≠ 0, lag-1 autocorrelation ≠ 0, etc.)

H₂ is formulated explicitly to enable the construction of a test statistic. While it approximates H₁, it is directly testable—though this comes at the cost of reduced precision.

Step 2: Choosing a Test Statistic, Conditional on H₀ and the Alternative Link to heading

Suppose we use a Z-score test statistic , where under H₀ the statistic follows a standard normal distribution (mean 0, standard deviation 1). We obtain a Z-value reflecting test accuracy: Z = 1.8.

Step 3: Choosing the Significance Level α Link to heading

The value of α controls the probability of rejecting H₀ when it is actually true (Type I error) . Based on the test’s accuracy, we select:

Suggested significance levels:

Test Precision Suggested α Objective
High (precise) 0.01 or 0.05 Avoid falsely rejecting H₀
Low (approximate) 0.10 or even 0.50 Reject H₀ more easily

Step 4: Calculating the Acceptance Region Link to heading

The acceptance region is the interval in which Z must fall in order not to reject H₀.

α Acceptance Region
0.50 [−0.674, +0.674]
0.10 [−1.645, +1.645]
0.05 [−1.96, +1.96]
0.01 [−2.576, +2.576]
0.001 [−3.29, +3.29]

Step 5: Interpreting the Result Link to heading

Suppose we obtained Z = 1.8.

For α = 0.50 Link to heading

  • Acceptance region: [-0.674, +0.674]
  • Z = 1.8 is outside → Reject H₀

For α = 0.05 Link to heading

  • Acceptance region: [-1.96, +1.96]
  • Z = 1.8 is within → Do not reject H₀

For α = 0.01 Link to heading

  • Acceptance region: [-2.576, +2.576]
  • Z = 1.8 is within → Do not reject H₀

In some cases, the choice of α can be guided by:

  • Test power (1 − β) – if we can estimate the distribution under H₂ (the alternative), then α may be adjusted to achieve a desired power level (e.g., 80%).
  • In practice, α may also reflect an acceptable risk threshold in a specific context (e.g., finance, healthcare, engineering).

Choosing α is ultimately a strategic decision, reflecting the test’s reliability, acceptable risk level, and the nature of the hypotheses compared.

Conclusions Link to heading

  • The significance level α is not fixed or universal—it should be chosen according to the accuracy and robustness of the statistical test .
  • If the test is only approximate, a higher α may be justified to detect deviations from H₀ more quickly, despite uncertainty.
  • Conversely, for highly precise tests, a small α helps reduce false positives, but may miss relevant deviations.
  • Thus, α acts as a calibration parameter , balancing Type I error control and test sensitivity.

Note Link to heading

This approach is inspired by actuarial practice, such as the independence test in the Chain Ladder method, where Mack (1993) suggests a 50% confidence interval specifically because the test is approximate.