Math With AmarA C A D E M Y

Advanced · 15 minute lesson

What evidence does a small p-value provide?

Run a two-sided z-test with a prespecified threshold while separating evidence, effect size, and truth claims.

Lesson 9 of 12 in Statistics. Take the time you need; the lesson estimate is a guide.

Jump to practice

Learn with Amar

Teaching video

A teaching recording for this chapter has not been published yet. Start with the worked example below and explore the related animations where available.

Browse grades and teaching videos

01 · Read and understand

What you will learn

  • Standardize a sample-mean discrepancy.
  • Calculate a two-sided p-value.
  • Distinguish rejection from proof and statistical from practical importance.

Before you start

Sampling distributions, standard error, normal tail areas, and confidence intervals.

Keep paper nearby. Read the question once for the context, then again to identify what is known and what you need to find.

Start with a question

An independent normal sample of 64 calibration readings has mean 102 units and known population SD 8 units. Test H₀: μ = 100 against H₁: μ ≠ 100 at α = 0.05.

Why this math matters

A hypothesis test asks how surprising the observed statistic would be under a specified reference model. It does not directly calculate the probability that the reference claim is true. A clear analysis chooses the alternative and decision threshold before seeing the result, then reports the observed effect and assumptions alongside the decision.

Make a representation of your own.Sketch the quantities or relationships in this question before working through the solution. The cover image sets the learning scene; it does not show this problem’s exact values.

Set up the model

A useful answer starts with clear assumptions:

  • Observations are an independent random sample from a normal population.
  • The population standard deviation is known to be eight units.
  • The two-sided alternative and α = 0.05 were specified before looking at these data.

02 · Work through the example

Follow the reasoning, one step at a time.

Try to predict the next step before reading it. After each calculation, explain why the operation makes sense and how it helps answer the original question.

AI-edited portrait of Amar

Work through it with Amar

See this example unfold.

The complete worked example, one idea at a time.

Text-led walkthrough · no audioAmar’s portrait was edited with AI.

What evidence does a small p-value provide?

Paused

Question: Start with the question. Paused.

Question

Start with the question

An independent normal sample of 64 calibration readings has mean 102 units and known population SD 8 units. Test H₀: μ = 100 against H₁: μ ≠ 100 at α = 0.05.

Before you calculate

Read what is known and what you need to find. Make a prediction before moving to the first calculation.

Starts paused. Play advances through the full text at a reading pace; pause whenever you need more time. Previous, Next, and the phase buttons let you set your own pace. Playback pauses when this walkthrough leaves the screen or you switch tabs.

Your device’s reduced-motion setting keeps each phase still. Manual controls remain available. The full written solution stays below.

  1. Calculate standard error under the model

    SE = 8/√64 = 1 unit

    The variability relevant to testing a mean is the sample mean's standard error, not the eight-unit spread of individual readings.

  2. Measure distance from the null mean

    z = (102 − 100)/1 = 2

    The observed sample mean is two standard errors above the null value. The alternative also treats equally large negative discrepancies as evidence.

  3. Evaluate both tails and decide

    p = 2P(Z ≥ 2) ≈ 0.0455 < 0.05

    Under the null model, about 4.55% of samples give a standardized discrepancy at least this extreme in either direction. The chosen rule therefore rejects H₀.

The result

Reject H₀ at the prespecified 5% significance level; the observed mean difference is +2 units.

This decision does not prove the process mean changed or measure whether two units matter in practice. At a stricter prespecified α = 0.01, the same data would not reject. That different decision would not establish that H₀ is true.

Common mistakes to catch

  • The p-value is not P(H₀ is true | data).
  • Choosing a one-sided alternative after seeing a positive effect invalidates the original testing plan.

03 · Practice independently

Try it before revealing the answer.

Use paper or a calculator as needed. Write your units and reasoning, then open the hint or explanation to check your approach.

Practice 1

Keeping n and SD unchanged, a mean of 103 gives what z-score and approximate two-sided p-value?

Show a hint

The standard error remains one.

Reveal answer and explanation

z = 3; p ≈ 0.0027

The discrepancy is three standard errors, with about 0.135% in each normal tail.

Practice 2

A valid test returns p = 0.20 at α = 0.05. What is the correct decision?

Show a hint

Compare p with the prespecified threshold without declaring the null proven.

Reveal answer and explanation

Fail to reject H₀

The data do not meet the rejection rule. Limited evidence against H₀ is not evidence that its exact claim must be true.

Take the idea with you

Pair every test decision with the effect estimate, uncertainty, sampling assumptions, and practical context.

04 · Reflect and continue

Can you explain it in your own words?

Before moving on, explain the main idea without looking at the worked example. Try both practice questions, check your reasoning, and name one mistake you now know how to avoid. Return to a step if you still need support.

Next lesson

Up next: How does a regression line choose its slope?

Completion is your own study record, not a test score. It stays in this browser, does not sync to another device, and can be removed by clearing browser data.