What Is Effect Size? Why Real Effects Can Still Be Tiny
Effect size tells you how large a difference or relationship actually is, separate from whether it is statistically real. A study can find an effect that is almost certainly true and still be describing something so small that it makes no practical difference to your life.
The Number Behind the Headline
Effect size is a number that captures the size of a difference or a relationship. If a new painkiller reduces pain scores, effect size tells you by how much. If two variables move together, effect size tells you how strongly. It is a different question from whether the result is real, and a different question from whether it matters.
Researchers express it in several ways. Sometimes it is a plain real-world unit, such as a difference in blood pressure measured in millimetres of mercury, or a difference in exam scores measured in marks. Sometimes it is a standardised measure, such as Cohen's d, which expresses a difference in terms of how spread out the data are, so that results from different studies can be compared. Sometimes it is a ratio, such as relative risk, which says how much more or less likely something is to happen in one group compared with another.
Significant Does Not Mean Large
A result being statistically significant means the researchers are confident the effect did not simply arise from random noise in the data. It says nothing about the size of that effect. A tiny, practically meaningless difference can be statistically significant, and a large, meaningful one can fail to reach significance if the study was too small to detect it reliably.
This is the gap effect size is built to close. Once you know a result is probably real, the next question is how big it is. Both pieces of information matter, and a headline that reports only one of them is giving you half the picture.
Big Samples Find Small Effects
The more people in a study, the easier it becomes to detect even a very small effect with confidence. This is one of the quieter traps in reading research. A study with a handful of participants needs a fairly large effect to show up as statistically significant. A study with tens of thousands of participants can pick up an effect so small that no individual would ever notice it, and still call it significant, because the sheer size of the sample makes the signal reliable even when it is faint.
Imagine a trial of a new supplement involving a very large number of people, which finds that it reduces average resting heart rate by a single beat per minute compared with a placebo. The result might be entirely real and robust. It is also very unlikely to change how anyone feels, performs, or lives. The study answered the question of whether an effect exists. It did not answer the question of whether that effect is worth anyone's attention.
An Average Is Not a Person
Effect sizes usually describe an average across a group, and averages flatten individual variation. Within any study, some people respond strongly to a treatment, some respond weakly, and some do not respond at all, or even move in the opposite direction. The reported effect size is the net result of all that variation pooled together, not a prediction of what will happen to any one person.
This matters because two studies can report identical average effect sizes while hiding very different stories underneath. One might show a modest benefit shared fairly evenly across everyone. Another might show no effect in most people and a large effect in a small subgroup, averaging out to the same number. Without knowing which pattern you are looking at, the average effect size tells you less about your own likely experience than it appears to.
Deciding If It Matters to You
The most useful habit is to translate the effect size into something concrete before deciding whether to care. Instead of asking whether a diet
significantly reduces weight,
ask how many kilograms, over what time period, and whether that amount would be noticeable on a scale or in how your clothes fit. Instead of asking whether a drug reduces risk, ask what the risk was to begin with and what it becomes afterwards, since a result described as a large relative reduction can still mean a tiny change in absolute terms if the starting risk was already low.
It also helps to weigh the effect against cost, effort, and side effects. A small effect might still be worth pursuing if it is free and harmless. A larger effect might not be worth pursuing if it demands a daily pill, a strict routine, or money you would rather spend elsewhere. Effect size is one input into that decision, not the decision itself.
Common questions
- Is a bigger effect size always better?
Not necessarily. A bigger effect size usually means a more noticeable or useful result, but it needs to be weighed against how the effect was measured, how variable people's responses were, and what it would cost you to act on it.
- What counts as a small or large effect size in research?
Statisticians sometimes use rough labels, such as small, medium, and large for a measure called Cohen's d, but these are loose conventions rather than fixed rules. What counts as meaningful depends heavily on the specific outcome being studied, so the label alone should not substitute for looking at the real-world units.
- Can a result be statistically significant but have a tiny effect size?
Yes, and this is common in very large studies. The study can be confident the effect is real while the effect itself is too small to matter for anyone's daily life or health decisions.
- How do I find the effect size in a news article about a study?
News coverage often reports only whether a result was significant, so you may need to look at the original study's abstract or results section for the actual numbers, such as a mean difference, a correlation value, or a risk ratio, to judge the size for yourself.
- Does a study's effect size predict what will happen to me personally?
Not directly. It describes an average across the people studied, and individual results can vary widely around that average, so it is better treated as a general indication than a personal forecast.
Lucent, once a week
We read the new research and write up what actually holds, in the same plain language as this page. No jargon, no hype, three papers a week.
Read the latest issueRead next
- What a Meta-Analysis Actually Does to the Evidence
A meta-analysis combines results from many studies into one estimate, but it only clarifies the truth if those studies were asking the same question well.
- Observational Study vs Experiment: Why Diet Advice Flip-Flops
Most nutrition headlines come from observational studies that track habits, not experiments that test them, which is why the advice keeps changing.
- What a Randomised Controlled Trial Actually Buys You
A randomised controlled trial sorts people into groups by chance, which cancels out hidden differences and lets researchers credit any gap to the treatment.