Experimental design
01 // What experimental design covers
Experimental design is the overall plan for how an experiment will be run, decided before any data is collected. It covers which variables to change and measure, what to hold constant, how many trials to run, and how to keep bias from creeping into the results — get this planning stage wrong, and no amount of careful measuring afterward can fix it.
02 // The core decisions
1. Define the variables
Decide exactly what you'll change (independent) and exactly what you'll measure (dependent).
2. Plan the controls
List everything else that needs to stay identical across every group so it can't become a confounding variable.
3. Set up a control group
Include a baseline group that doesn't receive the treatment, to compare against.
4. Decide on sample size and trials
Plan enough subjects or repeated trials that a single unusual result won't distort the conclusion.
5. Reduce bias
Decide how subjects or samples will be assigned to groups — randomly, whenever possible — to avoid stacking one group unfairly.
03 // Common design flaws
- Too few trials makes it hard to tell a real pattern apart from random chance — one or two data points can look like a trend that isn't really there.
- No control group means there's nothing to compare the treatment group against, so there's no baseline to know whether the treatment actually changed anything.
- Non-random group assignment can accidentally sort subjects in a way that's connected to the outcome, creating a hidden bias before the experiment even starts.
04 // Questions people ask
What is the difference between experimental design and the scientific method?
The scientific method is the whole process, from asking a question through drawing a conclusion. Experimental design is specifically the planning stage within that process — deciding variables, controls, sample size, and how to avoid bias before any data is collected.
Why does sample size matter in experimental design?
A small sample size makes it hard to tell whether a result reflects a real pattern or just random chance. Testing more subjects, or running more trials, makes an unusual single result less likely to distort the overall conclusion.
What's the point of randomly assigning subjects to groups?
Random assignment helps spread out any unknown differences between subjects roughly evenly across groups, so those differences are less likely to become a hidden confounding variable that skews the result.