Next Lesson - Models of Health
Core
See our summary and glossary article which explains the terms used in this article.
There are four main types of observational studies used in the study of populations.
They are split into descriptive (which describe how common a disease is and who has it, without trying to test what causes it) and analytical (which compare groups to test whether an exposure, such as mould, is linked to a disease). Cross-sectional and ecological studies can also be used to look for such links. This article treats them as descriptive because they mainly describe how common a disease is.
The two types of descriptive study discussed in this article are ecological and cross-sectional studies, and the two analytical studies discussed are cohort and case-control studies.
In this article, asthma is the example used in the sections on each design. The worked confidence interval examples near the end use lung cancer and household data instead. Using one disease makes the study designs easier to compare. However, these study designs could be used for a wide variety of epidemiological hypotheses (testable ideas about the causes and patterns of health and disease in populations), and their use is not limited to disease investigations.
A “Case” in this article refers to a study participant that has the disease. A “Control” is a study participant who is included in the study for comparison. In a case control study, a control does not have the disease. In a cohort study, the comparison group is made up of people who have not been exposed to the factor being studied.
Apart from the published figures quoted (the population of London and asthma in England), the numbers in this article are purely hypothetical and do not represent the true results of any known studies.
Common Flaws with Study Designs
All study designs have common flaws, relevant to all types. These are grouped together here so that they don’t have to be explained four times, as they apply to all four study designs.
Random error: This is the effect of chance on a study’s results. Random error can affect every study design. It is impossible to remove chance completely, but its effect can be reduced and measured. For example, confidence intervals in the results show how much chance may have affected them. Increasing sample size is a very useful tool to reduce random error as it dilutes the effects of chance.
Lack of generalisability (general applicability): Lack of generalisability applies to any study where a sample of the population is chosen rather than studying the whole population. Suppose a researcher takes 1000 people from London for a study. The researcher cannot necessarily say that those 1000 people represent the 8.8 million people who live in London (2021 Census). This means that their study would have low generalisability. This could be improved by choosing a sample that represents the whole population, for example by taking a random group of people from each borough. Simply taking more people from the same unrepresentative group would not fix this. Representative sampling is expensive.
Confounding variables: Confounding variables are outside factors that influence the people in the study. The study organisers may not know about them. A confounder is linked to the exposure being studied (the independent variable, which is the factor whose effect is being tested). It is also linked to the outcome (the dependent variable, which is the result being measured), independently of the exposure (that is, even in people who are not exposed). It is not a step on the path between the exposure and the outcome. This means that a confounder can create a false association (a link that is not real) between the independent and dependent variable. For example, in a study of coffee drinking and pancreatic cancer, people who drink a lot of coffee may also smoke more. Coffee drinkers may then seem more likely to develop pancreatic cancer, when the real risk factor in this example is smoking (a well-established cause of pancreatic cancer). Smoking is the confounder here. It is linked to the exposure (coffee drinking) and to the outcome (pancreatic cancer), and it is not a step between them. Once identified, a confounder can be controlled for, for example by comparing only people who smoke a similar amount, or by adjusting the analysis for smoking (using a statistical method that takes smoking into account). This gives a fairer estimate of the association between coffee drinking and pancreatic cancer.
Ecological studies are the easiest study design to understand and an example of a descriptive study. They take a population and split it into groups by a characteristic, usually geographical (where people live). They then compare how often the disease occurs in each group, for example as the number of cases per 100,000 people (a rate). They are often fast and cheap to complete, because they can use data that have already been collected.
For example, the population of a town could be the target population (the group the study is about) for this study, and the characteristics that the population is being separated by could be distance from a motorway. If the disease investigated was asthma, the results of the study would be a rate (the number of cases for a set number of people), for example, “There are 4000 cases of asthma per 100,000 residents less than a mile from the motorway compared to 3000 cases of asthma per 100,000 residents more than a mile from the motorway.” Counts alone would be misleading, because the two groups may have very different numbers of people.
There are a number of flaws with the ecological study design:
- Confounding variables are common and difficult to identify.
- Lack of generalisability is common.
- Ecological fallacy - the data describe whole groups, so a pattern seen in the groups cannot be assumed to apply to the individuals within them.
Cross-sectional studies are a second type of descriptive study commonly used. To perform this study, the researcher must find the number of cases at a certain time within a group of the population. This ‘snapshot’ provides good information about prevalence (how common a disease is at one point in time). Exposure and disease are measured at the same time, so it is difficult to show which came first or that one causes the other. This study also is less helpful when studying rare diseases as it is likely that individuals will be missed in the sample size.
For example, “In 2023/24, about 3.9 million people over the age of 6 in England had a diagnosis of asthma.”
This study is commonly carried out via door-to-door surveying and other similar methods.
This study design also has a new term associated with it:
- Prevalence - the number of people with the disease in the population divided by the total number of people in the population at a given time.
There are some flaws with the cross sectional study design:
- Sampling bias - this occurs when some members of the population are more likely to be selected to participate in the study. This could happen, for example, if the study takes place during the day. This would then exclude individuals who work during the day, and would therefore skew the results of the study because these people cannot be included. This would then mean that the study would not accurately represent the population.
- Participant bias - this is when the participant answers or acts in a certain way because they think that is how the researchers want them to behave, or because they know they are being studied. For example, in a survey about diet, people may report eating more healthily than they do. This can be unintentional on the part of the participant. Participants may also not remember past events accurately (recall bias).
Cross sectional studies can still be used to test hypotheses, including a null hypothesis, for example when comparing prevalence between groups. However, exposure and outcome are measured at the same time. These studies cannot show which came first (the temporal sequence), so it is difficult to show that an exposure causes a disease.
Cohort studies are a common type of analytical study. They involve classifying the healthy participants based on their exposure and then investigating how many of each type go on to develop disease.
For example, imagine a study of the influence of mould in a home on the development of asthma. The participants would be put into two groups: those who had lived in a house with mould (the exposed group) and those who had not (the unexposed group). The mould is the exposure, which is the independent variable. The development of asthma is the outcome of interest, which is the dependent variable. The study would then follow both groups for a set period (for example 20 years) to see if and when they develop asthma. This is useful because it can show associations between exposures (mould) and conditions (asthma). It can help show that an exposure causes a disease.
Cohort studies can come in two types:
- Prospective cohort studies take participants who are free of the disease at the start, record whether each one is exposed or not, and follow them forward in time to see who develops the disease.
- Retrospective cohort studies use past records to put participants into exposed and unexposed groups, then follow them forward from that earlier date to see who developed the disease. The outcome may already have happened when the study starts. This is still classified as a cohort study because the participants are put in groups based on their exposure rather than on their disease state.
Cohort studies also use some specific terms:
Person years - the total number of years that all individuals in the study are observed for. A person is no longer observed when they die or when they leave the study. Person years is simply a measure of time. If 100 people are observed for 2 years, that is 200 person years.
Incidence rate - this is the number of exposed people who developed the disease, divided by their total person years at risk. It gives the incidence rate of the exposed group. For example, if 100 exposed individuals each contribute an average of 5 years at risk, this gives 500 person years. If 10 people develop asthma, the incidence rate is 10 / 500, which is 0.02 cases per person year at risk. While incidence rates are based on person time rather than headcount, the sample size still affects the precision of the estimate.
Incidence rate ratio - this is the ratio between the incidence rate of the exposed and the incidence rate of the unexposed. This shows how many times higher (or lower) the rate of new cases is in the exposed group than in the unexposed group.
Cohort studies also have many issues:
- Loss to follow-up - outcomes may be unavailable when participants leave a prospective cohort or when records are incomplete in a retrospective cohort. Fewer observed outcomes reduce precision. Loss can also bias the exposure–outcome comparison if the reasons for missing outcomes are related to exposure and disease risk. Record the reasons for loss and assess their relationship to both exposure and outcome. Examples include:
- Survivor bias - restricting analysis to participants who survive or remain available can select a healthier group. This may distort the exposure–outcome comparison, rather than merely make the sample less representative.
- Differential loss - the frequency or reasons for missing outcomes may differ between exposure groups. If loss is also related to the outcome, the estimated association may be biased. Even loss that does not introduce bias reduces the information available.
- Observation bias (a type of information bias, which includes observer bias and recall bias) - this is a systematic error in how the exposure or the outcome is measured or recorded. For example, participants may report their exposure inaccurately, or observers may record outcomes differently in the exposed and unexposed groups.
Cohort studies are usually expensive. They need large samples, to allow for people lost to follow up, and the resources to follow these large groups for many years. However, cohort studies are very helpful for producing a large amount of data. They allow incidence (the number of new cases over time) to be calculated.
The following considerations should be made when designing a successful cohort study:
The acronym PICO (Population, Intervention, Comparison, Outcome) can be used to remember what needs to be established when designing a cohort study. In a cohort study the ‘I’ is the exposure of interest, because the researchers do not give an intervention (a treatment). Some authors use the acronym PECO (Population, Exposure, Comparison, Outcome) instead.
Population - who will be sampled from?
Intervention - what is the exposure of interest
Comparison - who is the control group from the sample population (those who lack the exposure).
Outcome - what is the end condition being determined.
In our example
P - Children who live in x city.
I - the presence of mould in the family home.
C - children who live in the city in non-mouldy houses.
O - development of asthma in 20 years.
Case control studies are the final type of study discussed in this article and are an analytical type of study. They involve finding participants who suffer from the disease (cases) and those who do not (controls) and looking into their past to see if they have been exposed. This means that case control studies usually work backwards from the disease to past exposure, so they are retrospective in design. A nested case-control study (cases and controls drawn from a cohort that is already being followed) can use exposure data recorded before the disease began.
For example, participants with asthma and participants without asthma are included, and everyone is investigated to see if they have been exposed to mould in their houses previously.
This data can be put into a simple table, as seen below.
Table - Cases vs Controls in exposed and unexposed circumstances for a case control study
SimpleMed Original by Dr. Maddie Swannack
This table can then be used to work out the following:
- The odds of exposure in the cases - exposed cases divided by unexposed cases, or a/c. (Odds compare the number of people who have something with the number who do not.)
- The odds of exposure in the controls - exposed controls divided by unexposed controls, or b/d.
- The odds ratio is the odds of exposure in the cases divided by the odds of exposure in the controls. This is a/c divided by b/d, which simplifies to ad/bc. In words, (exposed cases x unexposed controls) is divided by (unexposed cases x exposed controls). An odds ratio of more than 1 suggests that the exposure goes with higher odds of the disease. An odds ratio of less than 1 suggests lower odds.
The issues with case control studies are the following:
- Selection bias - this happens when the cases and controls are not chosen in a similar way, so the two groups differ in more than just the disease. The controls should be like the cases in every way except that they do not have the disease. Choosing the controls well is a key part of the design. There is also no randomisation (random allocation of people to exposed or unexposed groups), because participants are chosen after the disease has developed.
- Recall bias - cases may remember past exposures more fully than controls. For example, the mother of a child with a congenital anomaly (a condition present from birth) is more likely to recall drugs taken during pregnancy than the mother of a child without one.
- Random error - chance can change the odds ratio. The confidence interval shows how large this effect might be.
Case control studies also have strengths. They are cost effective and do not require any follow up. They are also more useful than cohort studies for rare diseases, and for diseases that take many years to develop, because they start by selecting people who are already ill.
Case-control studies are susceptible to confounding, and the order of events can be unclear. When exposure is recalled after diagnosis, it can be hard to know exactly when it happened. This can make it difficult to establish whether exposure came before the disease. Early disease may also have changed behaviour, so the disease may have caused the exposure rather than the other way round (reverse causation). However, case control studies can show that exposure came first if there are reliable dated records, or samples collected before the disease began. Case control studies do not directly give incidence or prevalence. This is because the researchers choose how many cases and controls to include, so the numbers do not give the denominator (the total number of people in the population).
Confidence Intervals - Case Control and Cohort Study Designs
Study results are uncertain because of the flaws described above for each study design. A confidence interval shows how uncertain they are. A 95% confidence interval comes from a method that would capture the true value for the whole population (the population parameter) in 95% of repeated samples, if its assumptions hold. It does not mean that there is a 95% probability that the true value lies inside any one interval.
Figure: approximate 95% intervals for an odds ratio, an incidence rate ratio using event counts (the number of new cases in each group), and a binomial proportion (a proportion of people with a yes or no outcome). The intervals are based on the normal distribution (a symmetrical, bell-shaped spread of values). The figure uses 1.96. The case-control and cohort worked examples below use 2 as a rounded approximation to 1.96, so their numerical limits differ slightly from the figure. These methods work well only when the sample is large enough for the normal approximation to hold. The proportion panel warns about small samples and values near 0 or 1.
SimpleMed: corrected formula diagram (2026-10-01)
The confidence interval is worked out through a number called the error factor (EF). This is worked out using an equation specific to the study design, as shown in the figure above.
- In the case control study, the letters a-d in the error factor equation reflect the values in the table used in this study design.
- In the cohort study equation, d1 and d2 represent the cases in population 1 (exposed) and the cases in population 2 (unexposed).
The confidence interval (CI) is worked out using the error factor, and one piece of data from the study (notated here for clarity as the randomly chosen symbol ꝋ). To find the lower bound of the CI, ꝋ is divided by the error factor. To find the upper bound of the CI, ꝋ is multiplied by the EF. The CI is the range of values from the lower bound to the upper bound.
Working out Confidence Intervals
ꝋ is a symbol randomly assigned to represent the data chosen from the study to work out the CI. It stands for the following:
- The CI for a case control study uses the odds ratio (OR).
- The CI for a cohort study uses the incidence rate ratio (IRR).
As explained above, a confidence interval is a range of values that are compatible with the data from the study. There is some doubt because studies cannot guarantee to be correct 100% of the time.
Confidence Intervals - Cross Sectional Study Design
The equation used for working out the confidence intervals for cross sectional study design is different to the type used for cohort and case control studies. This is because in a cross sectional study, there is only one result (the proportion), whereas the other designs compare two or more groups. This equation can seem a little confusing at the start, but it will be gone through step by step.
Figure - The equation used to find the confidence interval of a cross sectional study
SimpleMed original by Dr. Maddie Swannack
This is the equation used to work out the confidence intervals in cross sectional studies. The larger brackets are added to make the order of operations clearer.
In this example, the study results will be that out of 500 households observed, 337 had at least one member of that household currently under 18 (called ‘child’).
This means we need to find the proportion of households with at least one child in them, out of the total sample size of 500 (represented by the letter n). This means we do 337 / 500, giving us a proportion of 0.674. In this equation, this represents the symbol p̂ (called ‘p hat’).
However, this sample might not be representative of the whole population. A confidence interval gives a more realistic picture of the proportion in the whole population. To calculate a meaningful confidence interval, your sample must meet two conditions:
- The size of your sample must be no more than 5% of the size of the population it was drawn from. This might seem counterintuitive but stick with it.
- The sample must be chosen at random, and be large enough for the normal distribution (a symmetrical, bell-shaped spread of values) to be a good approximation. The approximation becomes unreliable when the sample is small or when the proportion is close to 0 or 1. Here, 337 households had a child and 163 did not, so both groups are large.
This next section will be the same regardless of the study results, so long as they are using a 95% confidence interval:
Next we have to work out the 95% confidence interval. This means that there is 5% of the distribution not included in our confidence interval, called α (alpha). Using the diagram below, we can see that 95% of the distribution is in the centre, leaving 2.5% in each ‘tail’ (the two ends of the distribution). Together, these two tails make up the 5% of the distribution that is called α. This means that α=0.05 and α/2=0.025. Then, we use a Z table (a table of values for the standard normal distribution) to find how many standard deviations (units of spread) from the mean cut off 2.5% in each tail. This number is called Zα/2.
Graph: the standard normal sampling distribution. Approximately 95% lies between −1.96 and +1.96, with approximately 2.5% in each tail. Under the normal model, confidence-interval coverage (how often the intervals contain the true value) refers to repeated samples; the true parameter (the real value in the whole population) is fixed.
SimpleMed: new mathematical plot (2026-10-01)
For a central nominal 95% interval (one built to capture the true value 95% of the time) based on the standard normal distribution, the upper critical value (the cut-off point) is approximately 1.96, leaving approximately 2.5% in each tail. Other ways of calculating the interval may use different critical values or may not rely on the normal distribution.
Now we must put the data into the equation:
So now we have the information to plug our data into the equation.
p̂ = 0.674 - this is the part that changes based on the proportion
Zα/2 ≈ 1.96 for a 95% interval. This is the Z value that leaves 2.5% in each tail, so it allows for a difference in either direction (two-sided).
n = 500 - this changes depending on the sample size
Remember that the equation must be done twice: the first time using p̂ PLUS Zα/2 × √(p̂(1 − p̂) / n) to find the upper bound of the confidence interval, and the second time using p̂ MINUS Zα/2 × √(p̂(1 − p̂) / n) to find the lower bound of the confidence interval.
Diagram - shows the equations used to work out the upper bound (UB) and the lower bound (LB) of the confidence interval in our study.
SimpleMed original by Dr. Maddie Swannack
Working out Statistical Significance
You can tell whether a study’s results are ‘statistically significant’ by looking at the confidence interval and checking whether it includes the value of the null hypothesis.
If the value of the null hypothesis is within the confidence interval, the results are not significant, because the results of the study could indicate that ‘there is no difference’. This applies to all study designs that can have a confidence interval worked out (case control, cohort and cross sectional).
If a study is using a calculated ratio, the null hypothesis would be that the independent variable (the exposure or factor being compared in the study) has had no effect on the outcome. This is represented by the value of ‘1’, because no difference in a ratio is when the numbers are the same, so the ratio is 1:1.
If the confidence interval includes the number 1 in this case, the true value could be 1, meaning there could be no difference. This means that the null hypothesis should not be rejected, and the correct phrasing of this is: “The number 1 is included in the confidence interval, meaning that the confidence interval is large enough to include the null hypothesis. This means there could be no difference, the null hypothesis cannot be rejected, and the results are not statistically significant.”
If a study is not using ratio but is using the difference between two numbers (so the numbers are subtracted one from the other) then the null hypothesis should be represented by 0. This is because when subtracting numbers, the ‘no difference’ value is 0.
For a difference between groups, the null value is 0. If the confidence interval includes 0, the corresponding two-sided test does not reject the null hypothesis at the matching significance level (5% for a 95% interval). The data remain compatible with no difference; this does not prove that the groups are equivalent or that there is no effect.
Interpret a confidence interval against the null value: 1 for a ratio and 0 for a difference. The usual simple interval (called the Wald interval) is centred on its point estimate (the single best-guess value from the study) when it is used for a difference or a proportion. For a ratio such as an odds ratio, the Wald interval is centred on the logarithm (log, a scale on which ratios behave like differences) of the estimate. On the ordinary scale the estimate is then not in the middle: it is divided and multiplied by the same error factor, as in the worked examples below. Some other methods give intervals that are not centred on the estimate even on the log scale. Statistical significance depends on whether the interval contains the null value, not on whether it contains the estimate.
Examples of Confidence Intervals
A case control study classifies patients based on their disease state (lung cancer or not lung cancer) and investigates past exposure.
The results are as follows:
The odds ratio is ad ÷ bc = (15 × 27) ÷ (38 × 20) = 0.5329 to four decimal places.
For this case-control example, the error factor is e2 × √(1/15 + 1/38 + 1/20 + 1/27) = 2.3363. The multiplier 2 approximates 1.96 for a normal-based 95% interval. Because the exponent is positive, the error factor is greater than 1.
The upper confidence limit is OR × EF = 0.5328947 × 2.3363131 = 1.2450.
The lower confidence limit is OR ÷ EF = 0.5328947 ÷ 2.3363131 = 0.2281. Calculate with unrounded values and round only the final limits.
Using the illustrated approximation, the confidence interval runs from 0.2281 to 1.2450. It includes the null odds ratio of 1, so the data remain compatible with no association.
The confidence interval for the odds ratio includes 1, the null value for no association. The result is therefore not statistically significant at the 5% significance level (the level that matches a 95% confidence interval). We fail to reject the null hypothesis; we do not establish that there is no association.
A cohort study classifies people based on their exposure state (second-hand smoke) and investigates the occurrence of lung cancer.
The results are as follows:
In the exposed group, 127 of 295 participants developed lung cancer. These figures are hypothetical. In real studies the effect was much smaller: living with a smoker raised the risk of lung cancer in non-smokers by about a quarter (24%). If the stated average of 8 person-years is the average time at risk per participant, the total is 295 × 8 = 2,360 person-years. The incidence rate is then 127 ÷ 2,360 = 0.0538 cases per person-year, or 53.8 per 1,000 person-years.
In the unexposed group: 52/229 people developed lung cancer during follow up. The incidence rate should be calculated using the total person-years at risk in this group.
An incidence rate ratio compares the exposed and unexposed incidence rates. The unexposed group’s total person-time is not supplied here, so a numerical rate ratio cannot be calculated from these data.
This rate-ratio example is incomplete, because the unexposed person-time is missing, but the error factor can still be shown. Using 2 as a rounded approximation to 1.96 in the illustrated normal approximation, the error factor is e2 × √(1/d1 + 1/d2), where d1 and d2 are the event counts. With 127 and 52 cases, the factor is 1.3900. These are event counts, not the four cells from the case-control example.
Once both groups’ person-time is available, calculate the rate ratio and multiply it by the error factor for the upper confidence limit.
Divide the rate ratio by the error factor for the lower confidence limit. A positive-exponent error factor is greater than 1, so division gives the lower limit and multiplication gives the upper limit.
The missing unexposed person-time prevents a numerical rate-ratio confidence interval from being reported for this example.
The null value for a rate ratio is 1. Statistical significance can be assessed against it once the interval has been calculated; these incomplete rate data do not support a conclusion about significance.
Edited by: Dr. Marcus Judge and Dr. Ben Appleby
- 8990
