Question 7: Dealing With Switchers
7 / 52
This post is a part of the Statistical 52, a weekly (lol) series where I write about questions and concepts that I think aspiring statisticians should be comfortable with.
Question
You are in charge of analyzing the data from a randomized controlled clinical trial. Some people in the intervention group stopped treatment due to side effects. Furthermore, you found out that some people in the placebo were given treatment due to worsening conditions. How should you handle these switchers in the analysis? How would you justify your approach?
Discussion
TLDR: Randomized controlled trials get much of their power from randomization. You need to analyze people how they are randomized, not how they end up in the trial.
Randomized controlled trials are considered to be the gold standard in terms of providing evidence that a treatment has a relationship to some disease. That is, they provide evidence for a causal relationship between the treatment and the disease. For a more detailed explanation behind this, you can check out this old video of mine.
Randomization is one of those things that is nice to have in theory, but it’s also an area where reality often has other things to say. Sometimes, you’ll have treatments whose side effects are too much for the participants to bear and they will stop taking the treatment mid-trial. Other times, people in the placebo groups may realize they’re in the placebo group and will seek out alternative treatments. With human participants, we have to make sure they are treated ethically.
When all is said and done, you’ll have data to analyze. Assuming that your data is rigorously maintained, you’ll know what group each person was randomized to as well as any other treatment-related events that happen to them.
On one hand, it’s tempting to analyze people based on the treatment they ended up in. The outcome you’re measuring is related to them being on what they’re taking, so that should be good right?
Wrong.
This is called a “as-treated” analysis, and it is very likely to bias your results. In our example, the treatment group is likely to drop because of side effects, which may or may not be related to the disease itself. We should strive to put the treatment in the most honest light, not the one that we personally want. This way of analyzing the data breaks the randomization, so it is not preferred.
Another way you might approach the analysis is to remove anyone who stopped their assigned treatment. They did not perfectly adhere to the trial protocol, so they bias the results. Hence, only including the completers should be reasonable… right?
Nope!
This is called a “per-protocol“ analysis, and it also has the potential to bias your results. If people are doing worse because of the treatment, then this should rightly be accounted for in the analysis. Similarly, if people drop out of placebo due to how severe the disease is, ignoring them may inadvertently make the treatment look more good than it actually is. When people drop out, it is important that we strive to keep collecting follow up data from them or have a solid way to impute their missing data.
The best way to analyze the data would be to preserve the randomization and keep each person in the group that they were randomized to, even if they switch or discontinue. Randomization plays a central role in breaking any and all confounding that may be present, enabling us to get to a causal effect. Even if people switch treatment, we are in effect analyzing the intent to treat. As such, this is called an “intent-to-treat analysis”, and it is the way that we should analyze data (and in fact it’s what the FDA demands).
To end this issue, I’ll end with a quote by Sir Austin Bradford Hill, who helped to pioneer the modern randomized clinical trial as we know it:
In many trials the original careful randomization of patients to treatment and control can be later disturbed by selective withdrawals of patients who cease to take a treatment or are proved sensitive to it so that they have to be withdrawn. The experiment is necessarily weakened – indeed we may on occasions have to assess the value of an intent to treat rather than a treatment.
Were you able to answer it correctly? Was there anything that surprised you? Let me know in the comments! See you in the next question.
📦 Check out my other stuff!
Read through my Statistical Garden on the Very Normal website! This digital garden houses all the knowledge I gained as a biostatistics graduate student. It’ll grow as I learn more, and it’s free for you to look through.
You can support me on Ko-fi! YouTube and Substack are the best and easiest ways to support me, but if you feel like going the extra mile, this would be the place. Always appreciated!