Should we condemn small clinical trials?

15th December 2016 (Last Updated July 18th, 2018 14:04)

Debora Dongo-Soria questions the merits of small clinical trials

Should we condemn small clinical trials?

Investigators are often urged to conduct large clinical trials, however, a limited pool of patients make it very difficult or near impossible to do so. Even though some people tend to disregard small studies due to their lack of statistical significance, they are common and useful in certain circumstances, such as rare diseases, unique study populations, emergency situations or isolated environments, or when there are no alternatives.  

Although the definition of “small” depends on each study, it is more likely the patients will share several unique characteristics, such as exposure or disease. However, small clinical trials generally raise some concerns. The main problem is related to the interpretation of results, especially confidence intervals and p-values. Also, as a small number leave much to chance, it is difficult to determine proper cause and effect relationships. Another limitation is that outcomes may not be generalized and it may be hard to identify side effects and safety concerns. As such, small clinical trials can produce false-positive results or it might be possible to over-estimate the scale of an association.

Due to these uncertainties, small clinical trials require as much or more thought than traditional studies. And though small, they still need to be sufficiently designed and accurately analyzed to obtain good, reliable results.

There are innovative design and analysis approaches that can improve the quality. The Committee on Strategies for Small-Number-Participant Clinical Research Trials at the Institute of Medicine in the United States suggests the following to avoid statistical errors and faulty data:

  • To carefully define the research question before conducting a small clinical trial
  • To tailor the design according to the research question and the study population
  • To clearly describe all sample characteristics and methodology of data collection and analysis
  • To perform alternative statistical analyses to evaluate the robustness of the results  (including sequential analysis, hierarchical analysis, Bayesian analysis, decision analysis, statistical prediction, meta-analysis, and risk-based allocation)
  • To be cautious when interpreting the results, especially before attempting to generalize

The Institute of Medicine also recommends doing more theoretical and empirical research on alternative designs that can be applied to small studies.

Even though small clinical trials do not always provide reliable estimates, they offer some benefits compared to large, traditional studies. For example, they are quicker to conduct in terms of enrolling patients, obtaining ethical approval, and getting the data. They can also be conducted over a few sites only and are less expensive.

Some people in the industry acknowledge the shortcomings of large clinical trials and argue that they fail to account for variability between patients, which would cause some drugs to be ineffective and very expensive. That is why some experts advocate for more N-of-1 clinical trials, which focus only on one patient to identify specific characteristics for the drug to be effective, such as the right dosage or possible causes of side effects. The data of each single patient with similar genetic and physiological profiles can then be analyzed together to identify correlations. By focusing on target patients, reducing drug use among patients and not having to conduct large-scale clinical studies, drugs could become more accessible and effective for patients.

If each clinical study is well-designed and properly conducted, the size will not be an issue. The only thing that will be mandatory in all trials, especially small ones, will be to interpret the results carefully. This means researchers should be careful not to come up with strong conclusions, regardless of the results, and that they should give careful consideration to the assumptions obtained. Ideally, the data from small studies should be used in larger studies, when possible, to confirm results. 

 

References

Small Clinical Trials: Issues and Challenges, Institute of Medicine; Board on Health Sciences Policy; Committee on Strategies for Small-Number-Participant Clinical Research Trials – http://nationalacademies.org/hmd/~/media/Files/Report%20Files/2003/Small-Clinical-Trials-Issues-and-Challenges/SmallN6pager.pdf