Medical Devices

A Tale of Three Medical Device Study Designs

Medical Devices

13:00, April 6 2017


Steve Sisk, Director Clinical Operations, CytoSorbents Corporation, analyzes three study designs that can be implemented in a medical device trial

The Question

Which study design is best for a Post-Market Device Efficacy Study – a prospective randomized clinical trial, a prospective cohort observational study, or a retrospective cohort observational study?

The Problem

510K approval and initial acceptance by a small group of interested physicians may only be the first step toward widespread use and for making the case for reimbursement, leading wider market acceptance. Oftentimes, device marketers hear physicians say, “I will only accept data from a Randomized Controlled Clinical study of this device.”This sentiment is largely based on a belief that an RCT is the gold standard and other study designs are inferior. An RCT can generate reliable, high quality data, but how much better are they compared to prospective or retrospective observational studies? Benson and Hartz (NEJM June 22, 2000) careful reviewed this topic and concluded that there were few differences in treatment effects between the two types of studies. However, the problem with 510K cleared medical devices is that they can be cleared by the FDA with the minimum ofsafety and efficacy data. This can put the device manufacturer in the circumstance of tryingto market a device with sparse data on its clinical utility, cost-effectiveness, or even safety data. This can, and often does create the situation where a newlycleared device is not unacceptable for reimbursement and cannot readily be used by the prescribing physician, even when it is very likely the best modality available due hospital purchasing requirements. 

To justify safety and efficacy on the cleared device, clinical studies are the chiefmeans to generate the needed data. But clinical studies can be long, expensive, and may not always yield convincing data.   Moreover, by the time that a suitably powered, randomized clinical trial completes, the device may no longer have the relevancy that it had two or three years earlier because medical devices have a much shorter life cycle than pharmaceuticals. Timely data is often the critical factor to bring them to market while the device is still relevant.

Generating the clinical signal

There are three general trial/study designs that can be used and each has its advantages and disadvantages; Randomized controlled clinical trial (RCT), prospective cohort observational study, and retrospective cohort observational study.

RCTs are considered the gold standard. Clean design, clear endpoints, data reflecting current practice, and well defined comparators lead to highly accepted data. On the downside, they are very expensive (must also pay cost associated with the control arm, slow to generate data, cost prohibits larger population, enrollment issues, higher regulatory hurdles than an observational, and may not be ethical. As an example, if the cleared device has been shown to be much better by observational studies, lives or quality of life may be lost within a control population.

A common model for a Prospective Observational Cohort Study employs Case controls or Historical Controls. These studies have lower regulatory hurdles, are less expensive (can be much less) than RCTs, may offer the most ethical modality and can use larger populations/more sites. Documentation can and should be as carefully controlled as an RCT.

Where a site uses historic case controls or a literature study is used as the “control”, finances that would be used to pay for the control group in an RCT can make a larger “treatment” group more affordable. The result being a larger population for the study. To reduce site bias, the site should use the device exclusively within the target population, the exception being where the physician feels an individual patient would not benefit. Huge comparator databases can be used. For this study design to be meaningful it is essential that a clean, recent, pertinent literature comparator exist,.It may be useful to build into the protocol the ability to use future studies that may provide better benchmarks than the one(s) chosen at the outset of the trial.  A potential downside is that these studies can also be lengthy because they still require that the site usethe device (prospective) so that data can be collected.  These studies will also require monitoring, EDC systems, clinical files, site selection/training, and can be nearly as complex as an RCT in terms of data handling. It should be noted that it can take some effort to findcomparator data that has the variables needed to make the case for efficacy or safety for your device. With Retrospective Cohort Observational studies, these studies utilize medical data when a device has previously been used (early adapting physicians). They will utilize a historical (non-treatment) comparator (case control or literature) and look at treatment differences between use and non-use at similar or the same institutions. Huge comparator databases may be possible. The study can be controlled only to the extent of ensuring existence and accuracy of the entered data versus hospital records. These studies are rapid (patients have already been treated/exposed to device), require only minimal regulatory approvals, do not require patient informed consents, and can be operated for a fraction of the cost of an RCT or a prospective observational study (in my work on cardiacinfections, the difference in per patient costs was about 100 fold, not including study operational costs).


The Randomized clinical trial is considered to offer the highest level of data quality, but it is much slower, very expensive, may have only minimal populations, and faces regulatory challenges compared with observational studies. A number of publications have made strong cases that a well planned prospective cohort observational study can generate data that is just as accurate. In a situation where an approved product will require safety and efficacy testing, it would make sense to consider the available trial modalities in terms of cost, data quality, and time to generate pertinent data. With adequate planning, you can save time, money, and generate as good or better data using an observational approach.



Steve Sisk, MSc Pharmacy

Director Clinical Operations,

CytoSorbents Corporation

Monmouth Junction, NJ 09952