Apply Real-World Trials to Enhance Evidence Capabilities


14:00, January 20 2017


Jesus Irazabal, Novartis, discusses the challenges of implementing real world research and how it can be applied throughout a product’s lifecycle

I’ve always believed that for a better understanding of some ideas and/or opinions, it is key to gain insight into where those opinions are coming from. Therefore, I’ll begin by providing the context on which this article is written.

I’m writing from my own experience having worked over the last 15 years in clinical research management, always from the sponsor position. The projects I’ve been involved with range from small to large size endeavors. They are always multinational, some global, others regional, but always within Canada and Latin America; most of them prospective in nature. Now that you know where I coming from let’s get right to it.

Without question the randomized clinical trial (RCT) has been the pillar of evidence on drug development, as well as regulatory reviews and approvals, and the main argument on efficacy and safety. However, it is not until recent years that adding other sources of data/evidence has started to take a hold both on regulatory and stakeholders’ decision-making.

Certain conditions favor this: Patients and clinicians live in the real world of the hospital or office practice, not in the abstract, controlled environment of an RCT. Alternative sources of Real World Data (RWD) and Real World Evidence (RWE) are increasing, as we can see by a simple search on What’s noticeable is that there’s been a consistent increase in new observational trial postings, going from a little more than a thousand in 2006, to almost 5,000 new postings in 2015, with no sign of decline in 2016. Finally, the fact the demand for RWE by stakeholders is increasing and the timeline for data needs has shortened, study sponsors are more motivated to design, plan and execute real world data generating projects.

We have mentioned RWD, RWE and Observational Studies. Let’s formally define them, for the purpose of this article (please note that you may find several definitions;theseare the ones that from my point of view better reflect the point I’m trying to make):

Real World Data: Defined as “data collected from sources outside of traditional clinical trials, including registry studies, retrospective database studies, case reports, and routine public health surveillance.”1

Real World Evidence: Defined as the “evidence derived from aggregation and analysis of RWD.”1

Observational Study: “Study that provides estimates and examines associations of events in their natural settings without recourse to experimental intervention.”2

We have defined RWD and RWE, mentioningsome of the conditions that currently favor the use of it in order to make decisions, not only in clinical practice, but also both on the regulatory and administrative side of things.

At this point, it is logical to present some of the “barriers” yet to be jumped by this data, which I believe are based on preconceptions coming from the well-known RCT execution world. I call it “The RCT Paradigm”. This isnone other than aset of rules, norms, methods and expectations surrounding RCTs and its deliveries, which frequently serve as the comparator/control or measuring stick to an observational trial proposal, whichare completely different. Let’s make a point clear: RCTs are clearly the gold standard for efficacyand safety evidence, as determined by the “Evidence Heriarchy.”3However, it must be recognized that the RCT model has its own limitations and can be challenged.Remember, RCTs are an “Experiment” designed to provide outcomes on an idealized environment on a limited population.

Today luckily, attitudes toward RWE are changing.There is acceptance and relevance are increasing, obviously driven by better “Observational Study methods, more sophisticated use of data sets, improved trial design and more robust statistical method” as the stepping stone for this new attitude4.

Authors like Britton (1998); Concato (2000); Ligthelm (2007) provide grounds for the use of observational studiesto complement RCTs. Britton goes even further and states, “a well-designed nonrandomized study is preferable to a small, poorly designed and exclusive RCT.”4,5

Another situation favoring this attitude change is the fact that guidelines have been developed in recent years providing a more robust system for design, conduction, analysis, reporting and even review of this data generated from the real world as the Safety Assessment of Marketed Medicines guidelines (UK, 1994); the European Post-Authorization Study Guidelines (EU, 2006) and the Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices-Draft (FDA, 2016).

Now that we have a better understanding of the RWD and RWE generation, if you plan to embark on Real World Research, it is important to understand there are challenges and opportunitiesrelated to it. The “RCT Paradigm” and the preconceptions defined above are a big part of the challenges that need to be overcome. Additionally, there area number of design challenges, such as broad targeting (trying to do too much with it) and the heterogonous nature of the collected data that make it harder to aggregate as a whole. These are some of the issues the study sponsor should try to avoid. On the other hand, evolving methodologiesand changes in attitude favor RWD use as the needs of stakeholders increase the demand for RWD/RWE. Additionally, technological advances, ease collection of reported outcomes, and the global trend for electronic health records (HER), are all on the opportunities column.

Now let’s see how RWD fit into the big scheme.

The key word here is ”complement.” Sometimes the nature of RCTs could very well leave data/evidence gaps: Treatment Compliance, Resource Utilization, QoL, etc. This is where RWD comes to play a gap filling role. By implementing a well-designed “Real Life” study, rich data could be obtained that would complement, show and support the added value of a treatment and/or procedure. Some examples of that are presented by Lleven et al (Oct 2007)6, signaling out routine uses of real life data on: Patient profiling and prevalence; Treatment Flows-Patient Journeys; Treatment Compliance and Patient Persistance; Treatment costs as well as costs in disease stages and finally Health Outcomes and long term disease sequelae, for which prospective long term observation is a viable solution.

Other examples of real life designs and their influence are: RWD can very well support key results from Phase III RCTs4, as in the case of Wirth’s PMS with Orlistat where a 10 times larger population supported previous RCT findings7,8. How treatment outcomes with a particular drug compare with those from a number of alternative therapies, as the PURE Study (NVS Sponsored) Psoriasis registry Canada and Latin America, treat with secukinumab, or one of the other indicated therapies regimens approved for the management of moderate to severe chronic plaque psoriasis9. Head-to-head comparisons of treatment on the basis of particular clinical endpoint, as in Osteoporosis large (n=33K) comparative retrospective analyses, such as the REAL study, are an important means of filling gaps in the evidence base. It is unlikely head-to-head comparative clinical fracture trials would be done4,10 and finally treatment outcomes in heterogonous population with complex chronic diseases as in diabetes the PREDICTIVE(2007), EDGE (2013) are examples of large (n>30K) prospective global observational studies4,11,12.

In the hope that this provides a better understanding ofthe considerations to bear in mind when working in real world research, the key messages are:

  • RWD/RWE fill the gaps left by classical CRTs
  • Acceptance of RWE is growing, but additional work needs to be done
  • Policymakers are considering and learning the value of real world generated data
  • New technological advances allow for better and reliable data collection in the Real World/Real Life Setting
  • Every tool has its limits;pros andcons must always be considered when trying to design a solution to a problem
  • Real-world research can be applied throughout a product’s lifecycle, as shown in theexamples above.


*Jesus Irazabal, M.D., is the Medical Affairs Clinical Research Associate, Director, Latin America & Canada Region at Novartis

Disclaimer: The opinions expressed herein are my own, based on my personal experiences and do not necessarily represent those of Novartis Pharmaceuticals



  1. FDA draft guidance: Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices. July 2016
  2. Mann CJ. Observational research methods. Research design II: Cohort, cross sectional, and case-control studies. EmergMedJ. 2003;20:54-60.
  3. US Preventive Services Task Force. Guide to Clinical Preventive Services: Report of the US Preventive Services Task Force. 2nd ed. Baltimore, Md: Lippincott Williams and Wilkins; 1996
  4. Lighthelm. Importance of Observational Studies in Clinical Practice. Clinical Therapeutics. 2007;1284-1292
  5. Britton A, McKee M, Black N, et al. Choosing between randomised and non-randomised studies: A systematic review. Health Technol Assess. 1998;2: i-iv, I-I 24
  6. Lleven et al. Real Life Data a growing need. ISPOR Connections. Available at:
  7. Wirth A. Reduction of body weight and co-morbidities by orlistat: The XXL-Primary Health Care Trial. Diabetes ObesMetab. 2005;7:21-27.
  8. Toplak H, Ziegler O, Keller U, etal.X-PERT: Weight reduction with orlistat in obese subjects receiving a mildly or moderately reduced-energy diet: Early response to treatment predicts weight maintenance. Diabetes ObesMetab. 2005;7:699-708
  9. Data on file.
  10. REAL Study press release (April 3,2007). Call for greater use of comparative effectiveness studies to help advance disease management.Available at: 2OO 7-O4/k-cFg04030 7.php. Accessed April 10, 2007.
  11. Dornhorst et al. Safe W and efficacy of insulin detemir in clinical practice: 14-Week Follow-up data From type 1 and type 2 diabetes patients in the PREDICTIVE European cohort. Int J Clin Pract.2007;61:523-528
  12. Mathieu. Int J Clin Pract, October 2013, 67, 10, 947–956. doi: 10.1111/ijcp.12252