We all know the importance of a complete clinical trial dataset, but ensuring all events are captured while accounting for every patient is often left until the end of a study. By then, retention activities should be referred to more accurately as a rescue effort. The efforts needed to locate missing patients become labor-intensive initiatives and come at a high cost both financially and scientifically. The activities are undertaken by trialists who are juggling many close-out activities or by vendors who often charge a premium since they know how critical a robust dataset is to a trial’s success. Based on experience with numerous large outcome trials, retention should be planned from end-to-end during a project and should be monitored by skilled staff that are committed to the success of the project.
Retention and missing data are critically linked. These interrelated issues can impact review and approval of trial results and may result in criticism of the trial quality including generalizability of the results.1 There have been numerous examples of poor retention leading to missing data, however, the regulatory response to a recent study shook the industry.2 The sponsor took on extensive efforts to re-engage patients, however, the lengthy delay in approval was a severe consequence to the company and potentially to patients awaiting new treatment. Although there are no established “gold standards” for minimum acceptable amounts of missing data, an unspoken rule is to ensure that no more than 1 percent of data are missing at the end of the trial. Reaching this goal is not easy but can be achieved by teams with the right skills.
Early Considerations on how to Retain Patients
For discussion purposes, focus will be placed on long-term outcome trials, however, it is important not to discount trials of shorter duration. When planning a trial, it is important to consider known factors, such as its length and design, the disease state, and most importantly, the skillset of the trial team. The number of study visits and overall trial length must be evaluated carefully before finalizing the protocol to minimize trial fatigue for patients and site staff. The highly-skilled team will also consider the voice of the patient during planning to better understand the burden of participation. The experienced trial team plans a study budget that provides sites with adequate remuneration for time and effort needed to find potentially lost patients.
Fair reimbursement for patients must also be considered since often patients must travel to the study site, may require childcare, and may even miss work time in order to participate. The team should consider local laws and guidelines in each country before inviting them to participate. It is well worth the time to understand if use of data obtained from the public domain is allowed and what alternate forms of searching may be done to find patients.
Careful selection of study sites within each region is critical, especially consideration of each investigator’s experience in long-term outcome trials. Checking on past retention rates and overall missing data is an important step towards predicting site performance. During the preparation stages, skills-based training for all site-facing and site staff must be created, delivered and provided as needed. In past trials, lack of understanding of terminology has been critical components to lost patients. For example, the term “withdrawal of consent” has been used as a catch all phrase and applied incorrectly.
As a result, site staff are fearful of violating patient rights by attempting to contact all patients randomized in an Intent to Treat (ITT) analysis. Experienced trialists challenge every case to look for true withdrawal and to identify clusters of reports per site and region. Major trials have been turned around by challenging study staff on each potentially lost patient and by speaking directly with principal investigators (PIs) to understand the details about each potentially lost patient.
Use of Retention Materials not always the Solution
Once the trial initiates, close monitoring of retention is as important as recruitment rates. Data review meetings at frequent intervals allow trial teams to take early action. Inexperienced teams often accept poor retention until the end of the trial, when rescue techniques are warranted. If the set threshold is exceeded, even in small increments, experienced trialists take immediate action and hold all stakeholders accountable for each patient in the trial. The ability to have targeted discussions with investigators, including the creation of an effective action plan, is a skill that is honed over time. Inexperienced teams may rely solely on patient correspondence and believe that sites will eventually find their patients.
Although there is great value in using retention materials, they are not the solution. Experienced teams use the materials as tools to be included in the overarching strategy. Knowledgeable trialists will also proactively identify regulatory and legal requirements in each region that may participate. It is important to challenge local “folklore” on these topics and to delve into what is actually allowed in each region. Some regions may need to be excluded if use of information found in the public domain (e.g. newspaper obituary) or other forms of follow-up is not allowed. Ethics Committees (ECs) and Institutional Review Boards (IRBs) should also be consulted for local requirements and it is important to stress that missing data may not allow them to conduct thorough periodic safety reviews on behalf of patients.
Leveraging New Technologies
During study conduct, early and constant retention monitoring is necessary and risk mitigation plans should be implemented even with minor increases in potentially lost patients. Targeted efforts include one to one discussions with PIs and their staff, including the development of an effective action plan. Based on informal interviews with less experienced teams, there is often a lack of direct communication with PIs and therefore, clear expectations are not presented to site staff. Accountability is another important concept which may be reinforced by engaging lead investigators and steering committee members.
Often calls between a lead investigator and a PI with low retention may have positive outcomes. The use of newsletters that include names of PIs with low retention rates can be also be impactful. With experience, trial teams can pull key influencers into site discussions before the volume of missing patients becomes insurmountable. Currently, the use of digital technologies is becoming effective retention tools but may only be effective when carefully implemented.3 The use of new technologies reduces the overall burden on site staff as well as patients, however, implementation must be well-planned to ensure compliance with the protocol and to deter patients from diminishing the importance of required physical evaluations.
The Ultimate Success Story
As the study comes to a close, retention efforts may intensify but should be focused on a small number of patients. Neither retention materials nor digital technologies can outweigh well-planned end-to-end strategies that are implemented strategically from start to finish. Although closing out a trial requires attention to many details, the most successful teams can deliver a robust dataset for analyses with minimal amounts of missing data. The final success factor is for teams to reflect on achievements and opportunities for future trials. Every aspect of retention is a learning opportunity and the knowledge gained should be shared with less experienced teams.
In summary, retention requires a variety of skills that are drawn upon during the stages of a trial. Proactive approaches can minimize costly rescue efforts, the need to have uncomfortable discussions with investigators and will allow teams to focus on other key operational activities. While the use of newsletters, patient correspondence and digital campaigns are important tools, the ultimate success of a trial is based on the efforts of an experienced trial team, who understand which techniques should be applied throughout the trial, and who are willing to hold stakeholders accountable. If approached proactively and monitored regularly, not only will the trial be successful, but patients awaiting novel therapies, may be the ultimate success story.