This contribution to ’52 Perspectives’ comes to us from Sherry Reuter, MS. Sherry is the President of Sherry Reuter & Associates and shares her thoughts about some of the commonly overlooked facets of site selection.
There are a multitude of factors that influence whether a site successfully conducts a particular clinical trial. Statistics indicate that the right criteria is not being considered when selecting sites, with on average 20% of sites participating in a trial never enrolling a subject and about 30% performing very poorly. And with these statistics getting worse over time, rather than better, it is important to question whether the right information is being considered during the current site selection process.
Pre-Trial Questionnaires have become very lengthy, requiring potential study sites to spend hours answering numerous questions. Then Sponsors somehow try to choose sites based on all this data. But how effective is all this work? While there has not been much research to shed light on which questions on Pre Trial Questionnaires most highly correlate with sites’ success, the poor overall performance of selected sites should prompt us to ask if the information currently obtained during site selection is really the best for assessing sites’ probability of successful conduct of an upcoming trial. Which information is critical to know, not as important or practically irrelevant?
For example, these questions are almost always included on Pre-Trial Questionnaires, but how meaningful are they in selecting sites?
Can sites use a Central IRB? – The thinking is that IRB approval will be faster for sites that can use a Central IRB. However, this generalization may not always be correct, as some local IRBs realize that they must shorten their turn-around time. And even if they take a week or two longer, is that really important in the long run if some sites rejected due to this could have done well conducting the study, while others selected because they can use a central IRB might do poorly?
Qualifications of site personnel – While this is important, their qualifications may not always translate into successfully conducting a specific study. Reasons this factor might not be a major influencer include: their qualifications might not apply to the specific study, the personnel might be working on so many studies that their potential cannot be realized or they might focus on other studies in process at the same time.
Past performance in “similar” trials – Protocols have increasing numbers of Inclusion and Exclusion Criteria, making the eligibility of potential subjects more and more specific. Even if a site was successful in conducting a study in a certain indication, just a few different Inclusion / Exclusion Criteria could make it much more difficult to enroll. Other reasons that enrollment might differ from previous trials are depletion of a site’s database, recruitment of other trials at the same time, and effects on recruitment by changes in standard of care or insurance coverage.
Plans for other trials at the same time as the study under consideration – Research conducted for a Masters Thesis by S. Reuter “How Effective are Research Site Questionnaires in Predicting Site Performance?” showed that data collected from this question was one of the least predictive. Sites have no way of knowing what will happen at some indefinite time in the future, especially when the date the study under consideration will definitely begin is unknown. And even if other studies are conducted concurrently, that does not necessarily mean problems; what matters is whether there is enough staffing for all the work, number of potential subjects in the database, access to subjects via other sources and how the site allocates potential subjects to each study.
Type of refrigerator / freezer at site – Equipment can be bought, but ability and motivation cannot.
It is well known in the pharmaceutical industry that one of the least reliable pieces of information is sites’ estimate of the number of subjects they can enroll in a trial. In fact, sponsors routinely cut that number by half or a third in this cynical dance of site selection. Dr. Louis Lasagna noted this phenomenon years ago:
Lasagna’s Law: “Investigators overestimate, many-fold, the pool of available subjects who meet the inclusion criteria and would be willing to enroll in a particular trial. The fastest way to decrease the number of patients afflicted with a disease or condition is to initiate a clinical trial”
The vast majority of Pre-Trial Questionnaires currently used consist mostly of questions calling for FACTS about the site’s infrastructure, staff and experience. But a site must not only have the ABILITY to successfully conduct a study, it must also have the MOTIVATION to do so. Even if a site has everything in terms facilities, experience and access to the study population, if the PI is not interested in or motivated to conduct the study, the chances are the site will perform poorly.
Comparatively little space and time are spent finding out what the PI really THINKS about the protocol and the site’s potential for successfully completing the study. There usually is only a question or two about this all-important issue on Pre Trial Questionnaires, and it might not even be the PI filling out the answer. And during the Pre Study Visit, what the PI really thinks about the trial is not discussed in depth. Most of the visit is spent checking off boxes on the Trip Report on facts about the facility, staff and superficial questions about conducting the study.
CRAs have not been instructed to obtain this information nor trained to do so. Soft skills such as communication techniques and people skills have not been valued as much as efficient completion of visits and checking boxes to quantify information. The problem is that the really important information needed for site selection, how the PI really feels about the study, cannot be found out quickly and quantified with checked boxes, but instead through open discussion.
About Sherry Reuter:
Sherry Reuter is President of Sherry Reuter & Associates, LLC, a consulting firm that focuses on the conduct of clinical trials, Site Selection, Study Start Up, Training, and Patient Recruitment and Retention. Sherry may be contacted at sreuter@gwu.edu or 203.775.6031.
