Why are metrics important to starting clinical trials? This question may seem counter intuitive, as we are exposed almost daily to the dire performance of clinical trials and their spiraling costs resulting from incurred delays.
According to a study by KPMG, within the pharmaceutical industry, the return on R&D expenditure has fallen from an industry average of approximately 20 percent 20 years ago, to 10 percent now. This is with the average cost of developing a drug rising during that period at a rate of seven point four (7.4) percent higher than inflation, with the increasing costs of conducting clinical trials responsible for most of this increase. It is estimated that it now costs upwards of $2 billion dollars to bring a new drug to market.
Perhaps most disturbing fact is that cycle times associated with starting clinical trials (i.e., steps involved in study startup, such as the selection of sites to conduct the study and activation of the site to receive a first subject) have not changed in more than two decades. According to the Tufts Center for the Study of Drug Development (CSDD), 37 percent of sites selected for clinical trial studies under-enroll and 11 percent fail to enroll a single subject. Eventually, 89 percent of studies meet enrollment goals. But this is often at the expense of sponsors faced with doubling the original timeline due to poor enrollment. Additionally, it takes an estimated eight months to move from pre-visit through to site initiation, with the associated cost of initiating one site ranging from $20,000 to $30,000. Overall, poor site selection, the inability of sites to predict the rate of enrollment, and the subsequent need for study rescue may increase cost of trials by 20 percent or more.
Metrics are indeed critical to efforts for reining in clinical trials that are either poorly initiated or have incurred unforeseen events which place the original timelines and/or budgets at risk of overages. They also drive competitive performance among those organizations performing trials.
Metrics provide the foundation for business intelligence (BI) and afford clinical research teams an opportunity to intervene before the effects of a risk have occurred. This risk mitigation is therefore optimal using systems which can provide timely, preferably real-time data on trial bottlenecks, which indicate red flags to be reviewed and addressed, or at least tracked carefully throughout the trial.
BI has become an increasingly popular topic in clinical trials as clinical project managers are expected to make smarter decisions on intelligence derived from clinical trial data. Sponsors and contract research organizations (CROs) are looking for ways to incorporate BI into the eClinical systems that they are already using to empower oversight to turn raw trial data into actionable information.
By 2020, 72 percent of clinical trials are anticipated to be outsourced, up from just 23 percent in 2012. With this in mind, technology that can provide sponsors with real-time insights into clinical operations is essential. This technology should also provide CROs with automated alerts for workflows. and it should offer sponsors multiple reporting options, including on-demand static reports, snap-shot reports with status data that can be manipulation for further analysis, and full access.
Having technology which can automate or assist in the timely monitoring of trials is a huge improvement over the current status quo of manual methods such as spreadsheets, which are cumbersome and erroneous, and not to mention, only provide a dated, snap-shot of trial performance. But how do metrics drive performance competitiveness?
Benchmarking of trial data allows clinical research teams to gauge their performance and progress against internal data, as well as, external information (i.e., trails run by other organizations). It allows them to see at a glance if they are on par with past trials that the organization has run of a similar size, geographic footprint, therapeutic area, indication, etc. If not, why not?
But equally as important, and maybe arguably more so, is how is the clinical research team performing against other organizations? This is particularly important in the case of a CRO vying for an outsourced pharma study contract. It is also critical for a pharma organization that needs to justify the outsourcing of trial work by capturing the benefits that the benchmarking information could bring, while allowing the pharma team to refocus on their core competencies of research. A review of benchmarking data may indicate red flags not otherwise raised during the monitoring of the trial and may be country specific. But, benchmarking is not without its challenges.
From an internal perspective, organizations can capture cycle-time metrics on whichever artifacts they deem important to measure. As long as these metrics have clear definitions and are measured consistently between trials, these measurements then become internal benchmarks for gauging future trials.
For external purposes, allowing organizations to gauge performance against one another requires clear, consistent and concise industry-wide standards. This ensures a true apples to apples comparison. It has the added benefit of improving trial data quality, because data that might not have been previously recorded, such as certain start or stop dates, is now required. Negative cycle times or cycle times that are outliers should be reviewed to ensure accurate data entry.
Interestingly today, a number of organizations claim to have the ‘best cycle time metrics in the industry’. This is a claim which should be treated with much skepticism in the absence of globally recognized standards. With standards in place that can be applied across all studies, global milestones need to be utilized.
Global milestones are important because they recognize that the nomenclature of artifact naming conventions is not consist across organizations, or even countries, and nor will they ever be. For example, these terms are dependent on an organization’s standard operating procedures (SOPs) in which the events Activated, IP release, and Site Initiated could be synonymous. Nevertheless, what is important is that these cycle-time metrics can be accurately measured and mapped to an industry defined standard.
With industry standards and global milestones in place, the goal of benchmarking in clinical trials is achievable. But is this the end of the story? No, in reality it is just the beginning.
Benchmarking allows for gamification, which could be extended beyond country or CRO (if the trial is outsourced to multiple vendors) to be based on role assignment, with associated financial incentives. Some might view this option as unethical or raise questions of quality and/or of individuals potentially gaming the system to reap the rewards and accolades of exceeding the threshold of industry performance for their position. But nevertheless, it is a logical progression and many of these arguments don't carry much weight in a system with numerous check and balances. Moreover, it allows for greater transparency in the process of conducting clinical trials and would allow management the opportunity to highlight those clinical research associates (CRAs) and others who are star performers.
Gamification in the pharmaceutical industry has been used to improve relationships with patients by using games to encourage disease management, with Sanofi, Boehringer Ingleheim and Eli Lilly, developing apps. In the context of clinical trials, gamification presents an excellent opportunity to improve performance and reduce costs. There are a number of areas that hold promise, including: patient recruitment, patient retention, disease research, investigator and site training, and site performance improvement.
Benchmarking would also allow for efficient resource allocation. A review of subpar performance may indicate that any inefficiency is simply due to staffing issues. It could , afford executives the option of either allocating more staff to critical steps in the progress in project management terminology -- called crashing the schedule -- or opting to incur the subsequent financial ramifications from a delayed market launch. Ultimately, sponsors stand to lose up to $8 million dollars daily due to a trial delaying a product's development and launch.
Lastly, benchmarking is the precursor to predicative analytics or forecasting. It enables clinical research teams to estimate future outcomes based on their current state of progress, and prescriptive metrics which provide intelligent recommendations for optimal steps to achieve the desired outcomes and reduce timelines and costs. This is critical to risk mitigation and a preemptive weapon in the fight against the dreaded rescue study.
CROs, often seen as the innovators in clinical trials, are leading this charge into the BI foray.
Top CROs have been aggressively acquiring data sources to leverage for data mining. PPD acquired Acurian to gain analytics-driven feasibility capabilities, LabCorp acquired Covance for collective data resources to drive greater R&D productivity, and Quintiles merged with IMS Health to improve clinical trial execution using patient data.
Informatics is the new frontier in CROs’ innovative efforts, as they look to gain insights into operational data and drive improvements via targeted enrollment efforts.
What is the common thread? We now operate in a data driven environment.
Revised and abridged version of article published in Drug Development and Delivery, February 2018
Learn more about optimizing your study startup.
Contact us for a conversation.