The concept of pragmatic clinical trials (PCTs) dates back to the 1960s (Schwartz and Lellouch, 1967). Many “large and simple” trials conducted in 1980s and 1990s in cardiovascular disease can be considered as PCTs . Current PCTs are primarily designed to compare strategeies for the prevention, diagnosis and treatment of diseases, conducted by academic medical institutes such as Patient-Center Outcome Research Institute (PCORI). Recently a new interest of PCTs emerges for marketing authorization application of new drugs with an increasing regulatory attention to using real world evidence in drug approval. GSK’s Salford Lung Study (New et al, 2014) was the first phase III pragmatic clinical trial supporting registration of the new drug. EMA Adaptive Pathway (EMA, 2016) encourages the use of PCTs to generate real world evidence (RWE) for the final drug approval. We expect PCTs will be increasingly adopted into new drug development as regulatory acceptance or requirement of RWE evolves. This session will discuss the future use of PCTs for the new drug or new inidcation approval from pharmaecutical industry, regulatory and academic aspects. Unlike the traditoinal randomized controlled trials, PCTs enroll the diverse population, follow the real world clinical practice, do not enforce adherence of patients and physcians, compare the real world different treatment alternatives. The appropriate guideline to design, conduct and analyze the PCTs should be developed before the PCTs can be used for drug registration purpose. How PCTs play the role in drug development and approval process is worthy investigating further in the near future.
Clinical trials with adaptive designs (ADs) use accumulated subject data to modify the parameters of an ongoing study without compromising the integrity of the study. These ADs are employed with the goal of being more efficient than a standard design, with efficiencies coming from various aspects, for example, possibly increasing power with fewer subjects or moving a compound through clinical development more expeditiously. With two guidance documents published by the Food and Drug Administration and theoretical advancements, it is of interest to review how adaptive designs are carried out in practice to understand any possible barriers to AD utilization so that we can continue to move forward with valuable clinical trial innovation. In this session, two parallel sets of surveys will be presented. The first is a set of four consecutive surveys conducted by the Drug Information Association (DIA) Adaptive Designs Scientific Working Group (ADSWG) Survey Subteam. These four surveys each span a four-year period (2000-2003, 2004-2007, 2008-2011, and 2012-2015) with earlier results being published by Quinlan et al in 2010 and Morgan et al in 2014. These four surveys have a structure of consistent questions for better understanding of the AD usage trends while a group of questions were added in later versions to better understand current circumstances. Responders include both industry and academic institutions. In addition to the first set of surveys, reviews of literature and registry entries over the four-year intervals were also summarized for wider understanding of current and past acceptance of ADs. The second set of surveys to be presented was conducted across centers (including CBER, Center for Biologics Evaluation and Research and CDRH, Center for Devices and Radiological Health) on what reviewers see through various submissions as well as challenges and pitfalls. Caroline Morgan will serve as discussant.
The Clinical Trials Transformation Initiative Data Monitoring Committees Project, which aimed to 1) describe the current landscape of DMC use and conduct, 2) clarify the purpose of and rationale for using a DMC, 3) identify best practices for independent DMC conduct, 4) describe effective communication practices between independent DMCs and trial stakeholders, during all phases of DMC activity, and 5) identify strategies for preparing a robust pool of DMC members, conducted a survey to assess current use and conduct of DMCs and assess training practices for DMC members and convened focus groups to gain an in-depth understanding of needs and best practices related to DMC use. High level survey and focus group findings will be presented.
Based on data gathered via these evidence-gathering activities and feedback from discussion at an expert meeting the project convened, the project team, made up of a diverse group of stakeholder from across the clinical trials enterprise, developed recommendations intended to improve the quality and efficiency of DMCs. The recommendations will be presented and will cover: 1. Role of the DMC, including issues related to DMC access to blinded data and independence; 2. DMC Composition, including issues related to conflict of interest and use of patient advocates in DMCs; 3. Communication related to the DMC, including communication between the DMC, trial sponsor, statistical analysis center, IRBs, and regulatory bodies; 4. DMC Charter, including a sample Table of Contents and points for consideration; and 5. Training DMC members and statistical analysis center statisticians, including suggested training formats and apprenticeship opportunities;
Additional tools related to the following will also be presented: 1. Specific responsibilities of the DMC 2. Best practices for conduct of DMC meetings
Speaker 1: Karim Calis, FDA Speaker 2: Ray Bain, Merck
In the 2014 concept paper for the ICH E9 (Revision 1) “Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trial”, two problems have been identified: the incorrect choice and unclear definitions for estimands and the absence of a framework for planning, conducting and interpreting sensitivity analyses. These problems could lead to inconsistencies in inference and decision making within and between regulatory regions. Consequently, an ICH E9(R1) Expert Working Group has been formed to provide recommendations on these problems, with a draft Addendum expected to be released in the second half of 2017. Following the initial recommendations, this session will discuss the intended impact of the suggested framework and any challenges for broad implementation in clinical trials. Topics to be approached include types of estimands related to various study objectives, statistical methods to be employed to handle different choices of estimands, and defining a set of sensitivity analyses for each estimand. Estelle Russek-Cohen as a discussant.
Former FDA Commissioner Robert Califf challenged us at last year’s workshop about how our National Clinical Research System is flawed. One problem he highlighted is that we are not focusing enough on how to decide who benefits from which treatment. To specifically answer Dr. Califf’s question, we have started sharing information across companies, e.g., using platform trials which not only compares treatments from different companies in the same trial but also incorporates important biomarker information. On the other hand, this question is also being addressed with how the FDA uses Drug Trial Snapshots to provide consumers information about whether there were any differences in the benefits and side effects among sex, race and age groups. Clinical researchers may expand on this approach by exploring additional subgroups to provide more information to the FDA for consideration of being included into communication with consumers. FDA speakers will present their views on how patients should be informed about expectations of how a drug should perform, both in terms of average treatment effect and on specific subsets of the population. Academic and/or industry leaders will present case studies where emerging subgroup identification methods provide reproducibility for identifying patients who benefit from a treatment. Since the numbers of patients in some groups are too limited to allow meaningful comparisons, another topic to be discussed is how to combine clinical trial data and real world evidence to improve subgroup findings based on clinical trial only.
Meaningful clinical benefit in the drug development should evaluate how patients “feel, function and survive”. How patients feel and function are usually captured by patient-reported outcomes (PROs). New development in oncology compounds are showing unprecedented efficacy using objective efficacy endpoints (survival and radiographic endpoints). PRO data would complement the objective efficacy endpoints to characterize the patient experience. Almost every oncology and hematology pivotal trial has PRO data; however, given the unique trial design characteristics in cancer trials, PRO endpoints could be potentially biased. They are therefore rarely used as primary or key secondary endpoints and neither are they included in the drug label. Challenges for assessing PROs in oncology include but are not limited to: lack of agreed upon instruments (questionnaires); trial designs not optimized for PROs; and lack of standardization in data analysis and data presentation. There is renewed interest in optimizing collection of high quality patient-centered data in the benefit-risk determination for cancer drugs. It is critical that we understand and find ways to mitigate the challenges associated with PRO data obtained in cancer clinical trials. In this session, we will invite speakers from FDA and industry to share their experience and discuss how PROs in oncology and hematology trials can be used to support overall drug development and regulatory approval.
Physicians often compare medical test results from an individual patient against a set of results from those deemed to be in good health, for the purpose of deciding how to manage medical care for the individual patient. This set of medical test results from those in good health is called a reference database. A common summary of reference databases is the reference interval, which is an interval that encompasses a specified percentage of the data in the reference database (e.g. 95%). Though the aforementioned comparison appears to be straightforward, there are many intricate study design and statistical issues associated with reference databases, including how to select individuals for a reference database and how to derive reference intervals (e.g. linear regression, quantile regression, nonparametric estimation of stratified data). As medical tests are becoming more sophisticated (e.g. reference databases for multiple outputs for Ophthalmological testing, reference databases for highly sensitive assays such as Troponin), there are new study design and statistical analysis challenges associated with reference databases and reference intervals. Given the importance of the reference database and reference interval concepts in the medical testing paradigm, this session aims to discuss study design and statistical analysis issues for medical tests with reference databases or reference intervals, from the perspectives of academia, industry, and government.
The growing area of data science has placed greater focus on visualization, predictive analytics and machine learning. This is in contrast to a drug development setting, where biostatisticians have tended to focus on techniques suited to the confirmatory paradigm, even when dealing with questions that are more exploratory in nature. Pre-planned hypothesis testing and static tables are also often used for exploratory analyses or in the earlier phases of development. Such an approach fails to do justice to the data collected in clinical research and in many cases does not address questions of real interest. In addition, interpretation is made complicated due to multiplicity and selection bias. Tukey, who emphasized the need for both exploratory and confirmatory paradigms, forcefully argued that in science important and relevant questions are generated from data exploration. He went on to refute the notion that exploratory analysis is just descriptive statistics, stating that it is an attitude, requiring flexibility, and a reliance on display but not a bundle of techniques. How can the thoughts of Tukey be combined with the latest developments in data science to improve exploratory analysis in drug development? In this session, we aim to examine this question through a series through a series of case-studies. Recent developments in dynamic visualization, machine learning and software tools will be discussed.
The role of modeling and simulation has forcefully and rapidly grown in the pharmaceutical and device industry. The practice of utilizing scientific innovation and adaptive design methodologies more often needs to be looked at through the lens of simulation.
This session will share the work of a cross-pharma working group composed of statisticians from industry and the FDA brought together within the Adaptive Design Scientific Working Group. This teamwork is focused on the best practices and recommendations around modeling and simulation conduct and reporting in various settings of the most frequently used adaptive designs landscape.
When adaptive designs are an integral part of a compound/device development program, the simulation report is to be regarded as a regulatory document. As stated in the 2010 FDA guidance on adaptive designs, detailed documentation on the simulation report and results should accompany the study protocol. The session will illustrate how the teamwork has expanded on recommendations for trial simulation reporting provided in Gaydos et al. (Drug Information Journal:43, 2009) and will provide the regulatory perspective as well as specific examples on key features associated with the simulation report. Commonly used adaptive designs are categorized into 4 main classes: dose escalation designs, dose ranging designs, designs that enable sample size re-estimation and early stopping, and multi-stage confirmatory designs.
Missing data and treatment adherence are common challenges in clinical trials. The final concept paper for ICH E9 Revision 1 (2014) - “Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials”, pointed out that “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making”. How to clearly define and choose an appropriate estimand for a specific study design? What primary and sensitivity analyses should be used to estimate the selected estimand? Do these answers vary from superiority trials to non-inferiority or bioequivalence/biosimilar trials? Much work needs to be done to resolve the remaining challenges and in understanding the implications and interpretation of different statistical methods to handle different choices of estimands in drug development.
With advancements in science and technology, research in drug development for orphan diseases, including pediatric trials, is of great interest. Evidence of this is demonstrated by the increasing number of fast track and breakthrough submissions that FDA has received during the past two years. Even though a variety of approaches have been proposed to cope with the challenges associated with the conduct and analysis of small-sized trials, it is not clear whether a consensus has been reached in terms of what level of evidence can be used to make regulatory decisions. In particular, when the safety of the study drug is a concern, how to factor in the benefits and risks during the drug evaluation and approval process can be very challenging. In these cases, rather than meeting the requirement of two adequate and well-controlled trials based on a single pre-specified primary endpoint for efficacy, it may be necessary to rely on different types of study endpoints and innovative study designs in assessing the “totality” of the evidence. The ultimate goal is to be flexible and at the same time maintain the standards for our evaluation of the drug’s safety and efficacy.
In this session we will focus on the research areas dealing with small-sized trials, where the major concern is the lack of sufficient study power to demonstrate efficacy. Methods for efficiently utilizing innovative designs and statistical analyses, and even borrowing external or historical trial data, will be discussed. For example, although “relaxing” the level of confidence in determining the non-inferiority margin seems to be an option for small trials, how much advantage this approach will gain compared to the use of superiority tests with “relaxed” alpha requirements is not clear.
Wearable devices provide the opportunity to measure how patients are functioning in their daily lives in the real world. This creates an opportunity to transform the functional endpoints used in clinical trials from artificial standardized measurements observed in clinical setting into something that is more meaningful to patients. The measurements vary device, but can include heart rate, number of steps taken, measurements of activity, and estimates of energy expenditure. These variables can be collected for each minute over an extended period of time, such as two weeks. There are challenges in the data collection including educating the patients on how to use the device and how to minimize missing data. Analysis of the data has many challenges including how to summarize the data collected and how to handle missing data. Some approaches reduce the data into summary measures, such as intensity categories (sedentary, light, moderate and vigorous activities). These approaches depend on specifying cut points for intensity such as <100 activity counts (sedentary activity), 100-300 activity counts (light activity), etc. These cut points are somewhat arbitrary and reducing the data in this way has the potential to lose information. Other approaches use mixed effects models, functional data analysis, and principal components. Some of these more complicated methods have the potential to handle missing data and can use all of the data in a more meaningful way. The session will focus on how these methods can best be used to define a clinically meaningful measure of a treatment effect.
Agreement studies are often conducted to evaluate if a new method is equivalent to an existing method so we can replace the existing method by the new, or use the two methods interchangeably. In the medical device regulation, a 510(k) regulatory pathway is that to be marketed devices should be demonstrated to be at least as safe and effective, that is, substantially equivalent, to a legally marketed device (predicate). The equivalence for safety and effectiveness are often evaluated in an agreement study where agreement between the two methods is assessed. The agreement assessment can be used in evaluating the acceptability of a new or generic process, methodology and/or formulation in areas of lab performance, instrument/assay validation for both quantitative and qualitative device outputs. This session will discuss statistical issues in agreement studies.
A well-designed meta-analysis can provide valuable information for safety and effectiveness assessment in the regulation of medical products. However, there are many statistical considerations in using meta-analysis for regulatory decision making. Meta-analysis may be subject to bias such as publication bias; heterogeneity in the study population, study design and study conduct, etc. can create difficulties in generalizing statistical inference and interpreting results. The quality assessment of selected publications in meta-analysis such as blinding, missing data, etc. is crucial in the evaluation. In this session, we will invite speakers from industry, academia and FDA to share their thoughts in using meta-analysis for regulatory decision making.
Drug development has rapidly been globalized. Multi-regional clinical trial (MRCT) for regulatory submission has widely been conducted in the ICH and non-ICH regions. In order to harmonize points to consider in planning/designing MRCTs and minimize conflicting opinions, an ICH working group was established in late 2014 to create an international guideline for MRCT (ICH E17). This guideline is intended to describe general principles for the planning and design of MRCTs with the aim of encouraging the use of MRCTs in global regulatory submission. The draft ICH E17 has been issued for public comments in 2Q2016.
Missing data have raised concerns in the statistical analyses in areas such as patient reported outcomes (PROs), cost effectiveness analyses (CEA) and efficacy analyses of new treatments in clinical trials. In the first situation, individuals with complete information tend to be systematically different from those with missing data within each provider and systematic biases may result from the proportions of non-response among the providers. Similar bias may occur in the third situation, where patients who drop out may respond to the treatment systematically differently from those who stay. Inappropriate methods to handle missing data may lead to misleading results and ultimately can affect the regulatory decisions and the decision of whether an intervention is of good value. In this session, the concerns of conventional missing data methods will be discussed and new and innovative methods will be introduced in these areas.
The ICH E14 Q&A was revised in December 2015 and now enables pharmaceutical companies to use concentration-QTc (C-QTc) modeling as the primary analysis for assessing QTc prolongation risk of new drugs. Because the C-QTc modeling approach is based on using all data from varying dose levels and time points, a reliable assessment of QTc prolongation can be based on smaller-than-usual TQT trials or based on single- and/or multiple- dose escalation (SAD/MAD) studies during early-phase clinical development in order to meet the regulatory requirements of the ICH E14 guideline.
In the revised document, the E14 Implementation Working Group intentionally did not provide the technical details on how to perform and report C-QTc modeling to support regulatory submissions. The rationale for this omission is that specific analysis methodology is likely to evolve over time as pharmaceutical and regulatory scientists implement this approach across drugs with diverse pharmacokinetic (PK) and pharmacodynamic (PD) attributes.
In 2016, the E14 Implementation Working Group tasked an expert group of statisticians and pharmacometricians from industry and regulatory agencies to provide the technical details on how to design, perform, report, and review C-QTc analysis to support regulatory decision. This group developed a White Paper to propose current best practices in designing studies to use C- QTc modeling as the primary analysis, conducting a C-QTc analysis, reporting the results of the analysis to support regulatory submissions, and reviewing the analysis for regulatory decision. The recommendations within the White Paper provide opportunities for increasing efficiencies in this safety evaluation.
Discussant: Christine Garnett, FDA/CDER
Patient-reported outcome (PRO) is the term used to denote health data provided by the patient through a system of reporting, without interpretation of the patient’s response by a clinician or anyone else. PRO has attracted a lot of attention from many researchers in that the information directly from patients can provide valuable insight that others such as observers can’t. PRO is an instrument to capture data from patients that is used to measure treatment benefit or risk in medical product clinical trials. In this session, speakers from academia, industry and regulatory agencies will present their current research in important methodological issues in analyzing PRO, provide case studies and examples of PRO instrument development and validation in clinical studies.
Diagnostic classifiers involve developing the model as a first step. Then the model's performance is checked during internal validation, and modified if necessary. At that point the model needs to be fixed before an independent dataset is obtained to validate the classifier to assure it works in future patients. In case the data collection for external validation starts before the classifier is finalized, people developing the classifier should be blinded to this data. There is a lot of confusion about the three steps, particularly between internal and external validation. With real world data becoming more acceptable, this is becoming even more of an issue. How independent the datasets are has big impact on the performance of the classifiers. And whether people are blinded to the validation data while developing the model is even harder to determine when the data is not collected prospectively. In this session, We will discuss ways to design the studies so we can get unbiased performance estimates of the classifiers.
Speakers: Lakshmi Vishnuvajjala, CDRH/FDA; Ravi Varadhan, Johns Hopkins University; Susan H. Gawel, Northwestern University Institute of Public Health and Medicine, Feinberg School of Medicine; Xiaoqing (Quinnie) Yang, Abbott Diagnostics R&D Statistics.
Often patients randomized to the control arm in clinical trials for new oncology/hematology products are permitted to switch treatments after disease progression, often to the new therapy. When this occurs, the control arm of the trial is contaminated by the new treatment and a standard intention to treat (ITT) analysis does not fully address the question that may be of greatest interest – that is, what is the safety and effectiveness of the new treatment compared to the control treatment. There are several methods available that may be used to evaluate the overall survival benefit adjusted for treatment switching, for example, rank-preserving structural failure time models (RPSFTM) and inverse-probability-of-censoring weighting (IPCW). These are commonly proposed by sponsors as sensitivity analyses to evaluate the overall survival benefit adjusted for switching. However, how well these models estimate the real survival benefit remains unclear. It is also unclear how these adjusted survival benefits can be used in the regulatory decision making process, as opposed to the reimbursement decision making process. In some European countries, the overall survival benefit of the new therapy is directly linked to medical reimbursements or payments to physicians and patients, and therefore accurate point-estimates of the overall survival benefit (and uncertainty around this) are critical. In the regulatory setting it is unclear how critical point-estimates of overall survival benefits are for decision-making. Speakers: Erik Bloomquist, US FDA; Uwe Siebert from UMIT, Austria; Nicholas R Latimer University of Sheffield, UK
We see in many aspects of our society the power of images to convey an idea. How can we utilize this power within biostatistics for communicating more effectively to others and for deeper insights ourselves? Faced with a large number of endpoints to summarize, and with the increasing reliance on sensitivity analysis to assess the robustness of study design features and results, statisticians struggle to uncover and adequately portray the story hidden within the data. A well-designed graph can be the quickest way to convey what the data have to say, but it takes time to a) frame the question based on available data and b) design and refine the graph to meet this purpose. In this overview, we illustrate how to construct well-designed graphs for study design and analysis, with specific applications to drug safety, subgroups, and post-market surveillance.
Comparative-effectiveness (CE) research aims to produce evidence regarding the effectiveness and safety of medical products outside of randomized and well controlled clinical trials. In recent years, Real-world evidence (RWE) research is an increasingly important component of biopharmaceutical (pharmaceutical, biologic and medical device) product development and commercialization. There is a growing industry need for broader data and information on real-world effectiveness and safety —both of which will influence the eventual reimbursement and utilization of new products. The ultimate decision is driven by regulators, public and private payers, and prescribers, all of whom seek to better understand the impact of a new product to patients through their treatment journey. Many countries already make reimbursement decisions based on RWE. In United States, the FDA and the Center for Medicare and Medicaid Services have agreed to work together more closely to allow the use of RWE in drug approval and reimbursement. Unlike randomized control trials (RCT), which remain the gold standard for drug approval, RWE data comes from outcomes of heterogeneous patients as experienced as treated in real world clinical practice. The relevant data sources include phase IV trials, pragmatic trials, registries, post-authorization safety/efficacy studies, observational studies (prospective and retrospective), pharmacoeconomics studies, and expanded access/compassionate use programs of a drug etc. The absence of randomization and the multifarious nature of the data creates methodological challenges in generating quality evidence. This includes choice of right methodology for proper design and analysis of RWE studies. The next hurdle is to ensure proper synthesis of results from RWE with other types of evidence to make better healthcare decisions and support product throughout its lifecycle.
It is well recognized that the treatment effects may not be homogeneous across the study population. Patients want to know whether a medicine will work for him or her as an individual with his or her own specific characteristics. Therefore, subgroup analysis is an important step in the assessment of evidence from clinical trials. In the confirmatory phase, this can be a critical strategy where conclusions for an overall study population might not hold. On other hand, it is an integral part of the early development to identify the appropriate patient population to increase the probability of success of a clinical program. At this stage these analyses are exploratory in nature. One notable distinction between confirmatory and exploratory subgroup analyses relate to the efforts devoted for planning. The goal of a confirmatory subgroup analysis is to provide sufficient evidence for decision. However, the exploratory analyses are rather hypothesis generating. Whether confirmatory or exploratory in nature, the investigation of subgroups poses statistical, interpretation and regulatory challenges. Confirmatory subgroup analyses are known to be prone to statistical and methodological issues such as inflation of type I error due to multiple testing, low power, inappropriate statistical analyses or lack of pre-specification. Although powering within each possible subgroup is not mandatory but powering within specialized subgroups of interest is imperative for proper interpretation. However, the primary challenge with exploratory subgroup analyses is making decisions using only limited information which increase the chance of false detection. Use of naive estimates of the treatment effect to find a subgroup with high treatment effect induces random high bias and potentially misleading. Therefore, analyses in both settings need distinct statistical methodologies to address the problems appropriately. To recognize these challenges and proper interpretations of subgroup in medical product development, regulators have devoted their efforts toward guidance development on subgroup analysis in recent years (CHMP 2010 and FDA 2012 in context of enrichment strategy).
Discussant: Kathleen Fritsch, CDER, FDA
Many of the statistical issues encountered in studies intended for animal drug approvals are similar to those in regulatory human clinical trials. However, there are also statistical issues and challenges unique to regulatory animal drug studies, often related to various experimental designs to support drug indications for specific animal species. In this session, we will present animal drug studies reviewed by the Center for Veterinary Medicine (CVM) and discuss statistical issues and challenges associated with these studies.