CADTH Health Technology Review

Platform, Basket, and Umbrella Trial Designs: Stakeholder Perspectives of Novel Therapeutics

Discussion Paper

Authors: Nicole Mittmann, Kelvin K.W. Chan, Matthew Cheung, Valerie McDonald, Catherine Moltzan, Brent Fraser, Daria O’Reilly, Heather McDonald, Jason Lee, Jean-Eric Tarride

Author Information

Authors

Nicole Mittmann,^1,2 Kelvin K.W. Chan,² Matthew Cheung,² Valerie McDonald,³ Catherine Moltzan,⁴ Brent Fraser,¹ Daria O’Reilly,⁵ Heather McDonald,⁶ Jason Lee,⁷ Jean-Eric Tarride.^8,9

Affiliations

1. Canadian Agency for Drugs and Technologies in Health (CADTH)

2. Department of Medicine, University of Toronto

3. pan-Canadian Oncology Drug Review Expert Review Committee (pERC) patient member, CADTH

4. Department of Internal Medicine, University of Manitoba

5. Telus Health Inc.

6. Bayer Inc.

7. At time of webinar: Bayer Inc.

8. Department of Health Research Methods, Evidence, and Impact, McMaster University

9. Centre for Health Economic and Policy Analysis, McMaster University

Acknowledgements

We would like to thank Case Market Access Consulting Inc. for transcribing the webinar and creating a transcript and outline for the manuscript.

Key Messages

• Advancements in genomic and precision medicine have changed the way oncology clinical trials are designed. Compared to the traditional single tumour-based trial, master protocols, which are often classified as basket, umbrella, or platform trials, conduct studies on multiple subgroups under a single overarching protocol.

• This paper presents the perspectives of patients, clinicians, public payers, private payers, and industry, which were discussed during the third webinar.

• The discussion section of this paper summarizes the major points raised by the presenters, as well as questions from the audience and answers from the presenters.

Introduction

Advancements in genomic and precision medicine have changed the way oncology clinical trials are designed. Compared to the traditional single tumour-based trial, master protocols, which are often classified as basket, umbrella or platform trials, conduct studies on multiple subgroups under a single overarching protocol. Regulatory bodies, health technology assessment (HTA) agencies, and decision-makers are faced with numerous challenges when evaluating the evidence generated from these novel study designs.

To provide a forum to present broad perspectives and considerations when assessing the evidence generated by these new study designs from an HTA perspective, CADTH organized a series of 3 webinars in late 2020. The first webinar discussed the past, present, and future of economic evaluations of precision medicine.¹ The second summarized the discussion held during our second webinar, which examined issues around HTA of evidence generated using these novel trial designs.² This paper presents the perspectives of patients, clinicians, public payers, private payers, and industry on issues related to tumour agnostic therapeutics. The discussion section of this paper summarizes the major points raised by audience questions and presenter responses.

Patient Perspective

Presenter: Valerie McDonald

There is increasing consensus that engaging patients in all stages of the research and development of medicines has potential benefits for all stakeholders. Engaging patients can promote improvements in informed consent and understanding, ensure optimal trial enrolment and retention, and ensure measurement of trial and patient-reported outcomes that are relevant and meaningful for those who will potentially use the new treatments. In health technology assessment (HTA) evaluation, there is less agreement on how patients contribute to decision-making. However, it is generally accepted that by engaging patients there is increased transparency and input, which can add meaning to clinical and economic data, potentially resulting in better-quality decisions. Patients, families, and caregivers offer insights into unmet needs, impact of illness and current treatment, patient expectations and values, and perhaps even experience with the new technology under evaluation.

There are a number of different opportunities where patients engage with CADTH Pharmaceutical Reviews as part of the HTA evaluation process for ongoing education as well as participation in the patient input process through the pan-Canadian Oncology Drug Review Expert Review Committee (pERC) Deliberative Framework (as illustrated in Figure 1).³

Figure 1: pERC Deliberative Framework³

pCODR = pan-Canadian Oncology Drug Review; pERC = pCODR Expert Review Committee.

Patient evidence is incorporated into the clinical and economic guidance panel reports shared with all pERC members. Despite these opportunities, a recent study has highlighted that patient groups are not always sure how their input is incorporated into final decisions and groups have communicated interest in engaging much earlier in the drug development process.⁴ A study of 21 patient groups registered with pCODR found that groups value the opportunity to contribute and consider it an important and meaningful activity in line with the mandate of their respective organizations. Of particular value is the opportunity to provide information and perspective that may not be captured or available in a clinical trial.⁴ However, competing priorities for groups and the resource-intensive nature of preparing input for an HTA review are challenges, especially for small and under-resourced groups. This is compounded by the short timeline from notification to deadline for submission, retrieval, and comprehension of available scientific information, preparation of surveys to solicit feedback from members, and outreach to membership. Reaching diverse patient populations, those who have had experience with the treatment under evaluation, and those dealing with illness also present challenges.⁴

These challenges are apparent in the example of the larotrectinib HTA process, where 7 patient groups (Canadian Cancer Survivor Network, Colorectal Cancer Canada, Lung Cancer Canada, Neuroblastoma Canada, Ontario Parents Advocating for Children with Cancer, Sarcoma Cancer Foundation Canada, and Thyroid Cancer Canada) collaborated for a collective submission. The groups worked together to design and distribute a survey, conducted detailed telephone interviews, and performed an environmental scan of patient forums. The resulting submission offered information and context about the unmet needs of patients, the impact of the illnesses, current treatments, and what patients valued in this new treatment where patients with metastatic or locally advanced tumours had no other treatment options. The tumour sites included in the trial were quite varied and the patient submission tried to reflect that diversity by including patients with cancers of various histologies. The experience with larotrectinib described a new treatment paradigm and detailed patient interviews highlighted the rapid onset of effect, clinically meaningful outcomes achieved with an oral treatment, which provided pERC with an understanding of the meaning of rapid improvement to patients. Finally, quality of life, experience, and management of treatment side effects data in the context of real-world experience was provided.

In the context of evaluation of larotrectinib through a basket trial design, patient groups also acknowledged the difficulties with data interpretation across all tumour types and expressed a keen interest in the gathering of real-world data (RWD) to address evidence uncertainties. The groups also supported and recommended some form of risk share strategy with industry.

In summary, faced with a complex clinical trial and an incredibly diverse group of patients, 7 patient groups were able to collaborate and create a detailed submission that helped pERC better understand the impact and value of a new treatment. Patient groups view the ability to make submissions as a meaningful activity, but have significant resource challenges to find appropriate patients, explain complex treatments, and prepare submissions. All stakeholders are challenged to consider what they can do to provide both financial and non-financial resources to patient groups to support their meaningful participation in drug development and the HTA process.

Clinical Perspective

Presenter: Dr. Catherine Moltzan

There is an understanding that it is challenging for the current HTA evaluation framework to address novel clinical trial designs for new products and that changes to the framework may need to be considered. These novel designs are further discussed in the following, including the intersection of evidence evaluation at the HTA agency level.

In basket trials, the target mutation or molecular target should be a driver clinical mutation, as opposed to a bystander or passenger clinical mutation. The mutation needs to be intricately involved in the pathogenesis and development of the cancer, such that if the target is inhibited, at least in theory, it would then lead to inhibition of cancer growth and control of the tumour. The basket trial is most appropriate for patients who have relapsed or have refractory disease with a driver clinical mutation, and there is a target against that mutation. However, when a driver clinical mutation is believed to be driving the cancer regardless of location or histology, it is not possible to have a clinical design with a control group. And, while scientific evidence is important to assert the role of the mutation, this mutation may be present in different histological cancers but may have a different role in the development of that cancer. An example of this is the BRAF mutation B600E, which has a pathogenic role in malignant melanoma and hairy cell leukemia, yet when targets against the same BRAF mutation were used in other types of cancer, the same effect was not observed. So, it is important that the mutation is truly a driver mutation and that this is true of all cancers in the basket.

The main advantage to the basket trial design is its efficiency, as it allows for an entire group with the same target to be evaluated in a tumour histology-agnostic fashion. So even though patients have different cancer histologies, they have the same target and can be enrolled in the same trial. The design is also important for patients with rare cancers, where the ability to conduct a standalone clinical trial compared to standard therapy is limited or perhaps nonexistent. The basket can be evaluated as a single group, allowing for more robust trial numbers and a faster pathway to regulatory approval and patient access.

The key to umbrella clinical trial designs is to start with a precise definition of the underlying disease, including the histology or pathology, stage, and presence or absence of other markers, which add complexity as they can all be challenging in clinical practice. Similar to basket trials, the molecular target needs to be a driver mutation in disease pathogenesis or at least be plausible to be a driver mutation. Ideally, there should be a comparison to standard of care, though this proves challenging when the precise definition of the underlying disease changes and the standard of care evolves over time.

The efficiency of the umbrella design is a core advantage, allowing for different molecular targets to be studied in parallel using the same clinical trial design. For patients who may have less common molecular targets, umbrella studies can be designed to enrich patients with these mutations. As part of the same trial, the umbrella design provides a common statistical analysis, which allows for direct comparisons between different molecular targets. Umbrella clinical trials face similar disadvantages to the basket design, with most being phase II trials without true randomization to a control group, while others may use historical controls, both of which can create issues with statistical analysis.

From a clinical perspective, platform clinical trials appear the most promising with the capability to test several interventions against a common control group, allowing for interventions to enter and exit the trial under a Bayesian decision rule framework based on the demonstration of efficacy or futility. The keys to a platform trial include a proper definition of the disease entity under study, a definition of standard of care, a well-defined statistical analysis plan with starting and stopping rules, rules for adding or dropping trial arms, and accounting for modifications of standard of care over time.

The efficiency of design is a central advantage of platform trials, as it allows for biomarker strata to be run in parallel to study different investigational targets and make changes to interventions as time progresses. Platform trials are perhaps the closest to model to how care is delivered at the individual patient level. Comparators are built in, as is standard of care, which has the potential to permit traditional HTA evaluation, yet there is no true randomization. In theory, platform trials can be multiple umbrellas or multiple baskets, as observed in the literature, especially if there is uncertainty that the mutation is a driver for all clinical entities. Challenges with platform trials include complex statistical designs with results that are challenging to interpret, and that changes to standard of care over time can limit comparisons.

In summary, basket, umbrella, and platform clinical trial designs all allow for multiple targets and/or multiple disease entities to be studied at the same time, which can lead to faster and expanded access to new treatments. While there are advantages and disadvantages to be taken into account, modification in HTA and evaluation may be required.

Public Payer Perspective

Presenter: Brent Fraser

Health Canada evaluates safety and efficacy related to surrogate outcomes such as Objective Response Rate and other clinical outcome measures. That said, Health Canada does not typically consider where a treatment could fit into a treatment algorithm. In other words, where would the treatment be inserted within the context of treatments currently available and how are treatment choices sequenced? This is where HTA evaluation considers the available evidence in relation to current standard of care. With tumour agnostic treatments, there is a struggle as standard of care varies by tumour type or histology.

From an HTA perspective, there are 15 jurisdictions in Canada that need to align and be comfortable with real-world evidence collection and risk share agreements. A key consideration for the HTA agency is that when recommendations are made, they can be implemented equally effectively by all jurisdictions. However, while there is a lot of interest within CADTH and some jurisdictions for risk share agreements, full health system readiness is required. This is different at the regulatory level at Health Canada where a decision can be made requiring the sponsor to return with more evidence in the future to Health Canada, a single federal agency.

Private Payer Perspective

Presenter: Dr. Daria O’Reilly

The private insurer is often referred to as the private payer; however, it is in fact the “plan sponsor” or employer who is responsible through premiums that are paid. Insurers design products or plans, including numerous different formularies, but insurers are the intermediaries. Employers purchase the products or plans developed by insurance companies because their goal is to keep employees healthy and productive with comprehensive drug coverage while balancing the sustainability of drug plans. Insurers can be viewed as stewards of plan sponsors’ precious health care dollars, by ensuring every dollar spent produces maximum value in terms of health outcomes. HTA and evaluation is being used more frequently in the private payer space as a way of developing insurance products and managing costs, but not all insurers conduct HTA evaluations of any products. The insurers use the cost-effectiveness results to build the products (or plans) that are purchased by employers.

Not unlike the public payer perspective, small sample sizes, lack of randomization, study outcomes that are surrogates for progression-free survival (PFS) and overall survival, and a lack of health-related quality of life outcomes measured present challenges with the interpretation of master protocol trials in the private payer space. The challenges present uncertainty when interpreting the clinical evidence and in turn the calculation of an incremental cost-effectiveness ratio with an acceptable degree of confidence. This leads to uncertainty in determining the cost-effectiveness analysis. While the size of plan sponsors (employers) varies significantly, so does risk tolerance for uncertainty in evidence, as well as budget implications. Sponsors are faced with choosing access for tumour types where a targeted product has the largest impact, only tumour types represented in a trial, or only tumour types where an acceptable proportion of patients were evaluated. For products with evidence uncertainty and unknown financial risk and budget uncertainty, plan sustainability is impacted. And sponsors (employers) are required to balance offering plans that are comprehensive for employees with tolerance for budget risk.

Possible solutions for private payers include postponing decision-making until more mature data are available and not offering promising treatments for employees. Solutions such as coverage with evidence development arrangements to address clinical and financial uncertainty could be feasible if infrastructure were in place, although data collection and evaluation takes time. Ultimately, the goals for private insurers and plan sponsors are to provide comprehensive health plans for employees while ensuring health benefits remain affordable for members.

Industry Perspective

Presenters: Dr. Heather McDonald and Jason Lee

As we move into the era of precision medicine and the promise of personalized medicine, novel trial designs will become common in evaluating targeted therapies. The uncertainty associated with novel trial designs may be addressed through real-world evidence generation in the context of a precision medicine evidence ecosystem, which will require a paradigm shift. For example, it may be unfeasible to incorporate a comparator arm in the design of basket trials. A possible adjustment to address this uncertainty is the concept of an intra-patient comparison of PFS, which could be an useful alternative to measuring treatment activity. Using the ratio of PFS from the drug in question compared to that of a prior line of treatment may help provide contextual evidence on whether the PFS for the drug is beneficial. However, it is unclear whether the HTA system can accept that type of outcome for contextual evaluation.

Several issues highlight the challenge with HTA evaluation and its associated uncertainties in precision medicine. First, there is a need for greater alignment in terms of the evaluation lens between the regulatory body (Health Canada), and the HTA body (pCODR), to address these uncertainties. Second, to address the uncertainty and satisfy the requirements of the Notice of Compliance with conditions (NOC/c) granted by Health Canada, manufacturers may be conducting clinical trials and observational studies. It is, however, unclear what new evidentiary requirements would be acceptable to support a resubmission and a change to an HTA recommendation. Should the HTA evidence requirements be the same as the regulatory requirements, the generation of additional data could take 4 to 5 years, a delay in access for patients in need, including access to treatment for patients in Canada who participate in the observational study. To avoid the delays in access to treatment for patients who could benefit, the concept of a precision medicine evidence ecosystem where stakeholders plan and collaborate ahead of the HTA review to prepare to address evidentiary uncertainties has been proposed.

Industry stakeholders are willing partners in this type of ecosystem to generate the required evidence, but the willingness and partnership of other stakeholders is required to transition from conceptual discussion to pragmatic solutions. This includes the opportunity to engage in pilot projects with upcoming precision medicine treatments to demonstrate value in the real world, which would be feasible given the small patient populations and the ability to track every patient on treatment.

The challenge of HTA and evaluation in precision medicine is not unique to Canada, and other countries (such as Germany, Spain, Italy, and the UK) are implementing new approaches to review and reimbursement that support the management of uncertainty. As innovative treatments (e.g., cell and gene therapy) continue to evolve, the associated clinical evidence packages will be different and face similar challenges for evaluation according to the traditional HTA evaluation framework. Manufacturers are willing to partner to manage evidence uncertainty and build solutions to close the evidence gap, recognizing that while no solution is perfect, there is a willingness to find a path forward. In Canada, if there is commitment to ensuring those living in Canada have access to future treatments, evolution of the Canadian precision medicine ecosystem is required.

In summary, the evidence for new innovations and its associated uncertainty will continue to face challenges with using traditional HTA evaluation frameworks. A potential solution in acknowledgement of this uncertainty is a focus on partnership between manufacturers, HTA, and payer stakeholders to commit to ongoing evidence collection to close the gap in uncertainty while patients have timely access to promising therapies. At the moment, there is no clear space within the Canadian system that permits this kind of partnership. Starting now will support learning and evolution for the introduction of new therapies in the future.

Discussion

From a clinical point of view, these novel trial designs have arrived and the challenge with tumour agnostic treatments in particular is that standard of care varies by tumour type or histology, making it difficult to compare available evidence with standard of care. This creates a perceived dichotomy in the approach and handling of precision medicines between regulatory approval by Health Canada and HTA by CADTH. Health Canada evaluates the safety and efficacy of a treatment, while HTA evaluation considers the available evidence in relation to current standard of care and/or comparators to help clarify appropriate treatment value. While CADTH has an Early Scientific Advice Program that allows for collaboration and provides sponsors an opportunity to receive input on clinical study design, it may not be feasible to alter global clinical trial designs to meet the needs of Canadian decision-makers. So, with new precision medicine treatments, the challenges with mechanisms to resolve uncertainty as part of decision-making will remain. Nonetheless, early patient engagement could potentially help improve the design of precision medicine trials. While quality of life measures are necessary, patients often comment on the burden of completing endless surveys to address quality of life questions. Moving forward, it may be helpful to have more education tools available for patients to assist with understanding the methods of quality of life tools. An appreciation of which measures are important to patients is also required.

It was also noted that the Canadian system needs to evolve to find a way to manage HTA evaluation in the context of uncertainty with evidence derived from basket or other novel trial designs. In the context of the introduction of COVID-19 vaccines, partnering to find solutions and a way to manage risk in the presence of data uncertainty has been accomplished. By necessity and collective will, the Canadian system has found a way forward and lessons learned from that experience can be applied to new product situations. However, in the context of managing the COVID-19 crisis, there is reduced fiscal capacity presented within the public health system and there may be challenges with engaging in risk sharing agreements where there is uncertainty at the provincial jurisdiction level.

While private payers follow a similar approach to CADTH when evaluating new treatments, productivity data are always important given that the target audience is the working patient population, for whom it is essential to understand the health impacts of any new treatment. Finally, budget impact analysis, pricing, and market share evaluations are considered to measure the financial impact of new therapies on overall costs. With regards to precision medicine, private payers, like public payers, are challenged with interpretation of sometimes uncertain clinical evidence as well as how to build a formulary when the financial risk may be uncertain and the impact on overall costs are unknown.

Overall Conclusions

This series of CADTH webinars on novel clinical trial designs had brought multiple stakeholders together to discuss their different perspectives on the challenges and opportunities in evaluating evidence generated from basket, umbrella, or platform trial designs. While these trial designs have their own advantages (e.g., efficiency, better understanding of disease) and limitations (e.g., non-comparative data, small sample size), a few themes emerged from the presentations and discussions. First, multi-stakeholder (e.g., Health Canada, CADTH, Institut national d'excellence en santé et en services sociaux [INESSS], patients, clinicians, industry, payers) dialogue and collaboration is required to create an ecosystem in which health technologies will be evaluated throughout their life cycle. Second, coverage with evidence development and RWD collection could be used to fill data gaps at the time of submission and to inform a reassessment post launch, thus allowing patients to have access to novel therapies characterized by a high level of uncertainty at product launch. However, new frameworks, processes, and infrastructure will be required to accommodate the complexity of the Canadian health care system (e.g., multiple public and private payers, data infrastructure). Finally, patient engagement in designing trials, evaluating evidence, or collecting RWD is critical and there is a need to continue to develop and apply new methods to derive patient values and preferences.

References

1.Chan KKW, Cheung MC, Regier DA, et al. The past, present, and future of economic evaluations of precision medicine at the Committee for Economic Analyses of the Canadian Cancer Trials Group. Curr Oncol. 2021;28(5):3649-3658. PubMed

2.Tarride J-E, Cheung M, Hanna TP, et al. Platform, basket, and umbrella trail designs: issues around health technology assessment of novel therapeutics [in press]. Can J Health Technol. 2022;2.

3.pCODR Expert Review Committee: deliberative framework. Ottawa (ON): CADTH; 2016: https://www.cadth.ca/sites/default/files/pcodr/The%20pCODR%20Expert%20Review%20Committee%20%28pERC%29/pcodr_perc_deliberative_frame.pdf. Accessed 2022 Apr 28.

4.Mercer RE, Chambers A, Mai H, McDonald V, McMahon C, Chan KKW. Are we making a difference? A qualitative study of patient engagement at the pan-Canadian Oncology Drug Review: perspectives of patient groups. Value Health. 2020;23(9):1157-1162. PubMed