Drugs, Health Technologies, Health Systems

Health Technology Review Recommendation

RapidAI for Stroke Detection

Summary

What Is the Indication Under Review?

The indication under review is the detection, using artificial intelligence (AI) software, of large vessel occlusion (LVO) (i.e., ischemic stroke) and intracranial hemorrhage (ICH) (i.e., hemorrhagic stroke) for people of any age with a suspected acute stroke.

What Is RapidAI?

RapidAI is an AI-enabled software platform that facilitates the viewing, processing, and analysis of CT imaging to aid clinicians in assessing patients, including those with suspected stroke. It builds on the original RAPID software, initially developed to automate and expedite the postprocessing of CT perfusion imaging. Currently, the platform also incorporates modules that perform AI-driven detection of LVO and ICH, which were the functionalities evaluated in this review. RapidAI is intended to complement, rather than replace, clinician interpretation of CT imaging and is to be used as a supportive tool rather than a standalone diagnostic intervention.

How Did Canada’s Drug Agency Evaluate This Technology?

To examine the value of implementing RapidAI for detecting LVO and ICH, Canada’s Drug Agency (CDA-AMC) conducted an evidence review on RapidAI that identified, synthesized, and critically appraised literature evaluating RapidAI’s effectiveness, accuracy, and cost-effectiveness. CDA-AMC highlighted and reflected on the ethical and equity implications of using RapidAI for stroke detection, engaged a patient contributor, sought feedback from knowledge users, and consulted an expert panel. While RapidAI has numerous features and functionalities, what is relevant for this report are the modules that perform AI-driven detection of LVO and ICH to inform stroke diagnosis. No additional features or functionalities, such as the postprocessing of CT perfusion imaging, were assessed.

What Else Did CDA-AMC Do?

CDA-AMC applied Digital Technology Assessment Criteria (DTAC), a checklist currently used in the UK, to the health care context in Canada to determine whether we have equivalent or similar measures, strategies, and policies to implement digital health technologies safely. CDA-AMC also conducted a literature review to identify implementation guidance specific to AI-enabled medical device use and relevant to Canada to supplement DTAC. For this work, CDA-AMC integrated ethics and equity considerations, leveraged patient engagement activities conducted in the concurrent RapidAI review, sought feedback from knowledge users, and consulted an expert panel.

What Is the Health Technology Expert Review Panel Recommendation for RapidAI?

In locations where RapidAI has already been implemented for use in detecting suspected LVO and ICH, the Health Technology Expert Review Panel (HTERP) recommends RapidAI is used only as indicated, alongside clinician interpretation of CT imaging, to reduce the risk of incorrect results; and the generation of evidence to evaluate its value in health care systems, including its use in less-resourced centres with limited access to stroke care specialists.

In locations considering the implementation of RapidAI for use in detecting suspected LVO and ICH, given the uncertainty and gaps in the evidence regarding clinical, economic, and equity value of RapidAI, HTERP cannot provide recommendations for or against its implementation.

What Is the Indication Under Review?

The indication under review is the detection, using AI software, of LVO (i.e., a finding that can support the decision to perform urgent mechanical thrombectomy in acute ischemic stroke) and ICH (i.e., hemorrhagic stroke) for people of any age with a suspected acute stroke. Stroke is 1 of the leading causes of death and a major cause of disability in Canada. For patients with suspected stroke, prompt evaluation using CT imaging and other tests can help to determine the type of stroke, assess the severity of damage, and guide urgent treatment decisions.

What Is RapidAI?

RapidAI (iSchemaView, Inc., Menlo Park, California) is an AI-enabled software platform that facilitates the viewing, processing, and analysis of CT imaging to aid clinicians in assessing patients, including those with suspected stroke. It builds on the original RAPID software, which was initially developed to automate and expedite the postprocessing of CT perfusion imaging and is widely implemented in some jurisdictions, such as Ontario.¹^,² More recently, the software platform has expanded its suite of products to include several static AI-derived algorithms for evaluating the brain’s physiological status, such as Rapid ICH, Rapid ASPECTS, Rapid CTA, Rapid LVO, and Rapid HVS.³^,⁴ These algorithms are considered static because they remain fixed and unchanged after development through machine learning processes, performing their tasks based on pre-established rules, and training data. While RapidAI has numerous features and functionalities, what is relevant for this report are the modules that perform AI-driven detection of LVO and ICH to inform a stroke diagnosis. No additional features or functionalities, such as postprocessing of CT perfusion imaging, were assessed. RapidAI is intended to complement, rather than replace, clinician interpretation of CT imaging and is to be used as a supportive tool rather than a standalone diagnostic intervention.⁵ As of March 2024 (i.e., when CDA-AMC checked its regulatory status), RapidAI (version 4.9.2.1) is licensed for sale in Canada as a class III medical device.

How Did CDA-AMC Evaluate RapidAI?

To examine the value of RapidAI in stroke detection, CDA-AMC:

identified, synthesized, and critically appraised literature evaluating the effectiveness, accuracy, and cost-effectiveness of RapidAI for the detection of LVO and ICH
used methods that were guided by the Scottish Health Technologies Group’s health technology assessment framework⁶
highlighted and reflected on the ethical and equity implications of using RapidAI for stroke detection found in the clinical literature, integrating these considerations throughout the review
engaged 1 patient contributor who had experienced a hemorrhagic stroke
incorporated feedback from 3 peer reviewers (i.e., 1 clinical expert with expertise in stroke assessment, 1 clinical expert with expertise in AI radiology, and 1 ethics expert with expertise in AI), the manufacturer, and other interested parties
consulted an expert panel to deliberate on unmet clinical need, clinical value, economic considerations, impacts to health systems, and distinct social and ethical considerations regarding RapidAI.

What Else Did CDA-AMC Do?

CDA-AMC conducted an additional review to assist health systems in Canada in preparing for the uptake of AI-enabled medical devices, as these technologies pose new challenges. CDA-AMC assessed whether the safeguards and assessment criteria captured by DTAC⁷ and other AI-related resources are in place to inform decision-making around the digital infrastructure elements of implementation. To conduct the AI implementation review, CDA-AMC:

applied DTAC (a checklist used in the UK as an add-on component to health technology assessments [HTAs])⁷ to the health care context in Canada by determining whether we have equivalent or similar measures, strategies, and policies to implement digital health technologies safely
conducted a literature review to identify implementation guidance specific to AI-enabled medical device use and relevant to Canada to supplement DTAC
highlighted and reflected on the ethical and equity considerations
leveraged patient engagement activities conducted in the concurrent RapidAI review
incorporated feedback from 3 peer reviewers (i.e., same reviewers as RapidAI review) and other interested parties
together with the concurrent RapidAI review, consulted an expert panel to deliberate on unmet clinical need, clinical value, economic considerations, impacts to health systems, and distinct social and ethical considerations.

Health Technology Expert Review Panel

HTERP is an advisory body to CDA-AMC that develops guidance and/or recommendations on nondrug health technologies to inform a range of decision-makers within health care systems in Canada.

HTERP comprises 7 core members serving all topics under consideration during their term of office: chair, ethicist, health economist, patient member, 2 health care practitioners, and an HTA specialist. In addition to the core members, HTERP will appoint up to 5 expert members to provide their expertise on a specific topic. For this review, HTERP appointed 2 members with clinical expertise in stroke neurology and 1 member with expertise in neuroradiology and AI.

To make its recommendation, HTERP considered the following information:

CDA-AMC’s review of:
- RapidAI, including:
  - 2 cohort studies and 11 diagnostic accuracy studies that assessed the effectiveness and accuracy of RapidAI for detecting stroke
  - ethics and equity considerations relevant to RapidAI, which were identified through published literature and patient, clinician, and other expert input
- digital infrastructure elements of implementation considerations for digital health technologies, including ethics and equity considerations and additional considerations for using AI-enabled medical devices in Canada.

Using the available evidence, HTERP deliberated on and answered the question, “Should RapidAI be implemented to detect stroke in Canada, and how?”

Recommendation

In locations where RapidAI has already been implemented for use in detecting suspected LVO and ICH, HTERP recommends:

RapidAI is used only as indicated, alongside clinician interpretation of CT imaging, to reduce the risk of incorrect results
the generation of evidence to evaluate its value in health care systems, including its use in less-resourced centres with limited access to stroke care specialists.

Rationale for the Recommendation

HTERP recognized an unmet clinical need: stroke is a clinical condition associated with significant morbidity and mortality. There are effective treatments available for individuals experiencing acute stroke. These treatments require rapid clinical assessment and imaging interpretation to select the most appropriate therapy. Still, timely access to these treatments varies within Canada (across jurisdictions, rural versus urban settings, hospitals with comprehensive stroke centres versus less-resourced centres).

Two cohort studies and 11 diagnostic accuracy studies that assessed the effectiveness and accuracy of RapidAI, a software platform that includes specific modules such as Rapid LVO for detecting LVO and Rapid ICH for detecting ICH, resulted in the following:

Low certainty evidence suggests that evaluation of CT angiography images by Rapid LVO combined with clinician interpretation, compared to clinician interpretation alone, may result in clinically important reductions in radiology report turnaround time (i.e., the time it takes the radiologist to interpret the CT angiography images and provide a report or readback verification to the referring clinician) in patients with suspected stroke.
The evidence is very uncertain about the effects of RapidAI on other time to intervention metrics, measures of physical and cognitive function, and response to therapy (e.g., reperfusion rates). CDA-AMC did not identify any evidence on the effects of RapidAI on many important clinical outcomes, including patient harms, mortality, health-related quality of life, length of hospital stay, and health care resource implications.
Low certainty evidence (from the diagnostic accuracy studies) suggests that Rapid ICH combined with clinician interpretation, using neuroradiologist interpretation (board-certified or board eligible) as a reference standard, has a sensitivity of 92% (95% confidence intervals [CI], 78% to 98%) and a specificity of 100% (95% CI, 98% to 100%) for detecting ICH. Estimates of sensitivity and specificity for detecting LVO varied, based on studies using different modules of RapidAI as a standalone intervention, providing only indirect accuracy data.

However, HTERP acknowledged the limitations of these studies given the current evidence is of limited certainty. It is not clear how the observed reductions in radiology report turnaround times in patients with suspected stroke translate into patient outcomes, or in which centres these reductions would be most useful.

HTERP emphasized the lack of cost information and cost-effectiveness evidence, which prevents any conclusions to be made on demonstrated value for money or budget impact.

HTERP underscored RapidAI, the health technology under review, is being assessed for implementation within a complex system with many situational factors that require consideration. Given the available evidence, the committee could not produce a strong recommendation for or against the implementation of RapidAI. The committee provides elaboration and rationale for each component of the recommendation, described in Table 1, to add further context to the main recommendation.

Table 1: Recommendation Elaboration and Rationale

Component of recommendation	Elaboration and rationale	Examples of related implementation considerations from DTAC⁷ and other resources applicable to the health care context in Canada
Unable to provide recommendations for, or against, RapidAI regarding new investments and implementation or to recommend disinvestment	The current evidence is of limited certainty, making it insufficient to support recommendations regarding new investments and the implementation of stroke detection add-in functionalities to existing imaging platforms. Similarly, there is insufficient evidence to recommend disinvestment in centres in which RapidAI is already in use.	—
Where already in use, RapidAI to be used only alongside clinician interpretation	The evidence to date reflects the use of RapidAI in combination with clinician interpretation. There is no evidence to support the safe or effective replacement of clinician judgment with RapidAI. RapidAI is intended to complement, rather than replace, clinician interpretation of CT imaging and should be used as a supportive tool rather than a standalone diagnostic intervention.⁵ Users should be aware of the potential risk of overreliance on the AI system (e.g., automation bias).	Monitoring, maintenance, and sustainability, e.g., monitoring for automation bias (when a user’s conclusion is overly reliant on the device output while ignoring contrary data or conflicting human decisions).⁸ Ensuring responsibility and accountability in using AI-enabled medical devices, e.g., training, competency requirements, and guidelines are required to reduce the potential harm and liability for malpractice.⁸ The use of RapidAI alongside clinician(s) can help mitigate the potential harms regarding false positives and false negatives (e.g., an inaccurate diagnosis by RapidAI can be corrected or reinterpreted by clinicians before making care decisions).
Where RapidAI is in use, organized collection, monitoring, and evaluation of evidence is advised	The available clinical evidence has high levels of uncertainty. There is an incomplete understanding of ethical and equity considerations, including patient data considering PROGRESS-Plus⁹^,¹⁰ criteria, and location of health care setting (e.g., rural or remote). No cost information or cost-effectiveness evidence to inform decision-making (value for money or budget impact) was identified. AI-enabled medical devices require additional considerations for safe and successful implementation, including monitoring, maintenance, and sustainability throughout the AI product life cycle. Current use of RapidAI in clinical practice affords the opportunity for the generation of evidence to evaluate its value in health care systems. Evidence regarding its use in less-resourced centres with limited access to stroke care specialists would be important for identifying opportunities to improve equity in access to timely, high-quality stroke care.	Monitoring, maintenance, and sustainability to ensure its relevance, accuracy, efficacy, and safety¹¹ (e.g., monitoring to validate their performance and manage risks of overfitting, unintended bias, or degradation of the model, monitoring the risk of harm outcomes, such as technical failures in added time to diagnosis and delay in diagnosis). Additional collection of study data from clinical study participants and datasets that are representative of the intended population (continued collection of evidence within adopted centres and at new centres as they adopt the technology) Inclusiveness, equity, and reducing bias. It is important to attend to relevant ethics and equity considerations, such as algorithmic bias, lack of representation, data ownership, transparency, explainability, alongside evaluations of effectiveness and accuracy. For example, mismatches between the study populations and target populations could lead to a risk that the performance of RapidAI may not be applicable in all clinical settings (i.e., spectrum bias).¹²
Value for locations with limited access to stroke care specialists (e.g., rural and community settings)	This could not be assessed due to lack of evidence. There is potential value added for hospitals with limited access to stroke care specialists for early triage to inform, rather than make, stroke management decisions (e.g., preventing missed cases, reducing interpretation delays by speeding up the detection of LVO to inform decision-making about transferring patients for endovascular therapy). Stroke imaging interpretations by general radiologists can vary in turnaround time and accuracy, and a tool like RapidAI may be helpful and may reduce inequities in access to timely, high-quality stroke care. However, demonstrated evidence in these settings is required. There may be less value added for hospitals with comprehensive stroke centres, which have established and timely stroke management processes in place.	Compliance with all technical security obligations, with special consideration for implementation models where a shared server is used across multiple sites (e.g., hub and spoke model would require an external threat risk assessment by an external provider). User buy-in and organizational readiness may be a challenge in certain settings; for example, rural and community hospitals generally have fewer resources (e.g., funding, staff, capacity, and technical infrastructure).
Value for money or budget impact	This could not be assessed due to lack of evidence. There is a lack of both direct and indirect (such as technician staffing or training or data oversight) cost information and information about cost-effectiveness (e.g., cost per quality-adjusted life-year gained). Without these data, no conclusions can be made on budget impact or demonstrated value for money.	User buy-in and organizational readiness considerations take into account certain economic factors (e.g., demonstrated value, budget allocation, return on investment).¹¹

AI = artificial intelligence; DTAC = Digital Technology Assessment Criteria; LVO = large vessel occlusion.

Deliberation

Table 2 provides a detailed summary of the key discussion points raised during the meeting, organized by the applicable domains of value. The committee deliberated using the following 5 domains of value included in the CDA-AMC deliberative framework:

Unmet clinical need: Unmet clinical need refers to morbidity and/or mortality arising from a condition or symptom that is not addressed effectively by available treatments.
Clinical value: Clinical value is the value that patients derive from a health technology in terms of its effect on their health and health-related quality of life. The determination of the clinical value of a health technology requires the measurement of its clinical benefits and harms and an assessment of the impact of these effects on patients. Clinical benefits and harms are assessed against relevant comparators.
Economic considerations: Economic considerations refer to economic evidence to inform the financial, human, or other resource implications associated with the technology under review, and whether it is reasonable to allocate resources to the technology under review given its expected clinical benefits. Considerations may include the potential resource or cost impacts of the technology under review versus relevant comparator(s) and/or the potential economic value of the technology under review versus relevant comparator(s).
Impacts to health systems: This domain considers 2 distinct but interrelated components: organizational feasibility of adoption is the ease with which the health technology can be implemented in the health system while realizing its clinical value, while economic feasibility of adoption (affordability) considers how the adoption of a health technology will financially impact the payer or budget holder.
Distinct social and ethical considerations: This domain considers the distinct social and ethical implications of health technologies (including in their design, evaluation, and implementation) not already assessed in the other domains and how they affect patients, caregivers, populations, and the organization of health systems.

Table 2: Summary of Deliberation

Overarching questions	Discussion points
Unmet clinical need
Is there significant clinical need arising from the condition despite available treatments?	HTERP recognized that stroke is a clinical condition of relatively high incidence that is associated with significant morbidity and mortality. HTERP discussed that while there are effective treatments available for individuals experiencing acute stroke, access to these treatments requires timely, accurate identification of stroke. This access varies within Canada (e.g., across jurisdictions, rural vs. urban settings, hospitals with comprehensive stroke centres vs. centres with limited resources), and as such, there are some unmet clinical needs in the timely detection of stroke. For effective management of acute stroke, it is vital to get the correct diagnosis (e.g., LVO vs. ICH) and the right treatment at the right time (“time is brain”), which may require access to higher levels of care and transfer to stroke centres. Timely stroke diagnosis and treatment may help improve functional outcomes for patients who have experienced a stroke (e.g., severity of neurologic impairment). In hospitals in Canada without a comprehensive stroke centre, stroke imaging interpretations by general radiologists can vary in turnaround time and accuracy. For centres with limited access to stroke care specialists, HTERP members discussed how RapidAI could meet the clinical need of reviewing CT imaging for acute stroke alongside general radiologists (as a double-check) in a time sensitive manner. This clinical need for RapidAI is similar to how collision avoidance systems are now implemented in cars. However, with experienced drivers (i.e., neuroradiologists working in comprehensive stroke centres), a system such as RapidAI is probably unnecessary and can sometimes introduce dangerous noise. Therefore, identifying the exact settings of need for a tool like RapidAI (i.e., less-resourced centres with limited access to stroke care specialists) is important.
Clinical value
Does the technology under review demonstrate acceptable clinical value vs. relevant comparators in the setting in Canada?	HTERP discussed the lack of rigour in the available evidence. For example, HTERP acknowledged evidence from the cohort studies, identified in the RapidAI review, was of low or very low certainty because of imprecision and critical risk of bias due to confounding. For detecting LVO, the diagnostic accuracy studies to date examined RapidAI as a standalone diagnostic tool, which limits their applicability to clinical practice, where RapidAI is to be used to assist clinicians in interpreting CT imaging. For detecting ICH, the diagnostic accuracy study provided low certainty evidence due to risk of bias and imprecision. HTERP also noted the findings from these studies cannot be generalized to many settings in Canada, as the representativeness of the data used to train the algorithms is unknown, and the studies were conducted exclusively in centres with expertise in stroke management. HTERP recognized the difficulty in conducting high-quality studies in smaller stroke centres; however, such studies would help inform RapidAI’s potential impact and value in these settings. The uncertainty of the evidence makes it difficult to make a strong recommendation either for adoption or disinvestment. HTERP discussed that despite being limited and of low quality, the evidence available is encouraging, providing it translates into demonstratable clinical benefits (e.g., mortality, length of hospital stay, health-related quality of life, and patient harms). In hospitals without comprehensive stroke centres, in addition to clinical judgment, RapidAI may demonstrate added clinical value as high sensitivity and negative predictive value may improve the accuracy of early detection of stroke due to LVO or ICH and identify patients requiring transfer for timely treatment. Additional research is required to continue demonstrating high sensitivity, with subsequent human confirmation for LVO or ICH diagnosis, given the critical nature of stroke diagnosis. However, there are several caveats to acknowledge the number of uncertainties associated with this technology that might impact its realized value when implemented within the complex health care systems. HTERP acknowledged the need for more research to have confidence in RapidAI’s place in care. HTERP discussed a need for more robust clinical evidence including the risk of harmful outcomes (e.g., technical failures in added time to diagnosis or delay in diagnosis), particularly those associated with false negatives (missed cases). HTERP acknowledged that the potential added clinical value depends on existing installed resources, which vary across jurisdictions and regions. Conflicts may exist for centres in terms of weighing the potential for the highest utility and their ability to implement such technologies. Depending on the volume of stroke presentations and in-house imaging or clinical expertise, technologies such as RapidAI may offer higher utility for smaller regional or rural locations, but these sites often have resource constraints that may act as a barrier to acquiring, implementing, using, or monitoring these technologies. For larger academic centres, there are more likely to be organizational structures, adequate scanners, good computer infrastructure, and technicians with the skills and availability to conduct the scans. In these centres, installing an additional technology (e.g., RapidAI) may be plausible and practicable. However, there may be lower utility in implementing the technology in these centres as adequate resources and expertise for the timely detection of stroke are likely already in place. The available evidence was collected in comprehensive stroke centres, and its applicability has not been tested in smaller, remote locations that are less likely to have the volume of stroke cases and resources necessary to conduct studies. HTERP discussed the need for evidence on the performance of RapidAI and role in different clinical pathways to understand its utility in different locations (e.g., urban or larger centres vs. rural or smaller centres). Moreover, the HTERP discussed the comparator or reference standard for studies included in this RapidAI review, which was CT imaging review by clinicians alone (e.g., assessment by a single radiologist, consensus obtained from a panel of neuroradiologists). The committee suggested the most relevant comparator in the setting in Canada is clinicians alone, but especially physicians who are not stroke care specialists (e.g., emergency physicians, general internists, family physicians) to help understand RapidAI’s potential added value in these smaller, remote locations.
Economic considerations
Are there economic considerations that are relevant to address when implementing the technology under review?	HTERP recognized the lack of cost-effectiveness evidence. HTERP raised examples of how there could be potential for cost savings if RapidAI reduced missed cases or untimely patient transfers. However, the CDA-AMC report did not identify evidence to support this, which suggests there is a need for more research in this area.
Impacts to health systems
Are there expected organizational impacts of implementing the technology that might affect health system sustainability? Does the magnitude of the expected budget impact of implementing the health technology or its uncertainty need to be addressed?	While it was acknowledged that RapidAI is an add-on to existing RAPID systems which, if currently in place, is unlikely to lead to a significant budget impact, HTERP discussed the lack of available cost information, including expected budget impact, cost of implementation (e.g., training personnel or supporting IT infrastructure), and the need to balance costs and benefits given limited resources and opportunity costs (e.g., investing in a new technology or replacing an old existing machine), especially in small facilities that could benefit from this technology. HTERP discussed considerations regarding the organizational feasibility of adoption. Even for well-resourced hospitals, there are difficulties in implementing software like RapidAI (e.g., requirement for local information technology department support). It is expected that there will be more challenges with rural or community hospitals. Depending on where RapidAI is implemented, it may require additional resources (e.g., infrastructure requirements, training, and additional radiologist technicians). Of those sites in which the RapidAI system has already been installed, it is not clear how many use the AI functionality for the detection of LVO and ICH. There are opportunities for learning from existing sites using this technology to understand the feasibility of adoption (e.g., single-site hub model vs. hub and spoke model, where the hub and spoke implementation model would enable remote centres that do not have the resources that comprehensive stroke centres do to use and benefit from tools like RapidAI, including the ability to notify stroke centres and identify patients for transfer in a more timely way). HTERP noted that the implementation of RapidAI may have implications for the use of other health services, such as emergency medical services and patient transport. For example, more accurate identification of patients requiring transfer to larger stroke centres could reduce the opportunity cost of ambulances unnecessarily travelling out of the communities they are meant to serve. HTERP acknowledged that the RapidAI software does not adapt to real-world data collected during its use. The panel described the need for validation and monitoring of the underlying algorithms, especially in real-life settings and with software updates.
Distinct social and ethical considerations
Is there a significant nonclinical need arising from the condition, despite available treatments, which would potentially be addressed by the technology under review? Are there any important measures that should be implemented to ensure that the use of the technology addresses relevant social and ethical implications?	HTERP discussed the lack of evidence related to subgroup effects that consider PROGRESS-Plus⁹^,¹⁰ criteria. More information is needed to determine whether this technology will improve the outcomes of equity-deserving groups. HTERP acknowledged that there are certain social and ethical considerations when considering the implementation of an AI-enabled medical device, such as the potential risk of bias in the stroke detection algorithms, human-machine interaction, and the potential risk of overreliance on the AI system (e.g., automation bias). HTERP discussed examples of guidance (e.g., DTAC) that could be useful as a resource for senior decision-makers when considering the implementation of AI health technologies, including RapidAI. The committee acknowledged CDA-AMC’s findings that no checklist like DTAC exists in Canada. There is a potential need for a comprehensive checklist like DTAC for use in Canada to provide confidence that digital health tools used in Canada meet our clinical safety, data protection, technical security, interoperability, and usability and accessibility standards. Like DTAC, the checklist would be designed for health care organizations to use to assess manufacturers at the point of procurement or as part of a due diligence process.⁷ This checklist could be an adaptation of DTAC for the health care context in Canada and include additional implementation considerations for AI-enabled medical devices to ensure that these technologies meet the minimum baseline standards set out by DTAC and inform the next steps for the safe and successful implementation of AI-enabled medical devices in Canada.

AI = artificial intelligence; CDA-AMC = Canada’s Drug Agency; DTAC = Digital Technology Assessment Criteria; HTERP = Health Technology Expert Review Panel; ICH = intracranial hemorrhage; LVO = large vessel occlusion; vs. = versus.

What Did CDA-AMC Find?

In this section, the CDA-AMC provides a summary of key findings from the CDA-AMC reviews, including:

clinical, economic, and ethics and equity considerations identified from the RapidAI evidence review
implementation considerations for digital health technologies described in the AI implementation review, including ethics and equity considerations and additional considerations identified for the use of AI-enabled medical devices in Canada.

To supplement the summary of HTERP’s deliberation, CDA-AMC provides a summary of key findings and uncertainties from the reviews (Table 3), organized by the HTA deliberative framework themes: unmet clinical need, clinical value, economic considerations, impacts to health systems, and distinct social and ethical considerations.

RapidAI Review

Clinical Evidence

This review included 2 cohort studies and 11 diagnostic accuracy studies evaluating the effectiveness and accuracy of RapidAI for detecting stroke. Of these, 3 studies examined RapidAI as it is intended to be used in clinical practice (i.e., to complement clinician interpretation of CT imaging), while the remaining 10 studies assessed RapidAI as a standalone diagnostic tool. Twelve studies were specific to patients with suspected LVO, and 1 study included patients with suspected ICH.

The overall certainty of the evidence for all outcome comparisons was assessed using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach. Outcomes for GRADE assessment were selected because they were important for those who might be affected by the intervention, including the patient contributor and experts who were engaged or consulted during this project.

Low certainty evidence suggests that using RapidAI to support clinician interpretation of CT angiography images may result in clinically important reductions in radiology report turnaround time for patients with suspected acute ischemic stroke. The evidence is very uncertain about the effect of RapidAI on other time to intervention metrics (e.g., time from door to intubation, time from door to revascularization). Additionally, the evidence was very uncertain regarding the effect of RapidAI on patient outcomes, including measures of neurologic deficit, degree of functional neurologic disability, and response to therapy (e.g., reperfusion rates).

For diagnostic accuracy, low certainty evidence suggests that when used alongside clinician interpretation (with neuroradiologist interpretation alone [board-certified or board eligible] as the reference standard), Rapid ICH has a sensitivity of 92% (95% CI, 78% to 98%) and a specificity of 100% (95% CI, 98% to 100%) for detecting ICH. When used as a standalone diagnostic tool, evidence from 10 diagnostic accuracy studies indicates that the sensitivity of RapidAI for detecting LVO ranges from 62% to 96%, with specificity ranging from 65% to 98%. Differences in the types of LVO assessed, the versions and modules of RapidAI used, the type of CT imaging analyzed (i.e., noncontrast CT or CT angiography), and the methods for determining reference standard diagnoses likely contributed to the variability in observed values for sensitivity and specificity. The certainty of the evidence for these findings was moderate, low, or very low, or there was insufficient information to judge certainty. These results have unclear applicability to clinical practice, as the accuracy of RapidAI as a standalone diagnostic tool does not directly answer how much it might improve the accuracy of a clinician reader.

Overall, the evidence suggests that RapidAI has potential to improve acute stroke care by creating efficiencies in the diagnostic process. However, the impact of RapidAI on many outcomes, including those that are important to patients, is uncertain due to the limitations of the available evidence. To improve the certainty of findings and provide a better understanding of the potential benefits and harms of RapidAI, there is a need for evidence from robustly conducted studies at lower risk of bias that enrol diverse patient populations and measure outcomes that are important to patients, with improved reporting.

Economic Evidence

Cost and Cost-Effectiveness

CDA-AMC did not find any studies on the cost-effectiveness of RapidAI for detecting stroke that met the selection criteria for this review; therefore, the cost-effectiveness of RapidAI for stroke detection is currently unknown. CDA-AMC reached out to the manufacturer but were unable to obtain updated pricing information on this technology. From publicly available sources, CDA-AMC identified resource considerations regarding the implementation of RAPID platform and a budget impact analysis by Ontario Health¹³ and provided it as relevant economic information in the discussion of the conclusions and implications for decision-making.

Ethics and Equity Considerations

CDA-AMC primarily leveraged and adapted WHO’s guidance on Ethics and Governance of Artificial Intelligence for Health¹⁴ to organize and reflect on these considerations and their implications. Additionally, CDA-AMC drew from the ethical considerations proposed by the AI Task Force of the Society of Nuclear Medicine and Molecular Imaging,¹⁵ whose recommendations on the major ethical considerations during the deployment of AI-enabled medical devices are directly relevant to the use of AI in diagnostic workups. The review explored the ethical concerns that could arise in the development, design, testing, and deployment of RapidAI or other AI-enabled digital health technologies. Considerations related to equity, autonomy, privacy, transparency, and explainability of machine learning models can influence how, when, and for whom technologies such as RapidAI are leveraged by clinicians, patients, and health care institutions. These considerations should be addressed throughout the technology life cycle to ensure fair and equitable decision-making around the risks, benefits, and trade-offs for patients, caregivers, clinicians, and health systems.

Patient Perspective

The patient contributor identified important outcomes for stroke care, including speed and accuracy of diagnosis, minimizing the damaging effects of stroke, and reducing mortality rates. She also highlighted ethical considerations regarding the use of AI in health care, such as data privacy, equitable access, and informing patients about the use of AI technologies in the care pathway.

AI Implementation Review

Applying DTAC to Health Care Context in Canada

CDA-AMC conducted this review to assist health systems in Canada in preparing for the uptake of AI-enabled medical devices. CDA-AMC assessed whether the safeguards and assessment criteria captured by DTAC and other AI-related resources are in place in Canada to inform decision-making around the digital infrastructure elements of implementation. Focusing on DTAC’s core areas (i.e., clinical safety, data protection, technical security, interoperability, and usability and accessibility),⁷ this review found many of DTAC’s assessment criteria have equivalent or similar guidance for the health care context in Canada with some important caveats. Some exceptions derive from the differences in Canada’s current governance and health care structure (e.g., the level of governance for Canada’s privacy laws depends on the type of data and jurisdiction and, unlike the UK, Canada does not have electronic health records managed at the federal level). The review suggests further investigation to confirm if certain policies in Canada, such as clinical safety, provide sufficient coverage to fulfill DTAC’s criteria.

AI Implementation Considerations

CDA-AMC identified additional considerations for implementing AI-enabled medical devices that health care decision-makers may consider for the safe and successful implementation of AI in health care in Canada. Much of the identified guidance has ethical and equity implications and emphasizes implementation considerations that apply to the AI system’s life cycle. CDA-AMC identified monitoring, maintenance, and sustainability throughout the AI product life cycle as a key consideration for DTAC’s core assessment areas. Additional considerations include:

AI data governance and data protection
multidisciplinary data governance teams throughout the AI product life cycle
technical infrastructure and integration
transparency, explainability, and intelligibility
inclusiveness, equity, and minimizing bias
responsibility and accountability
user buy-in and organizational readiness.

CDA-AMC suggests using the findings from this review for all AI-enabled medical devices alongside reviews of specific AI technologies, including the concurrent RapidAI review. The report will serve as a foundational report tailored to each AI topic and updated with the latest developments in the regulation and other aspects of managing AI in Canada.

Ethics and Equity Considerations

This review integrated ethics and equity considerations arising in DTAC,⁷ augmented with key ethics of AI tools and frameworks.¹⁴^,¹⁶^-¹⁸ CDA-AMC noted that the ethical considerations for digital health technologies were found in DTAC’s data protection domain, which relates to data privacy, management, and ownership concepts. CDA-AMC described the equity considerations for digital health technologies were found under DTAC’s usability and accessibility domain, which primarily relates to the involvement of relevant users in technology design, and whether their needs are incorporated into elements of technology design. CDA-AMC leveraged WHO’s Ethics and Governance of Artificial Intelligence for Health guidance that speaks to the fundamental ethical principles for AI use for health.¹⁴ This guidance aligns with certain DTAC criteria,⁷ but offers more specific considerations for ethical considerations inherent in AI technologies.¹⁴

Patient Perspective

Relevant to this review, the patient contributor engaged in the RapidAI review highlighted data protection and privacy considerations (e.g., informing the patient about using AI technologies as part of care provided) and accessibility and equity considerations (e.g., equitable access).

Key Findings and Uncertainties

Table 3: Summary of Key Findings and Uncertainties

Section	Key findings	Uncertainties
Unmet clinical need	Stroke, also known as cerebrovascular accident, is a life-threatening medical condition characterized by loss of neurologic function. In Canada, stroke is 1 of the leading causes of death and a major cause of disability (e.g., limitations to physical functioning and sensory impairment),¹⁹^-²¹ with more than 100,000 stroke events resulting in hospital or emergency department presentations each year.²² Stroke diagnosis and intervention is time sensitive. Accurately determining whether a stroke is ischemic or hemorrhagic with neuroimaging studies, often using CT imaging, is crucial for selecting appropriate treatment options.²³ Misidentifying the type of stroke can lead to inappropriate treatments that may exacerbate the condition. Health care inequities exist in stroke incidence, prevalence, symptoms, quality of care, and outcomes across factors such as race, ethnicity, gender, sex, disability status, age, geographic location, and socioeconomic status.²⁴^-³¹ For example, data from 2007 to 2011 indicate that rural hospitals in Canada have significantly higher 30-day in-hospital mortality rates following stroke compared to urban academic hospitals and the national average.³² RapidAI is an AI-enabled software that facilitates the viewing, processing, and analysis of CT imaging to aid clinicians in assessing patients with suspected stroke.	Existing methods for diagnosing stroke, such as imaging and clinical assessments, are often effective, but the accuracy and speed of diagnosis can vary across health care settings due to differences in the availability of imaging equipment and access to stroke care specialists.
Clinical value	CDA-AMC reviewed evidence from 2 cohort studies and 11 diagnostic accuracy studies that assessed the effectiveness and accuracy of RapidAI for detecting stroke. RapidAI has the potential to improve acute stroke care by creating efficiencies in the diagnostic process. For detecting LVO: Evaluation of CT angiography imaging by Rapid LVO combined with clinician interpretation, compared to clinician interpretation alone, may result in clinically important reductions in radiology report turnaround time in patients with suspected stroke (low certainty). As a standalone diagnostic intervention, the sensitivity of RapidAI for detecting LVO ranges from 62% to 96%, while estimates of specificity range from 65% to 98% (moderate, low, or very low certainty, or there was insufficient information to judge certainty; results from 10 diagnostic accuracy studies). Heterogeneity in the types of LVO assessed, the versions or modules of RapidAI used, the type of CT imaging analyzed (i.e., noncontrast CT or CT angiography), and the methods for determining reference standard diagnoses likely contributed to the variability in these results. For detecting ICH: Rapid ICH combined with clinician interpretation, using clinician interpretation as a reference standard, has a sensitivity of 92% (95% CI, 78% to 98%) and a specificity of 100% (95% CI, 98% to 100%) (low certainty; results from 1 diagnostic accuracy study). CDA-AMC investigated DTAC that captures the additional considerations for the implementation of digital health technologies not captured by traditional HTA to identify if it could be applied to the health care context in Canada. With some caveats, they found that many of DTAC’s assessment criteria have equivalent or similar guidance for the health care context in Canada. They identified a need for a checklist like DTAC that senior decision-makers can use in Canada. This checklist could be an adaptation of DTAC and include additional implementation considerations for AI-enabled medical devices to ensure that these technologies meet the minimum baseline standards set out by DTAC and inform the next steps for the safe and successful implementation of AI-enabled medical devices in Canada.	The impact of RapidAI on many outcomes, including those that are important to patients, is uncertain due to the limitations of the available evidence. Clinical experts and the patient contributor identified key outcomes of interest: mortality, length of hospital stay, health-related quality of life, and patient harms (e.g., administration of harmful therapies or undertreatment due to inaccurate diagnosis). CDA-AMC did not identify any evidence on the effects of these outcomes as well as health care resource implications. One study examined the diagnostic accuracy of RapidAI as it would be used in clinical practice (i.e., as a tool to support the review of CT imaging). More studies are needed to form conclusions. The evidence is very uncertain about the effects of RapidAI on other time to intervention metrics, measures of physical and cognitive function, and response to therapy (e.g., reperfusion rates). For detecting LVO: Evidence from the cohort studies was of low or very low certainty, primarily because of critical risk of bias due to confounding and imprecision. Estimates of sensitivity and specificity for detecting LVO varied and were based on studies using different modules of RapidAI as a standalone intervention, providing only indirect accuracy data. For detecting ICH: No evidence was found on the impact of RapidAI for detecting ICH on clinical outcomes, such as time to intervention or direct patient outcomes (e.g., functional status). CDA-AMC highlighted that it is unclear if the potential time saved during stroke diagnosis with RapidAI will improve patient outcomes and access to care broadly. Other aspects of the health system infrastructure, including the availability of emergency medical services, stroke care specialists, operating rooms, imaging equipment, radiology technologists, and other emergency care resources may still limit the speed of treatment. The patient contributor highlighted the important stroke outcomes, including speed and accuracy of diagnosis, minimizing the damaging effects of stroke, and reducing mortality rates. CDA-AMC did not identify any evidence on the effects of these outcomes. CDA-AMC highlighted the importance of attending to relevant ethics and equity considerations, such as equitable access, accountability, algorithmic bias, and lack of representation, alongside evaluations of effectiveness and accuracy. Many ethical considerations related to AI in health care, as outlined by the Scottish Health Technology Group’s HTA framework⁶ and other foundational ethics of AI tools and frameworks, are often inadequately addressed in studies evaluating the effectiveness or accuracy of commercialized AI-enabled medical devices (i.e., the types of evidence included in the RapidAI review). CDC-AMC stated that further investigation is required to understand if certain policies in Canada provide sufficient coverage to fulfill DTAC’s criteria (e.g., clinical safety).
Economic considerations	No economic evaluations were identified to assess the economic value of RapidAI.	The acquisition cost of RapidAI is uncertain. The cost-effectiveness of RapidAI for stroke detection is currently unknown. CDA-AMC did not identify any studies that evaluated the cost-effectiveness of RapidAI (with or without clinician interpretation) to detect ICH or LVO.
Impacts to health systems	From publicly available sources, CDA-AMC identified resource considerations regarding the implementation of RAPID. There are currently 2 approaches to implement RAPID. The first is a single-site (hub) model that loads the application onto a server within the organization’s firewall, so that the CT-MRI scanners at the site are connected to the system and the server allows the results to be processed for viewing and storage. The second approach is a multiple site model (hub and spoke) where the application is loaded on a server behind the firewall of 1 organization, which can be shared across sites. The CT-MRI scanners at each site are configured to allow the results at the hub and spoke sites to be processed at the hub site and sent back to the originator site for viewing and storage. In terms of costs, there is a hub licensing fee, as well as server installation costs and training fees; the latter 2 of which would be shared across sites in the hub and spoke model. The ‘hub and spoke’ model has additional connection requirements (e.g., bandwidth and connection) and requires an external threat risk assessment conducted by an external provider. Furthermore, a data sharing agreement would need to be in place between hub and spoke sites.² In 2020, Ontario Health published a health technology assessment¹³ on the use of automated CT perfusion imaging to aid in the selection of patients with acute ischemic stroke for mechanical thrombectomy. Their budget impact analysis stated the annual cost of a licence for the RAPID neuroimaging platform in 2019 was, depending on the number of connected scanners, between $27,500 (for 2 scanners) and $32,500 (for unlimited scanners) per hospital.¹³ There was an added cost in the first year due to 1-time fees related to the initial implementation and optimization of Rapid, as well as for training staff ($12,350 for hospitals with 2 or more scanners).¹³ It is estimated that publicly funding automated CT perfusion imaging across 42 hospitals in Ontario resulted in additional costs of $1.3 million in the first year and $0.9 million per year thereafter.¹³ This costing information was for the full RAPID platform, which included several modules for conducting various cerebrovascular diagnostic procedures, in addition to automated CT perfusion.	CDA-AMC was unable to address the potential budget impact of implementing RapidAI for stroke detection in Canada. It is unclear whether the results of the Ontario Health report could be generalized to other populations within Canada. CDA-AMC was unable to address RapidAI’s capacity to inform treatment selection (e.g., selecting patients for reperfusion therapy). The patient contributor raised access concerns about the availability of AI technologies in hospitals outside of urban stroke centres. She questioned whether AI-enabled stroke detection software would be available to all major hospitals for assisting in triaging and potentially transferring patients more quickly, or if its use would be restricted to certain facilities. CDA-AMC’s report explains the budget, personnel, infrastructure and training requirements needed to implement RapidAI may limit its use to better resourced hospitals or health care centres, despite its potential to potentially improve some access to stroke care in rural and remote settings. CDA-AMC explains that even if there were data to suggest the performance of RapidAI is robust across diverse patient populations (i.e., low risk of bias in the algorithm’s performance), limited access based on geographic location could exacerbate existing health inequities.³³
Distinct social and ethical considerations	CDA-AMC identified ethics and equity considerations relevant to RapidAI, which were found through published literature and patient, clinician, and expert input. Ethical and equity considerations related to patient autonomy, privacy, transparency, access, and algorithmic bias have implications across the technology life cycle when using RapidAI for detecting stroke. In addition to the evidence on the effectiveness and accuracy of RapidAI for detecting stroke, the CDA-AMC RapidAI report suggests decision-makers may wish to reflect on the ethical and equity considerations that arise during the deployment of AI-enabled technologies, such as those related to autonomy, privacy, transparency and explainability of machine learning models, and the need for considerations related to equity and access in their design, development, and deployment. Relevant for both CDA-AMC reviews, the patient contributor engaged for the RapidAI review highlighted several relevant considerations (e.g., data protection and privacy and accessibility and equity). CDA-AMC identified several considerations for implementing AI-enabled medical devices, with many having underlying ethical and equity implications. Much of the identified guidance emphasizes implementation considerations that apply to the AI system’s life cycle, such as ensuring AI-enabled medical devices are monitored, maintained, and sustainable. Other example considerations include AI data governance and data protection; transparency and explainability; and inclusiveness, equity, and minimizing bias.	CDA-AMC highlighted that it is unclear if the potential time saved during stroke diagnosis with RapidAI will improve patient outcomes and access to care broadly. Other aspects of the health system infrastructure, including the availability of emergency medical services, stroke care specialists, operating rooms, imaging equipment, radiology technologists, and other emergency care resources may still limit the speed of treatment. CDA-AMC acknowledged the importance of additional ethical and equity considerations, but these considerations tend to be underreported in the identified evidence that is generally examined when evaluating the effectiveness and accuracy of interventions. CDA-AMC highlighted some of these gaps, such as: Included studies did not provide details on the characteristics of study populations and did not conduct subgroup analyses based on these criteria, preventing CDA-AMC from evaluating how RapidAI might perform across different groups. Included studies did not describe the methods used to develop RapidAI's machine learning models, preventing CDA-AMC from assessing the representativeness and diversity of the training dataset and commenting on considerations related to inclusivity. It is unclear if RapidAI has undergone bias assessment to determine if certain patient subgroups (e.g., based on age, gender, and ethnicity) are disproportionately affected by the model outputs, preventing CDA-AMC from commenting on the potential bias risks in the stroke detection algorithms.

AI = artificial intelligence; CDA-AMC = Canada’s Drug Agency; CI = confidence interval; DTAC = Digital Technology Assessment Criteria; HTA = health technology assessment; ICH = intracranial hemorrhage; LVO = large vessel occlusion.

HTERP Core Members

Leslie Anne Campbell — Chair, Nova Scotia

Louise Bird — Patient member, Saskatchewan

Brian Chan — Health Economist, Ontario

Sandor Demeter — Health Care Practitioner, Manitoba

Lawrence Mbuagbaw — Health Technology Assessment Specialist, Ontario

Duncan Steele — Ethicist, Alberta

Note: As of January 2024, there is currently a committee seat vacancy for 1 health care practitioner member.

Expert Members

Michael D. Hill — Stroke Neurologist, Senior Medical Director for Stroke, and Professor, Alberta

Laurent Létourneau-Guillon — Neuroradiologist and Clinician Scientist, Québec

Frank Silver — Stroke Neurologist, Medical Director, and Affiliate Scientist, Ontario

Meeting date: September 12, 2024

Conflicts of Interest

Michael D. Hill is the President of the Canadian Neurological Sciences Federation. He reports funding from Alberta Innovates, Boehringer Ingelheim, and Canadian Institutes for Health Research, and receipt of stocks or stock options from Circle Cardiovascular Inc. and Basking Biosciences.

Laurent Létourneau-Guillon is a member of the Artificial Intelligence Committee with the American Society of Neuroradiology and the Canadian Association of Radiologists. He reports a salary award from Fonds de Recherche du Québec en Santé and Fondation de l’Association des Radiologistes du Québec and research grants from Fonds de Recherche du Québec en Santé, Fondation de l’Association des Radiologistes du Québec, and Québec Bio-Imaging Network for work in the space of health technologies and artificial intelligence.

Frank Silver has used RAPID perfusion imaging software (not the AI component) and advocated for the use of perfusion imaging as a tool for the management of acute stroke cases in Ontario, primarily at his affiliated hospital (University Health Network, Ontario), and as a member of the Ontario Hyperacute Stroke Steering Committee.

No other conflicts of interest were identified.

References

1.Straka M, Albers GW, Bammer R. Real-time diffusion-perfusion mismatch analysis in acute stroke. J Magn Reson Imaging. 2010;32(5):1024-1037. PubMed

2.CorHealth Ontario. Automated Stroke Imaging Software Package (Ischemaview RAPID) Implementation Frequently Asked Questions and Lessons Learned. Toronto (ON): Ontario Health; 2020: https://www.corhealthontario.ca/AutomatedSoftware-Lessons-Learned-and-Frequently-Asked-Questions-Version-8.0-Final-Version.pdf. Accessed 2024 Aug 26.

3.RapidAI. https://www.rapidai.com/. Accessed 2024 Jun 13.

4.Artificial intelligence (AI)-derived software to help clinical decision making in stroke. Diagnostics guidance (DG57]. London (GB): NICE; 2024: https://www.nice.org.uk/guidance/dg57. Accessed 2024 Jun 13.

5.RAPID User’s Manual. Menlo Park (CA): iSchemaView, Inc.

6.Scottish Health Technologies Group (SHTG). An evidence framework for the assessment of health technologies. Edinburgh (GB): Healthcare Improvement Scotland; 2023: https://shtg.scot/what-we-do/evidence-framework/. Accessed 2024 Jul 24.

7.Digital Technology Assessment Criteria (DTAC). London (GB): NHS England; 2021: https://transform.england.nhs.uk/key-tools-and-info/digital-technology-assessment-criteria-dtac/. Accessed 2024 Jun 19.

8.Canadian Association of Radiologists Artificial Intelligence Working Group. Canadian Association of Radiologists white paper on ethical and legal issues related to artificial intelligence in radiology. CAR J. 2019;70(2):107-118.

9.O'Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67(1):56-64. PubMed

10.Oliver S, Kavanagh J, Caird J, et al. Health Promotion, Inequalities and Young People's health: a Systematic Review of Research. EPPI-Centre report no. 161. London (GB): University of London; 2008: https://eppi.ioe.ac.uk/cms/Portals/0/PDF%20reviews%20and%20summaries/Inequalities%20Young%20People%20R2008Oliver.pdf?ver=2010-12-22-123934-167. Accessed 2024 Jun 4.

11.Dhalla A, Akinli Kocak S, Wan F, Da Silva J, Jain T. Vector Institute Health AI Implementation Toolkit. Toronto (ON): Vector Institute; 2023: https://vectorinstitute.ai/health-ai-implementation-toolkit/. Accessed 2024 Jun 14.

12.Willis BH. Spectrum bias—why clinicians need to be cautious when applying diagnostic test studies. Fam Pract. 2008;25(5):390-396. PubMed

13.Automated CT perfusion imaging to aid in the selection of patients with acute ischemic stroke for mechanical thrombectomy: A health technology assessment. Ont Health Technol Assess Ser. 2020;20(13):1-87. PubMed

14.Health Ethics & Governance (HEG). Ethics and governance of artificial intelligence for health. Geneva (CHE): World Health Organization; 2021: https://www.who.int/publications/i/item/9789240029200. Accessed 2024 Jun 20.

15.Herington J, McCradden MD, Creel K, et al. Ethical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance. J Nucl Med. 2023;64(10):1509-1515. PubMed

16.Recommendation on the Ethics of Artificial Intelligence. Paris (FR): United Nations Educational, Scientific and Cultural Organization; 2021: https://unesdoc.unesco.org/ark:/48223/pf0000381137. Accessed 2024 Jun 5.

17.EUnetHTA Joint Action 2. HTA Core Model, Version 3.0 Work package 8. Diemen (NLD): European Network for Health Technology Assessment; 2016: https://www.eunethta.eu/wp-content/uploads/2018/01/HTACoreModel3.0.pdf. Accessed 2024 Jun 5.

18.Benkhalti M, Espinoza M, Cookson R, Welch V, Tugwell P, Dagenais P. Development of a checklist to guide equity considerations in health technology assessment. Int J Technol Assess Health Care. 2021;37:e17. PubMed

19.Public Health Agency of Canada. Stroke in Canada. Ottawa (ON): Government of Canada; 2022: https://www.canada.ca/en/public-health/services/publications/diseases-conditions/stroke-in-canada.html. Accessed 2024 May 3.

20.FACT SHEET: Stroke Statistics. Ontario Stroke Network; 2012: https://www.ontariostrokenetwork.ca/pdf/Final_Fact_Sheet_Stroke_Stats_3.pdf. Accessed 2024 May 15.

21.Gittins M, Lugo-Palacios D, Vail A, et al. Stroke impairment categories: A new way to classify the effects of stroke based on stroke-related impairments. Clin Rehabil. 2021;35(3):446-458. PubMed

22.Holodinsky JK, Lindsay P, Yu AYX, Ganesh A, Joundi RA, Hill MD. Estimating the Number of Hospital or Emergency Department Presentations for Stroke in Canada. Can J Neurol Sci. 2023;50(6):820-825. PubMed

23.Tadi P, Lui F. Acute Stroke. StatPearls. Treasure Island (FL): StatPearls Publishing LLC.; 2023: https://www.ncbi.nlm.nih.gov/books/NBK535369/. Accessed 2024 May 15.

24.Muller CJ, Alonso A, Forster J, et al. Stroke Incidence and Survival in American Indians, Blacks, and Whites: The Strong Heart Study and Atherosclerosis Risk in Communities Study. J Am Heart Assoc. 2019;8(12):e010229. PubMed

25.Hyldgård VB, Johnsen SP, Støvring H, Søgaard R. Socioeconomic Status And Acute Stroke Care: Has The Inequality Gap Been Closed? Clin Epidemiol. 2019;11:933-941. PubMed

26.Thompson SG, Barber PA, Gommans JH, et al. Geographic Disparities in Stroke Outcomes and Service Access: A Prospective Observational Study. Neurology. 2022;99(4):e414-e426. PubMed

27.Ikeme S, Kottenmeier E, Uzochukwu G, Brinjikji W. Evidence-Based Disparities in Stroke Care Metrics and Outcomes in the United States: A Systematic Review. Stroke. 2022;53(3):670-679. PubMed

28.Cruise C, Mfoafo M’Carthy N, Ganesh A, Lashewicz B. Imperfect Patients: Disparities in Treatment of Stroke Patients with Premorbid Disability. CJNS. 2023;50(6):826-837. PubMed

29.Stroke and mental health: The invisible and inequitable effects on women. 2023 Stroke report. Toronto (ON): Heart and Stroke Foundation of Canada; 2023: https://issuu.com/heartandstroke/docs/womens_stroke_report_june_2023/5. Accessed 2024 Jun 4.

30.Lives disrupted: the impact of stroke on women. 2018 Stroke report. Toronto (ON): Heart and Stroke Foundation of Canada; 2018: https://www.heartandstroke.ca/-/media/pdf-files/canada/stroke-report/strokereport2018.pdf?rev=8491d9c349f7404491f36be67f649c0b. Accessed 2024 Jun 4.

31.Towfighi A, Boden-Albala B, Cruz-Flores S, et al. Strategies to Reduce Racial and Ethnic Inequities in Stroke Preparedness, Care, Recovery, and Risk Factor Control: A Scientific Statement From the American Heart Association. Stroke. 2023;54(7):e371-e388. PubMed

32.Fleet R, Bussières S, Tounkara FK, et al. Rural versus urban academic hospital mortality following stroke in Canada. PLoS One. 2018;13(1):e0191151. PubMed

33.Ratwani RM, Sutton K, Galarraga JE. Addressing AI Algorithmic Bias in Health Care. JAMA. 2024. PubMed

ISSN: 2563-6596

Canada’s Drug Agency (CDA-AMC) is a pan-Canadian health organization. Created and funded by Canada’s federal, provincial, and territorial governments, we’re responsible for driving better coordination, alignment, and public value within Canada’s drug and health technology landscape. We provide Canada’s health system leaders with independent evidence and advice so they can make informed drug, health technology, and health system decisions, and we collaborate with national and international partners to enhance our collective impact.

Disclaimer: CDA-AMC has taken care to ensure that the information in this document was accurate, complete, and up to date when it was published, but does not make any guarantee to that effect. Your use of this information is subject to this disclaimer and the Terms of Use at cda-amc.ca.

The information in this document is made available for informational and educational purposes only and should not be used as a substitute for professional medical advice, the application of clinical judgment in respect of the care of a particular patient, or other professional judgments in any decision-making process. You assume full responsibility for the use of the information and rely on it at your own risk.

CDA-AMC does not endorse any information, drugs, therapies, treatments, products, processes, or services. The views and opinions of third parties published in this document do not necessarily reflect those of CDA-AMC. The copyright and other intellectual property rights in this document are owned by the Canadian Agency for Drugs and Technologies in Health (operating as CDA-AMC) and its licensors.

Questions or requests for information about this report can be directed to Requests@CDA-AMC.ca.