CADTH Health Technology Review

Evidence Base for Virtual Primary Care

Technology Review

Authors: Chantelle C. Lachance, Melissa Severn, Joanne Kim

Key Messages

• The aim of this report was to identify and organize evidence and summarize key findings on the clinical effectiveness, cost-effectiveness, patient and provider experiences and preferences, and equity considerations of virtual primary care. It includes 60 relevant reports, of which we summarized 6 systematic reviews (SRs) and organized 54 additional reports into annotated reference lists.

• This report was not a formal evidence synthesis, and no conclusions were formed about the value of virtual primary care. The literature was not critically appraised, and the quality of the evidence from the included reports is unknown.

• The evidence about the clinical effectiveness of virtual primary care summarized in this report comes from a limited number of studies identified by 3 SRs and was mainly focused on health care utilization outcomes versus patient health or clinical harms outcomes. The findings were mixed, but overall, in-person visits may be associated with lower rates of health care utilization (e.g., emergency department visits, hospitalizations) compared with virtual visits. Virtual consultations may be associated with improved patient engagement compared with in-person care for certain conditions or settings (e.g., opioid addictions) but not others (e.g., smoking cessation, postnatal care, other general settings).

• The evidence about the cost-effectiveness of virtual primary care summarized in this report comes from 3 studies identified by 1 SR. Virtual primary care compared with in-person care might be cost-effective for patients with asthma and patients who received cognitive behavioural therapy, but these analyses considered very limited outcomes.

• The evidence about patient and provider experiences and preferences associated with virtual primary care that is summarized in this report comes from 4 SRs. The SRs identified many perceived benefits and drawbacks of virtual primary care from both patient and provider perspectives, including those around access to care, effectiveness, experience and satisfaction, and financial impact and cost.

• The evidence about potential equity issues associated with virtual primary care summarized in this report comes from 2 SRs. Equity considerations, such as age, may affect individuals accessing and using primary care health services. For example, telephone consultations tended to be used by older individuals (aged ≥ 85 years), while internet-based consultations tended to be used by younger individuals.

Context

Access to health care has been identified as an important social determinant of health and a key driver of health outcomes.^1,2 Virtual care may help to address barriers to accessing primary health care services faced by individuals, including those with unique needs or circumstances (e.g., underserved communities, individuals living in rural and/or remote areas, individuals who do not drive), but it also has the potential to exacerbate existing inequities in access to care experienced by certain groups (e.g., individuals without access to reliable phone or internet services),³ which warrants taking a closer look at all of its effects.

For this report, we considered virtual care as any interaction between patients (or members of their circle of care) and providers, occurring remotely, using any forms of communication or information technologies, with the aim of facilitating or maximizing the quality and effectiveness of patient care.⁴ This definition includes communication that may occur synchronously (real-time communication such as telephone calls and videoconferencing) or asynchronously (communication not in real time such as text or email messaging).⁴

In 2022, CADTH published Virtual Care Rapid Scoping⁵ for which an adapted version of the scoping review methodology was used to quickly identify the existing evidence and guidance on any virtual health care for all types of health care services and in all populations. Given its broad topic, the report was limited to select literature but still identified and described 230 scoping reviews with relevant evidence and 11 evidence-based guidelines with relevant recommendations.⁵ Consistent with the scoping review methodology, the Rapid Scoping report provided the breadth but not the depth of information available on the topic and did not provide a synthesis of findings.^5,6 Moreover, the results were organized at a high level mainly by health condition or outcome rather than by health care setting (e.g., primary versus specialist care).⁵ Primary care is distinct from specialized or acute care and the serviceability and acceptability of virtual care in this space may be different from other health care settings. Challenges within the primary care setting include inadequate continuity of care, poor access in remote and rural areas, low access in urban areas that do not have after-hours options, and demanding working conditions for providers.⁷ It is possible that the existing evidence regarding virtual care varies by health care setting, and a more focused literature search about virtual primary care will help to determine the quantity of evidence available for that specific setting.

The objective of this Health Technology Review is to identify and organize evidence and summarize key findings on the clinical effectiveness, cost-effectiveness, patient and provider experiences and preferences, and equity considerations associated with virtual primary care. This report will help our understanding of the amount of evidence available within the virtual primary care space and, importantly, the types of evidence that has been assessed along with key findings. This report does not include a formal critical appraisal of the literature; thus, making recommendations or conclusions about the value of virtual primary care is outside the scope of this report.

Research Questions

1. What literature describes clinical effectiveness of virtual care for individuals accessing primary care services?

2. What literature describes cost-effectiveness of virtual care for individuals accessing primary care services?

3. What literature describes patient and/or provider experiences or preferences associated with virtual care for individuals accessing primary care services?

4. What literature describes equity issues associated with virtual care for individuals accessing primary care services?

Methods

Literature Search Methods

An information specialist conducted a literature search on key resources including MEDLINE, the Cochrane Database of Systematic Reviews, the International HTA Database, the websites of Canadian and major international health technology agencies, as well as a focused internet search. The search approach was customized to retrieve a limited set of results, balancing comprehensiveness with relevancy. The search strategy comprised both controlled vocabulary, such as the National Library of Medicine’s MeSH (Medical Subject Headings), and keywords. Search concepts were developed based on the elements of the research questions and selection criteria. The main search concepts were virtual services and primary care providers. CADTH-developed search filters were applied to limit retrieval to health technology assessments, SRs, meta-analyses, any types of clinical trials or observational studies, and economic studies. The search was completed on April 6, 2023, and limited to English-language documents published since January 1, 2020.

Selection Criteria and Methods

One reviewer screened literature search results and selected publications according to the inclusion criteria presented in Table 1, first based on titles and abstracts (for all publications) and then based on full texts (for SRs only) because this report aimed to summarize select SRs and organize the other articles in annotated reference lists.

Table 1: Selection Criteria

ICER = incremental cost-effectiveness ratio; QALY = quality-adjusted life-year.

Exclusion Criteria

We excluded articles if they did not meet the selection criteria outlined in Table 1, were duplicate publications, or were published before 2020. If articles reported on topics broader than our selection criteria (e.g., included both primary care and specialist care), we included the articles and summarized the relevant content only (e.g., on primary care and not specialist care). We also excluded guidelines, guidance, and policy documents.

Summary of Evidence

Quantity of Research Available

Appendix 1 presents the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)⁸ flow chart of the study selection.

This report identified 60 relevant reports, some of which addressed multiple research questions. We found:

• 16 SRs and 22 nonrandomized studies that addressed research question 1 about the clinical effectiveness of virtual primary care

• 1 SR that addressed research question 2 regarding the cost-effectiveness of virtual primary care

• 9 SRs and 2 nonrandomized studies that addressed research question 3 about patient and/or provider experiences and preferences associated with virtual primary care

• 3 SRs and 21 nonrandomized studies that addressed research question 4 concerning equity issues associated with virtual primary care.

We did not identify any relevant health technology assessments or randomized controlled trials.

Of the 60 relevant reports, 6 SRs^9-14 were selected for our summary of evidence; their key findings, based on the full texts, are described by research question in the subsequent sections. These SRs were chosen because they comprehensively searched for evidence generally about virtual primary care (i.e., research questions did not focus on a specific patient population or health condition) with the intention of understanding the overall evidence about virtual primary care. These SRs came from well-known SR organizations or research groups (e.g., Agency for Healthcare Research and Quality government agency) and used rigorous methods in their review (e.g., systematic literature searching, critical appraisal of included studies). Thus, the decision to summarize select SRs versus other study designs was intentional based on the hierarchy of evidence because the literature included in this report was not critically appraised. The remaining 54 reports, included and organized by research question and by study design based on their titles and abstracts, are provided in Appendix 3. Appendix 4 provides additional references of potential interest that did not meet the inclusion criteria.

Clinical Effectiveness of Virtual Primary Care

In this section, we describe 3 SRs^11-13 that addressed the clinical effectiveness of virtual primary care. Additional details about these publications are summarized in Table 2 in Appendix 2.

Health Care Utilization

Three SRs^11-13 summarized relevant studies on health care utilization outcomes, including rates of emergency department visits, rates of hospitalizations, rates of lab and imaging orders, rates and duration of follow-up visits, as well as rates of patient attrition or engagement.

The SR by Hatef et al. (2023)¹² examined the clinical effectiveness of telehealth during the COVID-19 pandemic for any health care setting. This SR identified 2 primary studies comparing in-person versus telehealth visits for individuals with COVID-19 accessing primary care services. Results of the SR included the following:

• There were lower emergency department visit rates following in-person versus telehealth visits, with a statistically significant difference between the 2 groups (1 study).

• There were lower emergency department rates with the telehealth group versus the in-person group, but this finding was not statistically significant (1 study).

For hospitalization rates, 3 primary studies identified in the Hatef et al. (2023)¹² SR had mixed results:

• There were lower hospitalization rates with the in-person group versus the telehealth group (2 studies), with 1 study reporting a statistically significant difference and the other study reporting no statistical test values.

• There were lower hospitalization rates with the telehealth group versus the in-person group, but this finding was not statistically significant (1 study).

The Carrillo de Albornoz et al. (2021)¹¹ SR examined hospital readmission rates for individuals who received nutrition counselling in-person or over the telephone. The authors reported no statistically significant differences in hospital readmission rates between telephone and in-person visits for nutrition counselling at 30 days or 90 days after discharge (1 study).

The SR by Hatef et al. (2023)¹² included 1 primary study that examined lab and imaging orders and reported similar rates of orders placed between the 2 groups (i.e., telehealth visit group versus in-person visit group); this study did not report the completion rates of the orders placed for, or the difference between, the 2 groups. Additionally, there was 1 primary study from the Carrillo de Albornoz et al. (2021)¹¹ SR that reported on health care utilization outcomes more generally. This study found video consultations reduced health care utilization, including lab tests and imaging, compared with in-person visits.

Two SRs^11,12 found evidence from 3 primary studies regarding follow-up visits. Hatef et al. (2023)¹² found higher rates of follow-up visits among those who had an initial telehealth visit (via telephone or video) versus those who received an in-person visit (1 study). Carrillo de Albornoz et al. (2021)¹¹ described different results: 1 primary study found a statistically significant difference with a higher number of teleconsultations for postnatal care in postpartum patients using telephone versus in-person visits, and 1 primary study found a statistically significant difference with videoconferencing compared to in-person visits showing reductions in the number and duration of speech therapy appointments.

Two SRs^11,13 found evidence from 5 primary studies regarding patient attrition and engagement. Carrillo de Albornoz et al. (2021)¹¹ found telehealth had lower adherence (e.g., fewer sessions attended) and higher rates of discontinuation (e.g., more likely to discontinue smoking cessation) (4 studies). In contrast, Parker et al. (2021)¹³ SR found telehealth improved engagement with primary care compared to in-person care for patients with opioid addictions (1 study).

Quality of Care

For quality of care, the Carrillo de Albornoz et al. (2021)¹¹ SR found video teleconsultations had statistically significant, but not clinically meaningful, improvements in some quality-of-care measures (e.g., antibiotic use, antibiotic management), but in-person visits performed better on other quality measures (e.g., more appropriate testing, fewer follow-up visits) (1 study).

Change in Therapy or Medication

Hatef et al. (2023)¹² found higher rates regarding change in medication for in-person visits versus telehealth visits, which was indicated by the number of consultations with at least 1 prescribed medication (1 study).

General and Condition-Specific Patient-Focused Outcomes

The Carrillo de Albornoz et al. (2021)¹¹ SR examined patient-focused outcomes and found:

• similar accuracy of patient’s recall (for single and multiple problem consultations) between in-person and telehealth visits and significantly more repetition of advice given to enhance patient recall for in-person versus telehealth visits (1 study)

• mixed results for continuous abstinence rates after counselling on smoking cessation (2 studies)

• no statistically significant differences between telephone and in-person visits for counselling for weight loss (1 study).

Clinical Harms

The 3 SRs^11-13 that addressed the clinical effectiveness of virtual primary care would have included harms outcomes in their SR (e.g., clinical complications, patient safety, or adverse events); however, no harms data were reported from the relevant included studies. Of note, it is uncertain whether outcomes described previously (e.g., emergency department visits, hospitalizations, follow-up visits) are markers of complications. Some perceived clinical safety considerations by both patients and providers are also described in the Patient and Provider Experiences and Preferences of Virtual Primary Care section.

Cost-Effectiveness of Virtual Primary Care

We highlight 1 SR that described cost-effectiveness implications associated with virtual primary care. Additional details about these publications are summarized in Table 3.

The De Guzman et al. (2021)¹⁴ SR included 3 relevant studies for this report. These studies conducted cost-minimization analyses with effectiveness also considered (2 studies) or cost-effectiveness analyses (1 study) to examine cost-effectiveness implications of virtual primary care. Overall, telehealth was reported to be cost-effective compared with in-person care, with very limited effect measures considered in the analyses:

• Telephone asthma consultations were considered cost-effective compared with in-person consultations, from the health system perspective, because of reduced costs and reduced consultation times (2 studies).

• Telephone cognitive behavioural therapy was cost-effective versus with in-person cognitive behavioural therapy from a partial societal perspective (undefined) due to reduced costs and reduced mental health service use, patient travel, and patient waiting time (1 study).

Patient and Provider Experiences and Preferences of Virtual Primary Care

Four SRs^9-12 describing patient and/or provider experiences and preferences associated with virtual primary care are highlighted in this section. Additional details about these publications are summarized in Table 4.

Patient Experiences and Preferences

The SR by Hatef et al. (2023)¹² examined what was considered a successful telehealth intervention during the COVID-19 pandemic, which included reports of patient and physician experiences. Specifically, this review provided the numbers of qualitative studies and quantitative studies (i.e., using surveys) that addressed patient satisfaction and dissatisfaction. In the primary care setting, 3 themes of patient satisfaction were identified but without any details or examples about them provided: privacy, communication, and outcomes. This SR also included findings about patient dissatisfaction, but none were from studies relevant for this report.¹²

Carrillo de Albornoz et al. (2021)¹¹ found comparable levels of parental satisfaction with care using telehealth versus an in-person visits for postpartum patients who received postnatal care (1 study).

Ward et al. (2022)⁹ conducted an SR that included studies that reported benefits and drawbacks of using telehealth in primary care from patient and health care provider perspectives. For the purposes of their review, the findings were not organized according to patient versus provider perspectives but rather organized according to specific primary care visit types: chronic disease management, existing patients, medication management, new patients, mental health and behavioural management, post-test result follow-up, and postdischarge follow-up. However, Ward et al.⁹ provided detailed supplementary files that distinguished findings according to patient perspectives, primary care clinician perspectives, or both. Using the supplementary files, we extracted and organized key data that are presented in Appendix 2 (Table 4). Given the large amount of detail provided in our tables from the supplementary files, we provide a high-level summary of Ward et al.’s findings in this section. Ward et al. included studies that found benefits and drawbacks from patients’ and clinicians’ perspectives for the following categories: access to care, effectiveness, experience, and financial cost or impact. Key benefits of telehealth included: focused discussions, continuity of care despite practices of social distancing practices during COVID-19, and improved convenience. Key drawbacks of telehealth included: impersonal interactions, technical barriers, and semiestablished reimbursement models for providers.

Provider Experiences and Preferences

In addition to patient experiences, Hatef et al. (2023)¹² also examined health care provider experiences. In the primary care setting, 6 themes of provider satisfaction were identified but without any details or examples provided: ease of use, communication, outcomes, access, benefits, and suggestions. Five themes of provider dissatisfaction were described (details for each theme not provided): ease of use, communication, outcomes, preferences, and concerns. In addition to provider satisfaction and dissatisfaction, this SR also provided counts for qualitative and quantitative studies identifying facilitators and barriers from provider perspectives. Three facilitators were mentioned in the primary care context, also without any details or examples about them provided: telehealth literacy of patients, outcomes, and appropriateness of fit. Six barriers were mentioned in the primary care context: cost, privacy (in the context of maintaining confidentiality), communication, inequity, telehealth literacy of patients, and technical issues and digital literacy experienced by providers and their staff.

The Hoff and Lee (2022)¹⁰ SR looked at satisfaction of primary care physicians using telemedicine for virtual follow-up care for patients with chronic illness. The findings suggest that physicians are satisfied by the use of telemedicine and also noted that more research is needed for the primary care setting (1 study).¹⁰

The Ward et al. (2022)⁹ SR also examined primary care clinician perspectives. The key data are provided in Appendix 2 (Table 4) and in the Patient Experiences and Preferences section.

Equity Issues of Virtual Primary Care

In this section, we describe 2 SRs^11,13 that included studies on potential equity issues associated with virtual primary care. Additional details about these publications are summarized in Table 5.

Carrillo de Albornoz et al. (2021)¹¹ looked at individuals accessing primary care services and found that videoconference visits were more likely to be used by younger patients and younger physicians, with no differences by sex, between rural and urban settings, or by socioeconomic gradient (1 study).¹¹

The SR led by Parker et al. (2021)¹³ included 13 studies that explored the impact of telephone and internet-based consultations versus in-person consultations for various socioeconomic groups. Telephone consultations tended to be used by older individuals (aged ≥ 85 years), individuals from urban areas, and nonimmigrants. Internet-based consultations tended to be used by younger individuals. Compared with men, women used more remote forms of consultations than in-person consultations. The authors of the SR reported mixed findings when considering socioeconomic factors and ethnicity.

Limitations

This report was not a formal evidence synthesis, and no conclusions were formed about the value of virtual primary care. Except for the 6 SRs, which were summarized from their full texts, all evidence was organized and annotated according to the publication’s title and abstract. Due to this format, we likely missed details presented in the full texts, and it was not possible to exclude primary studies if they were captured by 1 or more of the included SRs. Similarly, we did not assess the overlap in primary studies in the included SRs, so it is possible that certain SRs included some of the same primary studies. Without knowing the extent of the overlap, we may be overestimating the amount of relevant literature available. The literature was not critically appraised, and the quality of the evidence from the included reports is unknown. For all research questions, no randomized controlled trials met the inclusion criteria for this report; the included SRs also contained studies from various study designs (i.e., they were not limited to randomized studies), and their findings should be interpreted with caution and as to not inferring causality. For the 6 SRs selected for the summary of evidence, we had to rely on the findings and data reported by the SR authors. When summarizing the relevant findings, we noticed missing details (e.g., unspecified health care setting or not specific to primary care, study design and analysis details of included studies not reported, rationale for including very few effect measures in cost-effectiveness analyses not provided), and the statistical tests and values reported were sometimes difficult to assess or interpret; thus, we need to take these findings with caution. Although several studies were included to answer the clinical effectiveness question, the studies found evidence primarily about health care utilization outcomes, and there was significantly less evidence about patient health or clinical harms outcomes. This report found little evidence regarding cost-effectiveness implications.

Conclusions

This report identified and described the literature on the clinical effectiveness (16 SRs and 22 nonrandomized studies), cost-effectiveness (1 SR), patient and provider experiences and preferences (9 SRs and 2 nonrandomized studies), and equity considerations (3 SRs and 22 nonrandomized studies) of virtual primary care.

This report is an extension of CADTH’s previous report, Virtual Care Rapid Scoping,⁵ but focuses on primary care. We found many studies that examined clinical effectiveness, patient and provider experiences and preferences, and equity considerations, but few studies that examined cost-effectiveness. We took the evidence identified in this report 1 step further by summarizing the findings at a high level, either as a part of the main summary of evidence (6 SRs) or in annotated reference lists (54 remaining articles).

The summarized evidence on clinical effectiveness from a limited number of studies identified by 3 SRs which was mainly focused on health care utilization. There was evidence suggesting in-person visits compared to telehealth may be associated with lower rates of health care utilization related to emergency department visits and hospitalizations. However, it is unclear whether virtual primary care leads to more follow-up visits or better quality of care as findings were variable from the included studies with insufficient information to draw any causal inferences. There was some evidence to suggest virtual primary care does not increase the number of lab and imaging orders placed. Findings on patient engagement were mixed. Because we selected for our summary of evidence SRs that searched for evidence generally about virtual primary care, it is understandable that the key findings of these SRs, and consequently those described in this report, were general clinical outcomes, such as health care utilization outcomes. Some studies listed in our annotated reference lists were on specific conditions (e.g., diabetes) and included more disease-specific, morbidity- or mortality-related, clinical outcomes (e.g., glycemic control). A formal synthesis of the evidence from the 3 SRs as well as from the annotated reference lists is needed to better understand the clinical effectiveness of virtual primary care on all clinical outcomes, including patient health and clinical harms outcomes.

The findings related to cost-effectiveness that we summarized from 1 SR suggest that virtual care compared with in-person care may be cost-effective in the primary care setting, costing less while improving some outcomes. However, this was based on 3 analyses that considered very limited effect measures, such as consultation times. Compared with the other research questions, we found evidence regarding potential cost-effectiveness implications of virtual care within the primary care setting to be less prevalent. Further research is needed to determine the cost-effectiveness of virtual primary care using comprehensive effect measures (e.g., quality-adjusted life-years).

The summarized evidence related to patient and provider experiences and preferences identified from 4 SRs found several perceived benefits and drawbacks of virtual primary care, from both patient and provider perspectives. Specifically, 2 SRs^9,12 summarized a large amount of evidence on this topic. A formal qualitative synthesis of the evidence is needed to adequately synthesize the findings from these SRs and the additional evidence identified from the annotated reference lists.

We described 2 SRs^11,13 that included studies on potential equity issues associated with virtual primary care. The findings from the SRs suggest equity issues related to several demographic and socioeconomic factors may be present in the context of accessing virtual care, but the evidence was from a limited number of studies from 2 SRs with a lack of data richness or in-depth analysis. A formal synthesis of the evidence also from the annotated reference lists is warranted to explore potential equity issues that may result in different quality or access to care when using virtual primary care services.

References

1.National Collaborating Centre for Indigenous Health. Access to health services as a social determinant of First Nations, Inuit and Métis health. 2019; https://www.nccih.ca/docs/determinants/FS-AccessHealthServicesSDOH-2019-EN.pdf. Accessed 2023 Apr 13.

2.McGibbon E, Etowa J, McPherson C. Health-care access as a social determinant of health. Canadian Nurse. 2008;104(7). PubMed

3.Ogunyemi B, Katyukha A. Canadian Doctors for Medicare Position Statement on Virtual Care. 2022; https://assets.nationbuilder.com/cdm/pages/183/attachments/original/1659726295/CDM_Position_Statement_on_Virtual_Care.pdf?1659726295. Accessed 2023 Apr 13.

4.Jamieson T, Wallace R, Armstrong K, et al. Virtual care: a framework for a patient-centric system. Toronto: Women’s College Hospital Institute for Health Systems Solutions and Virtual Care (WIHV). 2015: https://documents.pub/document/virtual-care-a-framework-for-a-patient-centric-care-a-framework-for-a-patient-centric.html?page=2. Accessed 2023 Apr 26.

5.Lachance CC, Darvesh N, McGill SC, Butcher R. Virtual Care Rapid Scoping: Main Report. CADTH Health Technology Review. 2022; https://canjhealthtechnol.ca/index.php/cjht/article/view/RE0041. Accessed 2023 Apr 13.

6.Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International journal of social research methodology. 2005;8(1):19-32.

7.Government of Canada. About primary health care 2012: https://www.canada.ca/en/health-canada/services/primary-health-care/about-primary-health-care.html. Accessed 2023 Apr 14.

8.Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151(4):W-65-W-94. PubMed

9.Ward K, Vagholkar S, Sakur F, Khatri NN, Lau AYS. Visit Types in Primary Care With Telehealth Use During the COVID-19 Pandemic: Systematic Review. JMIR Med Inform. 2022;10(11):e40469. PubMed

10.Hoff T, Lee DR. Physician Satisfaction With Telehealth: A Systematic Review and Agenda for Future Research. Qual Manag Health Care. 2022;31(3):160-169. PubMed

11.Carrillo de Albornoz S, Sia KL, Harris A. The effectiveness of teleconsultations in primary care: systematic review. Fam Pract. 2022;39(1):168-182. PubMed

12.Hatef E, Wilson RF, Hannum SM, Zhang A, et al. Use of Telehealth During the COVID-19 Era. Systematic Review. AHRQ Publication No. 23-EHC005. Rockville (MD): Agency for Healthcare Research and Quality; Jan 2023: https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/use-telehealth-during-COVID-19-systematic-review.pdf. Accessed 2023 Apr 19.

13.Parker RF, Figures EL, Paddison CA, Matheson JI, Blane DN, Ford JA. Inequalities in general practice remote consultations: a systematic review. BJGP open. 2021;5(3). PubMed

14.De Guzman KR, Snoswell CL, Caffery LJ, Smith AC. Economic evaluations of videoconference and telephone consultations in primary care: A systematic review. J Telemed Telecare. 2021:1357633X211043380.

Appendix 1: Selection of Included Studies

Note that this appendix has not been copy-edited.

Figure 1: Selection of Included Studies

Appendix 3: Annotated Bibliography

Note that this appendix has not been copy-edited.

In addition to the 6 SRs selected for our summary of evidence, 54 additional publications met the eligibility criteria for this report. Each of the following subsections contains an annotated bibliography, organized by research question followed by study design. After each citation, we provide an annotated description about the publication and its findings based on the title and abstract of the article. Further details can be found by consulting the full texts, which we linked in the references where applicable.

References Addressing Research Question 1: Clinical Effectiveness

Health Technology Assessments

No literature identified.

Systematic Reviews

Cantor A, Nelson HD, Pappas M, Atchison C, Hatch B, Huguet N, Flynn B, McDonagh M. Effectiveness of Telehealth for Women’s Preventive Services. Comparative Effectiveness Review No. 256. (Prepared by the Pacific Northwest Evidence-based Practice iii Center under Contract No. 75Q80120D00006.) AHRQ Publication No. 22-EHC024. Rockville, MD: Agency for Healthcare Research and Quality; June 2022. PubMed
This SR examined the effects of telehealth for women, specifically preventive services for reproductive health care and interpersonal violence. Authors examined several clinical effectiveness outcomes. This SR reported no differences in the rates of pregnancy, contraceptive use, and sexually transmitted infection between groups (i.e., telehealth versus comparator interventions to supplement contraceptive care). There was insufficient evidence to assess differences in abortion rates. There were no differences between telehealth and comparators in interpersonal violence services on outcomes measuring repeat interpersonal violence services, fear of partner, safety behaviours, self-efficacy, coercive control, depression, and posttraumatic stress disorder. There was insufficient evidence to evaluate harms outcomes.

Emonena H, Ojo O. The Efficacy of Tele-Monitoring in Maintaining Glycated Haemoglobin Levels in Patients with Type 2 Diabetes Mellitus: A Systematic Review. Int J Environ Res Public Health. 2022;19(24):13. PubMed
This SR evaluated the effect of telemedicine on the glycemic control of people with type 2 diabetes. Of the 6 studies, half reported a statistically significant improvement in hemoglobin A1C levels in the telemedicine group compared with the control group.

Greenwood H, Krzyzaniak N, Peiris R, et al. Telehealth Versus In-person Psychotherapy for Less Common Mental Health Conditions: Systematic Review and Meta-analysis of Randomized Controlled Trials. JMIR Ment Health. 2022;9(3):e31780. PubMed
Authors of this SR compared the effect of telehealth to in-person delivery of psychotherapy for people with mental and physical health conditions. Results of a meta-analysis found no statistically significant differences between the 2 groups in symptom severity, overall improvement, function, “working alliance client” [from original source], and “working alliance therapist” [from original source].

Hubschman-Shahar LE. Lactation Telehealth in Primary Care: A Systematic Review. Breastfeed Med. 2022;17(1):6-21. PubMed
This SR assessed the impact of receiving skilled lactation care through telehealth compared with routine care on breastfeeding outcomes. Authors observed associations between lactation telehealth and breastfeeding outcomes, including increases in any and exclusive breastfeeding rates and decreases in early breastfeeding cessation after lactation care.

Kirakalaprathapan A, Oremus M. Efficacy of telehealth in integrated chronic disease management for older, multimorbid adults with heart failure: A systematic review. Int J Med Inf. 2022;162:104756. PubMed
The focus of this SR was to assess the effect of integrated telehealth versus other strategies for the management of chronic diseases in older adults with heart failure and multiple comorbidities in primary care and community settings. Authors reported a reduction in cardiovascular-related hospitalizations, rehospitalizations, and mortality in the integrated telehealth group compared to the control group. However, authors of the SR cautioned the numerous discrepancies between studies, and the studies had a high risk of bias.

Mabeza RMS, Maynard K, Tarn DM. Influence of synchronous primary care telemedicine versus in-person visits on diabetes, hypertension, and hyperlipidemia outcomes: a systematic review. BMC Prim Care. 2022;23(1):52. PubMed
This SR compared the effectiveness of synchronous telemedicine to in-person visits with primary care clinicians for chronic disease management. Telemedicine was associated with greater reductions in hemoglobin A1C levels at 6 months and similar hemoglobin A1C levels at 12 months when compared to in-person visits. Authors did not find statistically significant differences between the 2 groups for blood pressure and lipid levels.

Scott AM, Bakhit M, Greenwood H, et al. Real-Time Telehealth Versus In-person Management for Patients With PTSD in Primary Care: A Systematic Review and Meta-Analysis. J Clin Psychiatry. 2022;83(4):23. PubMed
This meta-analysis compared synchronous telehealth (phone, video) to in-person therapy delivery to patients with posttraumatic stress disorder. This review reported no differences between groups regarding posttraumatic stress disorder severity, depression severity, or therapeutic alliance. One trial reported similar changes in quality-of-life outcomes when comparing telehealth to in-person.

Whitfield J, LePoire E, Stanczyk B, Ratzliff A, Cerimele JM. Remote Collaborative Care With Off-Site Behavioral Health Care Managers: A Systematic Review of Clinical Trials. J Acad Consult Liaison Psychiatry. 2022;63(1):71-85 PubMed
The aim of this SR was to assess the evidence regarding the impact of remote collaborative care model teams for people with mental health conditions in primary care and medical settings. Remote collaborative care was effective in treating a range of behavioural health conditions (including depression, anxiety, and posttraumatic stress disorder across various populations) and settings (including primary care) compared to control interventions.

Zhang A, Wang J, Wan X, et al. A Meta-Analysis of the Effectiveness of Telemedicine in Glycemic Management among Patients with Type 2 Diabetes in Primary Care. Int J Environ Res Public Health. 2022;19(7):31 PubMed
In this SR, authors assessed the effectiveness of telemedicine interventions for people with type 2 diabetes within the primary care setting. Authors reported reduced hemoglobin A1C, fasting glucose, and postprandial glucose levels after the telemedicine intervention compared with non-telemedicine care. Authors also reported that a statistically significant improvement in systolic blood pressure and self-efficacy was observed after the telehealth intervention compared to non-telemedicine care. However, authors did not observe statistically significant differences between the 2 groups for weight, lipid metabolism, or diabetes awareness.

Bakhit M, Baillie E, Krzyzaniak N, et al. Antibiotic prescribing for acute infections in synchronous telehealth consultations: a systematic review and meta-analysis. BJGP open. 2021;5(6). PubMed
This SR assessed the effect of telehealth consultations versus in-person on antibiotic prescribing in primary care for people with a history of community-acquired acute infection. The impact of telehealth on prescribing appears to vary between conditions (e.g., acute rhinosinusitis versus otitis media). Authors noted there was not enough evidence to make strong conclusions and that many of the studies did not control well for confounding variables and were at high risk of bias.

Robson N, Hosseinzadeh H. Impact of Telehealth Care among Adults Living with Type 2 Diabetes in Primary Care: A Systematic Review and Meta-Analysis of Randomised Controlled Trials. Int J Environ Res Public Health. 2021;18(22):19. PubMed
This SR evaluated the impact of telehealth intervention versus usual care for people with type 2 diabetes. The results of the meta-analysis suggest that telehealth interventions have a greater effect on reducing hemoglobin A1C levels compared to usual care.

Han SM, Greenfield G, Majeed A, Hayhoe B. Impact of Remote Consultations on Antibiotic Prescribing in Primary Health Care: Systematic Review. J Med Internet Res. 2020;22(11):e23482. PubMed
This SR summarized the impact of remote consultations on antibiotic prescribing rates for patients receiving primary care. Of the 12 studies identified, remote consultations resulted in lower prescribing rates for 5 studies, similar prescribing rates in 3 studies, and higher prescribing rates in 4 studies when compared to in-person consultations. Additionally, authors did not observe a statistically significant difference in guideline-concordant prescribing between remote and in-person consultations.

Kraef C, van der Meirschen M, Free C. Digital telemedicine interventions for patients with multimorbidity: a systematic review and meta-analysis. BMJ Open. 2020;10(10):e036904. PubMed
This SR included prospective interventional studies reporting patients with 2 or more chronic conditions receiving interventions with at least 1 digital telemedicine component. Results of a meta-analysis demonstrated that interventions with a digital telemedicine component linked to usual care were associated with a moderate decrease in systolic blood pressure, a small to moderate decrease in hemoglobin, and a moderate decrease in total cholesterol. For studies with no links to usual care, authors observed a small reduction in median time to hospital and health-related quality of life. No statistically significant differences were reported between the 2 groups for patient-reported depression.

Randomized Controlled Trials

No literature identified.

Nonrandomized Studies

Chami N, Shah HA, Nastos S, et al. Association between virtual primary care and emergency department use during the first year of the COVID-19 pandemic in Ontario, Canada. Cmaj. 2023 Jan 23;195(3):E108-E114. PubMed
In this observational study, the authors used administrative data from Ontario, Canada to evaluate whether a shift from in-person to virtual visits by primary care physicians is associated with an increased use of emergency department visits. After controlling for covariates, study authors did not observe a statistically significant change in emergency department visits per 1% increase in the proportion of virtual visits in the context of decreased availability of in-person care with their family physician during the first year of the COVID-19 pandemic.

Chen K, Zhang C, Gurley A, Akkem S, Jackson H. Appointment Non-attendance for Telehealth Versus In-Person Primary Care Visits at a Large Public Healthcare System. J Gen Intern Med. 2023 03;38(4):922-928. PubMed
In this observational study, the authors used electronic health records data to compare primary care nonattendance for telehealth versus in-person visits. Initially, nonattendance for telephone visits was higher versus in-person visits in the “early telehealth availability period” but decreased in later periods. Conversely, nonattendance for video visits was higher versus in-person visits during the early and later periods.

Dahlgren C, Spanberg E, Svereus S, Dackehag M, Wandell P, Rehnberg C. Short- and intermediate-term impact of DTC telemedicine consultations on subsequent healthcare consumption. Eur J Health Econ. 2023 Feb 24;24:24. PubMed
This interrupted time series used a constructed database with individual-level data to assess the short- and intermediate-term impact of direct-to-consumer telemedicine consultations on subsequent primary health care consumption in Sweden. The authors compared telemedicine users to in-person users. Telemedicine users increased their health care consumption more than in-person users, and this effect was mostly short-term (within 1 month) and immediate term (2 to 6 months after initial consultation). Authors described the results as being robust across age and disease groups.

Naja S, Alchawa M, Adli N, et al. Telephone versus in-person consultation on smoking cessation at primary health care centers in Qatar during COVID-19 pandemic. Prev Med Rep. 2023 Jun;33:102174. PubMed
This retrospective cohort study evaluated telephone versus in-person smoking cessation counselling within primary care. Compared to individuals who attended in-person visits, individuals who attended telephone consultations had significantly lower smoking cessation rates at all follow-ups.

Baughman D, Ptasinski A, Baughman K, Buckwalter N, Jabbarpour Y, Waheed A. Comparable Quality Performance of Acute Low-Back Pain Care in Telemedicine and Office-Based Cohorts. Telemed J E Health. 2022 Mar 28;28:28. PubMed
This retrospective cohort study evaluated the use of low back pain imaging between telemedicine and in-person clinical encounters. Compared to in-person visits, patients seen through telemedicine had significantly lower rates of imaging within 28 days of their physician encounter, and this was consistent across racial, ethnic, and risk score subgroups.

Baughman DJ, Jabbarpour Y, Westfall JM, et al. Comparison of Quality Performance Measures for Patients Receiving In-Person vs Telemedicine Primary Care in a Large Integrated Health System. JAMA netw. 2022 09 01;5(9):e2233267. PubMed
This retrospective cohort study examined whether quality of care among patients exposed to telemedicine differs from patients with only in-person care. Compared to patients with only in-person visits, patients with telemedicine exposure were associated with significantly better performance or no difference in several outcomes associated with quality of primary care.

Chavez A, Pullins C, Yennie J, Singh C, Kosiorek H. Association of Telehealth with Short-Interval Follow-Up. J Am Board Fam Med. 2022 May-Jun;35(3):485-490. PubMed
This observational cohort study examined the association of a telehealth program with a short-interval follow-up within an academic primary care practice during the COVID-19 pandemic. This study found that telehealth visits were associated with significantly higher rates of total short-interval follow-ups and higher rates of first short-interval follow-ups occurring in 0 to 15 days when compared to in-person visits.

Graetz I, Huang J, Muelly E, Gopalan A, Reed ME. Primary Care Visits Are Timelier When Patients Choose Telemedicine: A Cross-Sectional Observational Study. Telemed J E Health. 2022 09;28(9):1374-1378. PubMed
This cross-sectional observational study wanted to determine if there was an association between the type of primary care visit (telephone versus video versus in-person) and timeliness of appointment. The results found comparable scheduling availability between telemedicine (telephone or video) and in-person, and telemedicine was associated with more timely access to care.

Grauer A, Duran AT, Liyanage-Don NA, et al. Association between telemedicine use and diabetes risk factor assessment and control in a primary care network. J Endocrinol Invest. 2022 Sep;45(9):1749-1756. PubMed
This retrospective, 1:1 propensity score–matched cohort study examined whether telemedicine use was associated with diabetes risk factor assessment and control within the primary care setting. Compared with patients who had in-person visits only, telemedicine use was associated with a lower proportion of patients with all 3 risk factors assessed: hemoglobin A1C, low-density lipoprotein cholesterol, and blood pressure. Telemedicine use did not impact risk factor control when individual risk factors were assessed.

Kannenberg B, Stadter G. Analysis and Observations of Telehealth in Primary Care Follow Up Appointments for Vulnerable Populations. Wmj. 2022 Jul;121(2):116-120. PubMed
One of the main objectives of this cross-sectional study was to examine the effect of telehealth on the attendance of initial follow-up appointments at a primary care clinic following hospital discharge. Telehealth appointments were associated with better attendance versus in-person appointments after controlling for confounding factors.

Shakeri A, Chu C, Stamenova V, et al. Comparison of Healthcare Utilization Between Telemedicine and Standard Care: A Propensity-Score Matched Cohort Study Among Individuals With Chronic Psychotic Disorders in Ontario, Canada. Schizophr Bull Open. 2022 Jan;3(1):sgac046. PubMed
This retrospective propensity-matched cohort study used health care administrative data in Ontario, Canada to see the impact of telemedicine use on changes in health care utilization for individuals with chronic psychotic disorders. For emergency department visits due to any psychiatric conditions, authors found a significantly greater decline across time in the telemedicine group versus usual care. There were steeper declines in primary care visit rates, mental health outpatient visits with primary care, and all-cause outpatient visits with primary care for the usual care versus telemedicine group.

Wabe N, Thomas J, Sezgin G, Sheikh MK, Gault E, Georgiou A. Medication prescribing in in-person versus telehealth consultations during the COVID-19 pandemic in Australian general practice: a retrospective observational study. BJGP open. 2022 Mar;6(1). PubMed
A multisite, retrospective observational study compared medication prescribing in telehealth with in-person consultations in general practice in Australia. It was found that the prescribing rate was greater for in-person versus telehealth consultations for all drug groups except for Anatomical Therapeutic Chemical level 1 (nervous system).

Ye S, Anstey DE, Grauer A, et al. The Impact of Telemedicine Visits on the Controlling High Blood Pressure Quality Measure During the COVID-19 Pandemic: Retrospective Cohort Study. JMIR Form Res. 2022 Mar 23;6(3):e32403. .PubMed
This retrospective cohort study examined the association between telemedicine visits and failure to meet the Controlling High Blood Pressure quality measure (i.e., primary outcome). This study found that telemedicine visits were associated with poorer performance on the primary outcome versus in-person visits. The study authors noted telemedicine may not negatively impact blood pressure control for instances where at least 1 blood pressure was recorded.

Entezarjou A, Calling S, Bhattacharyya T, et al. Antibiotic Prescription Rates After eVisits Versus Office Visits in Primary Care: Observational Study. JMIR Med Inform. 2021 Mar 15;9(3):e25473. PubMed
The authors of this retrospective cohort study investigated whether e-visit management leads to higher rates of antibiotic prescription versus usual care (i.e., in-person, office visits). This study focused on the management of dyuria, sore throat, and other respiratory symptoms. The study found that antibiotic prescription rates were lower for patients who had an e-visit versus in-person for patients being treated for respiratory symptoms and sore throat. This study did not find statistically significant differences between groups in antibiotic prescription rates for dysuria or guideline-recommended antibiotics for sore throat. Guideline-recommended antibiotics were more often prescribed for patients who had an e-visit versus in-person for respiratory symptoms and dysuria.

Johnson KL, Dumkow LE, Salvati LA, Johnson KM, Yee MA, Egwuatu NE. Comparison of diagnosis and prescribing practices between virtual visits and office visits for adults diagnosed with uncomplicated urinary tract infections within a primary care network. Infect Control Hosp Epidemiol. 2021 05;42(5):586-591. PubMed: PubMed
This retrospective cohort study focused on comparing virtual visits with in-person visits within primary care on guideline-concordant antibiotic prescribing for urinary tract infections. It was found that patients who got treatment virtually versus in-person were more likely to receive antibiotics and guideline-concordant duration and less likely to have a urinalysis ordered, urine culture ordered, or have another visit within 7 days.

Leventer-Roberts M, Shimoni N, Feldman B, et al. Diagnoses and Health Care Utilization for After-Hours Telemedicine Versus Primary Care Visits. Acad Pediatr. 2021;21(8):1414-1419. PubMed
A retrospective, cross-sectional study used a large health care provider database to compare primary care pediatricians online after-hours telemedicine service with in-person visits. Compared with in-person visits, the users of the telemedicine service had higher rates of emergency department admissions and hospital admissions and lower rates of laboratory testing and medication prescription within 24 hours.

Lu AD, Gunzburger E, Glorioso TJ, et al. Impact of Longitudinal Virtual Primary Care on Diabetes Quality of Care. J Gen Intern Med. 2021 09;36(9):2585-2592. PubMed
This propensity score–matched cohort study evaluated the differences in diabetes care before and after the implementation of a longitudinal virtual primary care program a part of Veteran Affairs. Patients with diabetes who participated in the virtual program had larger changes in mean hemoglobin A1C levels versus those who had usual care. This study found no statistically significant differences between the 2 groups (virtual program versus in-person) for changes in hemoglobin A1C levels and in the proportion of patients with controlled blood pressure.

O'Shea AMJ, Batten A, Hu EY, Augustine MR, Hogan TP, Kaboli PJ. Association of Secure Messaging with Primary Care In-Person and Telephone Visits Among Veterans: a Matched Difference-in-Difference Analysis. J Gen Intern Med. 2021 04;36(4):946-951. PubMed
This study conducted 2 retrospective cohort analyses to determine if there were associations between secure messaging use and in-person or telephone visits in a primary care setting. Overall, secure messaging use was associated with decreased in-person visits and increased telephone visits.

Phillips JC, Lord RW, Davis SW, Burton AA, Kirk JK. Comparing telehealth to traditional office visits for patient management in the COVID-19 pandemic: A cross-sectional study in a respiratory assessment clinic. J Telemed Telecare. 2021 Feb 01:1357633X21990197. PubMed
A retrospective cross-sectional study examined whether telehealth is safe and effective compared to traditional office visits in assessing and treating patients with COVID-19 symptoms. This study did not find statistically significant differences in related follow-ups (telehealth versus in-person), including no statistically significant differences in emergency department visits or hospital admissions.

Reed M, Huang J, Graetz I, Muelly E, Millman A, Lee C. Treatment and Follow-up Care Associated With Patient-Scheduled Primary Care Telemedicine and In-Person Visits in a Large Integrated Health System. JAMA netw. 2021 11 01;4(11):e2132793. PubMed
This cohort study examined if prescribing and orders by a primary care physician are different between telemedicine and office visits and found that prescribing and orders were significantly lower for telemedicine visits. The study also examined if physicians conducting telemedicine visits are more likely to require in-person follow-up and found slightly higher follow-up office visits for the telemedicine group. Finally, this study examined if telemedicine visits versus in-person visits are associated with more health events and found no differences in emergency department visits or hospitalizations between groups.

Tarn DM, Hintz C, Mendez-Hernandez E, Sawlani SP, Bholat MA. Using Virtual Visits to Care for Primary Care Patients With COVID-19 Symptoms. J Am Board Fam Med. 2021 Feb;34(Suppl):S147-S151. PubMed
This retrospective electronic health record review examined the use of office resources by patients with COVID-19 symptoms who were initially evaluated through telehealth, telephone, or in-person within the primary care setting, The study findings suggest no difference in the number of office interactions or emergency department visit based on initial visit type.

References Addressing Research Question 2: Cost-Effectiveness

Health Technology Assessments

No literature identified.

Systematic Reviews

No literature identified.

Economic Evaluations

No literature identified.

References Addressing Research Question 3: Patient and Provider Experiences and Preferences

Health Technology Assessments

No literature identified.

Systematic Reviews

Goncalves RL, Pagano AS, Reis ZSN, et al. Usability of Telehealth Systems for Noncommunicable Diseases in Primary Care From the COVID-19 Pandemic Onward: Systematic Review. J Med Internet Res. 2023;25:e44209. PubMed
This SR synthesized evidence on the perceptions of health care professionals regarding the usability of telehealth systems in the primary care of individuals with noncommunicable diseases. The authors included 11 studies which contained data from 248 providers. This study reported good usability of telehealth and providers feeling satisfied and comfortable. The main predictor of using digital health technologies was ease of use, and the main barriers were connectivity issues, low computer literacy, technological challenges, inability to perform complete physical examination, and lack of training.

Cantor A, Nelson HD, Pappas M, Atchison C, Hatch B, Huguet N, Flynn B, McDonagh M. Effectiveness of Telehealth for Women’s Preventive Services. Comparative Effectiveness Review No. 256. (Prepared by the Pacific Northwest Evidence-based Practice iii Center under Contract No. 75Q80120D00006.) AHRQ Publication No. 22-EHC024. Rockville, MD: Agency for Healthcare Research and Quality; June 2022. PubMed
This SR examined the effects of telehealth for women, specifically preventive services for reproductive health care and interpersonal violence. One of the outcomes examined for this SR was patient preferences. The facilitators to telehealth interventions included strategies to ensure safety of individuals who receive interpersonal violence services. The barriers to telehealth interventions included digital literacy for English-speaking interpersonal violence survivors, technical challenges, confidentiality concerns for contraceptive care, and limited internet access.

Greenwood H, Krzyzaniak N, Peiris R, et al. Telehealth Versus In-person Psychotherapy for Less Common Mental Health Conditions: Systematic Review and Meta-analysis of Randomized Controlled Trials. JMIR Ment Health. 2022;9(3):e31780. PubMed
Authors of this SR compared the effect of telehealth to in-person delivery of psychotherapy for people with mental and physical health conditions. Results of a meta-analysis found no statistically significant differences between the 2 groups for client satisfaction.

Hubschman-Shahar LE. Lactation Telehealth in Primary Care: A Systematic Review. Breastfeed Med. 2022;17(1):6-21. PubMed
This SR assessed the impact of receiving skilled lactation care through telehealth compared with routine care on patient satisfaction. Authors observed high satisfaction rates for those individuals who used lactation telehealth (unknown if evidence was comparative).

Scott AM, Bakhit M, Greenwood H, et al. Real-Time Telehealth Versus In-person Management for Patients With PTSD in Primary Care: A Systematic Review and Meta-Analysis. J Clin Psychiatry. 2022;83(4):23. PubMed
This meta-analysis compared synchronous telehealth (phone, video) to in-person therapy delivery to patients with posttraumatic stress disorder. This review reported no differences between groups regarding treatment satisfaction.

Randomized Controlled Trials

No literature identified.

Nonrandomized Studies

Kannenberg B, Stadter G. Analysis and Observations of Telehealth in Primary Care Follow Up Appointments for Vulnerable Populations. Wmj. 2022 Jul;121(2):116-120. PubMed
One of the main objectives of this cross-sectional study was to obtain perspectives on the strengths and weaknesses of virtual primary care via patient and provider feedback. The authors stated that telehealth can help patients overcome barriers such as childcare and transportation issues.

Nijhof D, Ingram A, Ochieng R, Roberts EJ, Poulton B, Ochieng B. Examining GP online consultation in a primary care setting in east midlands, UK. BMC Health Serv Res. 2021 Sep 30;21(1):1030. PubMed
A retrospective chart review investigated the acceptability of an e-consultation system for health care users within primary care. This study found that the platform was positively received by its users, and the authors suggested that an online consultation was convenient for patients and has the potential to relieve pressure placed on primary care services. This study reported several challenges, including patient verification.

References Addressing Research Question 4: Equity Issues

Health Technology Assessments

No literature identified.

Systematic Reviews

Cantor A, Nelson HD, Pappas M, Atchison C, Hatch B, Huguet N, Flynn B, McDonagh M. Effectiveness of Telehealth for Women’s Preventive Services. Comparative Effectiveness Review No. 256. (Prepared by the Pacific Northwest Evidence-based Practice iii Center under Contract No. 75Q80120D00006.) AHRQ Publication No. 22-EHC024. Rockville, MD: Agency for Healthcare Research and Quality; June 2022. PubMed
This SR examined the effects of telehealth for women, specifically preventive services for reproductive health care and interpersonal violence. This SR examined access and health equity and found insufficient evidence for both outcomes to make any conclusions.

Randomized Controlled Trials

No literature identified.

Nonrandomized Studies

Chami N, Shah HA, Nastos S, et al. Association between virtual primary care and emergency department use during the first year of the COVID-19 pandemic in Ontario, Canada. Cmaj. 2023 Jan 23;195(3):E108-E114. PubMed
In this observational study, the authors used administrative data from Ontario, Canada to evaluate whether a shift from in-person to virtual visits by primary care physicians is associated with an increased use of emergency department visits. The authors reported the proportion of virtual visits was higher for patients who were treated by younger, female physicians and by physicians in urban areas. The number of visits to the emergency department was lower for patients who were treated by female physicians and by physicians in urban areas.

Chen K, Zhang C, Gurley A, Akkem S, Jackson H. Appointment Non-attendance for Telehealth Versus In-Person Primary Care Visits at a Large Public Healthcare System. J Gen Intern Med. 2023 03;38(4):922-928. PubMed
In this observational study, the authors used electronic health records data to compare nonattendance for primary care telehealth versus primary care in-person visits and described patient characteristics associated with telehealth nonattendance. The authors found video visits were associated with increased nonattendance for patients who were older, were male, had a primary language other than English or Spanish, and had public or no health insurance. The authors found telephone visits had fewer differences in nonattendance based on demographic features.

Cherabuddi MR, Heidemann D, Gwinn M, et al. Disparities in Use of Virtual Primary Care During the Early COVID-19 Pandemic. Telemed J E Health. 2023 Jan 13;13:13. PubMed
This cross-sectional retrospective study examined how certain factors may affect utilization of virtual care. This study found, across equivalent age ranges, patients who were Black were slightly less likely to use virtual care versus similarly aged patients who were white (not consistently across virtual modalities), women were more likely to use virtual care (across all modalities), and individuals aged 65 years or older were more likely to use telephone visits and less likely to use video and e-visits, irrespective of race. This study also found that patients living in areas with the greatest socioeconomic advantage were more likely to use video and e-visits.

Chumbler NR, Chen M, Harrison A, Surbhi S. Racial and Socioeconomic Characteristics Associated with the use of Telehealth Services Among Adults With Ambulatory Sensitive Conditions. Health serv. 2023 Jan-Dec;10:23333928231154334. PubMed
This retrospective cohort study examined the association of outpatient telehealth utilization with sociodemographic, clinical, and neighbourhood characteristics among adults with ambulatory care–sensitive conditions during the COVID-19 pandemic. Patients had higher rates of telehealth services if they were older, were female, had a mental disorder, and had more comorbidities. After controlling for covariates, study authors observed increased use of telehealth services among patients who were Hispanic and other race groups versus patients who were white. Additionally, patients who commuted more than 30 minutes to health facilities were slightly less likely to use telehealth services, and patients with mental health disorders who were Black or Hispanic were more likely to use telehealth services versus patients who were white.

El-Nahal WG, Chander G, Jones JL, et al. Telemedicine Use Among People With HIV in 2021: The Hybrid-Care Environment. J Acquir Immune Defic Syndr. 2023 03 01;92(3):223-230. PubMed
The authors used observational data to analyze all 2021 telemedicine and in-person primary care visits with patients with HIV who were enrolled in the Johns Hopkins HIV Clinical Cohort. The relative risk of completing telemedicine versus in-person visits was 0.65 (95% CI, 0.47 to 0.91) for patients aged ≥ 65 years versus age 20 to 39 years; 0.84 (95% CI, 0.72 to 0.98) for male patients compared with female patients; 0.81 (95% CI, 0.66 to 0.99) for Black patients compared with white patients; 0.62 (95% CI, 0.49 to 0.79) for patients in the highest compared with the lowest quartile of the Area Deprivation Index; and 1.52 (95% CI, 1.26 to 1.84) for patients more than 15 miles versus less than 5 miles from a clinic.

Oh A, Scott JY, Chow A, et al. Rural and urban differences in the implementation of Virtual Integrated Patient-Aligned Care Teams. J Rural Health. 2023 01;39(1):272-278. PubMed
This observational study used administrative data to compare characteristics for users and nonusers of a virtual primary care program offered through Veteran Affairs. After adjusting for certain characteristics, veterans from rural areas and veterans with higher comorbidity risk scores were independently associated with virtual program use. Veterans from highly rural areas and veterans who lived 40 miles or more from the closest Veterans Health Administration primary care site were less likely to use the virtual program.

Adepoju OE, Tran L, Agwuncha R, et al. Associations between Patient- and Provider Level Factors, and Telemedicine Use in Family Medicine Clinics. J Am Board Fam Med. 2022 May-Jun;35(3):457-464. PubMed
This retrospective study used electronic medical records to assess provider- and patient-level factors associated with telemedicine use in community-based family practice clinics. “Non-Hispanic white patients” [from original source] had 61% higher odds of a telemedicine visit, and patients who were “non-Hispanic Black” [from original source] had 32% higher odds of a telemedicine visit, both compared with patients who were identified as Hispanic. The odds of using telemedicine were lower for those who did not have health insurance. Patients living in urban or medically underserved areas had greater odds of a telemedicine appointment. The authors described a “dose-response relationship” regarding commute time and telemedicine use. There were no provider characteristics associated with telemedicine use.

Dai Z, Sezgin G, Li J, et al. Telehealth utilisation in residential aged care facilities during the COVID-19 pandemic: A retrospective cohort study in Australian general practice. J Telemed Telecare. 2022 May 11:1357633X221094406. PubMed
This retrospective cohort study assessed how sociodemographic characteristics affected telehealth utilization. People who had a pension and people living in rural areas were more likely to use telehealth. People living in rural areas were less likely to use video (versus telephone) for consultations with their general practitioner.

Govier DJ, Cohen-Cline H, Marsi K, Roth SE. Differences in access to virtual and in-person primary care by race/ethnicity and community social vulnerability among adults diagnosed with COVID-19 in a large, multi-state health system. BMC Health Serv Res. 2022 Apr 15;22(1):511. PubMed
This cohort study compared changes in virtual and in-person primary care use by community social vulnerability and by race or ethnicity in the year before and after COVID-19 diagnosis. Compared with patients who were white, patients who were Hispanic or Latino or another race or ethnicity were less likely to use virtual care before and after COVID-19 diagnosis; patients who were Native Hawaiian or other Pacific Islander, Hispanic or Latino, or another race or ethnicity, and patients living in socially vulnerable areas were also less likely to use in-person care during these time periods.

Haggerty T, Stephens HM, Peckens SA, et al. Telemedicine versus in-Person Primary Care: Impact on Visit Completion Rate in a Rural Appalachian Population. J Am Board Fam Med. 2022 May-Jun;35(3):475-484. PubMed
This retrospective cohort study aimed to understand how telemedicine delivery of family medicine care affects patient access and visit completion rates in a rural setting. The authors found that patients of “working age” [from original source] were more likely to complete their telemedicine visits. Patients who were older with higher risk scores were more likely to complete their telemedicine visits if they used telemedicine.

Pagan VM, McClung KS, Peden CJ. An Observational Study of Disparities in Telemedicine Utilization in Primary Care Patients Before and During the COVID-19 Pandemic. Telemed J E Health. 2022 08;28(8):1117-1125. .PubMed
This study investigated whether certain patient populations have experienced barriers in accessing primary care using telemedicine by comparing visit type and patient characteristics at a large, urban, academic medical centre. For patients who completed a telemedicine visit, racial minority status, Hispanic or Latino ethnicity, older age, and non–English-language preference increased the likelihood of a telephone visit versus patients who were younger adults, were white, were not Hispanic or Latino, and had an English-language preference.

Schenker RB, Laguna MC, Odisho AY, Okumura MJ, Burnett H. Are We Reaching Everyone? A Cross-Sectional Study of Telehealth Inequity in the COVID-19 Pandemic in an Urban Academic Pediatric Primary Care Clinic. Clin Pediatr (Phila). 2022 01;61(1):26-33. .PubMed
This cross-sectional study examined whether telehealth implementation at an academic pediatric practice led to disparities in access to health care. This study found age was the only significant difference in patient demographics between in-person and telehealth visits during COVID-19. Multivariate regression analysis found older age was a positive predictor of having a video visit and public insurance as a significant negative predictor.

Cao YJ, Chen D, Liu Y, Smith M. Disparities in the Use of In-Person and Telehealth Primary Care Among High- and Low-Risk Medicare Beneficiaries During COVID-19. J Patient Exp. 2021;8:23743735211065274. PubMed
This retrospective cohort study estimated differences in primary care outpatient clinic visit utilization among high- and low-risk Medicare aging beneficiaries during COVID-19 compared to a control cohort from the year prior. The authors found significant disparities in primary care utilization during COVID-19 among Medicare beneficiaries: there was lower utilization for patients who were older, male, or dually eligible for Medicaid in the high-risk group and for patients who were younger, male, or from racial and ethnic minority groups in the low-risk group. In general, it was less common for patients to use telehealth if they were older, dually eligible for Medicaid, or living in rural/suburban areas versus urban areas.

Chakawa A, Belzer LT, Perez-Crawford T, Yeh HW. COVID-19, Telehealth, and Pediatric Integrated Primary Care: Disparities in Service Use. J Pediatr Psychol. 2021 09 27;46(9):1063-1075. PubMed
This comparative study explored the variability between in-person and telehealth primary care consultation utilization among children. This study found the odds of nonattendance were greater for children scheduled for telehealth versus in-person, the odds of children with internalizing problems being scheduled for telehealth were greater than those with externalizing problems, and the odds of children who are Black being scheduled for telehealth were less compared to children who are white.

Dahlgren C, Dackehag M, Wandell P, Rehnberg C. Determinants for use of direct-to-consumer telemedicine consultations in primary healthcare-a registry based total population study from Stockholm, Sweden. BMC Fam Pract. 2021 06 26;22(1):133. PubMed
This study explored the determinants for use of direct-to-consumer telemedicine consultations within a primary care setting. The following factors were associated with higher probability of telemedicine use: younger age, higher educational attainment, higher income, and being born in Sweden. The following factors were associated with higher probability of in-person visits: higher age, lower educational background, and being born outside of Sweden.

Haynes SC, Kompala T, Neinstein A, Rosenthal J, Crossen S. Disparities in Telemedicine Use for Subspecialty Diabetes Care During COVID-19 Shelter-In-Place Orders. J Diabetes Sci Technol. 2021 09;15(5):986-992. PubMed
This explanatory sequential mixed-methods study aimed to identify patient-level factors associated with adoption of telemedicine for care of diabetes during the COVID-19 pandemic. Patients aged 65 years or older, whose primary language was not English, and with public insurance were less likely to use telemedicine.

Leventer-Roberts M, Shimoni N, Feldman B, et al. Diagnoses and Health Care Utilization for After-Hours Telemedicine Versus Primary Care Visits. Acad Pediatr. 2021;21(8):1414-1419. PubMed
A retrospective, cross-sectional study used a large health care provider database to compare primary care pediatricians’ online after-hours telemedicine service with in-person visits. The users of the telemedicine service were more likely to have a higher socioeconomic status.

Ryskina KL, Shultz K, Zhou Y, Lautenbach G, Brown RT. Older adults' access to primary care: Gender, racial, and ethnic disparities in telemedicine. J Am Geriatr Soc. 2021 10;69(10):2732-2740. PubMed
This retrospective cross-sectional study examined older adults' use of telemedicine versus in-person visits within primary care and compared hospitalization for ambulatory care–sensitive conditions between the groups. Patients who were Black had higher odds of using telemedicine versus patients who were white and patients who were Hispanic had lower odds versus patients who were not Hispanic. For patients who used telemedicine, patients who were Black had higher odds of hospitalization compared with white patients, and patients aged 85 years or older who used telemedicine had higher odds of a hospitalization versus patients between the ages of 65 and 74 years.

Eberly LA, Kallan MJ, Julien HM, et al. Patient Characteristics Associated With Telemedicine Access for Primary and Specialty Ambulatory Care During the COVID-19 Pandemic. JAMA netw. 2020 12 01;3(12):e2031640. PubMed
In this cohort study, authors investigated whether and what inequities are present in telephone and video use for telemedicine visits during the COVID-19 pandemic. By conducting a retrospective review of medical records, the authors found patients who were older, Asian, or non-English-speaking had lower rates of telephone use, while patients that who were older, female, Black, Latinx, and with lower income had less video use.

Jain T, Mehrotra A. Comparison of Direct-to-Consumer Telemedicine Visits With Primary Care Visits. JAMA netw. 2020 12 01;3(12):e2028392. PubMed
This cross-sectional study compared patients who used a direct-to-consumer telemedicine service with patients who used primary care visits in the 20 US states where the service was available. No other abstract details are available.

Reed ME, Huang J, Graetz I, et al. Patient Characteristics Associated With Choosing a Telemedicine Visit vs Office Visit With the Same Primary Care Clinicians. JAMA netw. 2020 06 01;3(6):e205873. .PubMed
This cross-sectional study examined patient characteristics associated with choosing a primary care telemedicine visit versus primary care office visit with the same clinicians. The study findings suggested that choosing telemedicine was statistically significantly associated with patient sociodemographic characteristics (e.g., older adults were less likely to choose telemedicine compared with younger adults), access (e.g., patients living in an area with better internet access were more likely to choose telemedicine than those who did not), and in-person visit barriers (e.g., patients who had to pay for parking were more likely to choose telemedicine than patients who had free parking to attend an in-person appointment).

Appendix 4: References of Potential Interest

Guidance and Policy Documents

Organisation for Economic Cooperation and Development (OECD). The COVID-19 Pandemic and the Future of Telemedicine. OECD Health Policy Studies; 2023. https://www.oecd-ilibrary.org/social-issues-migration-health/the-covid-19-pandemic-and-the-future-of-telemedicine_ac8b0a27-en Accessed 2023 Apr 14.

Ontario Health. Clinically Appropriate Use of Virtual Care in Primary Care; 2022. https://www.ontariohealth.ca/sites/ontariohealth/files/2022-11/ClinicallyAppropriateUseVirtualCarePrimaryCare.pdf Accessed 2023 Apr 14.

Canadian Network for Digital Health Evaluation. Optimizing the use of Virtual Primary Care Equity, Policies and Workflows. Canadian Network for Digital Health Evaluation, Women’s College Hospital Institute for Health System Solutions and Virtual Care; 2022. https://cdhe.wchwihv.ca/assets/files/PDFs/network/OptimizingUseOfVirtualPrimaryCareEquityPoliciesAndWorkflows.pdf Accessed 2023 Apr 14.

University of Calgary. The Regulation of Virtual Health Care Services in Canada; 2022. https://obrieniph.ucalgary.ca/sites/default/files/HARDCASTLE%20-%20The%20Regulation%20of%20Virtual%20Health%20Care%20Services%20in%20Canada%20-%20FINAL%20V2.pdf Accessed 2023 Apr 14.

Healthcare Excellence Canada. What we Heard: Results of a Policy Lab on the Appropriate Use of Virtual Care in a Primary Care Setting; 2022. https://www.healthcareexcellence.ca/media/rr2ctlt4/20220412_virtualcarepolicylab_en.pdf Accessed 2023 Apr 14.

Health Canada. Enhancing Equitable Access To Virtual Care In Canada: Principle-based Recommendations for Equity; 2021. https://www.canada.ca/content/dam/hc-sc/documents/corporate/transparency_229055456/health-agreements/bilateral-agreement-pan-canadian-virtual-care-priorities-covid-19/template-ett-report-docx-eng.pdf Accessed 2023 Apr 14.

Health Canada. The State Of Virtual Care In Canada As Of Wave Three Of The Covid-19 Pandemic: An Early Diagnostique and Policy Recommendations; 2021. https://www.canada.ca/content/dam/hc-sc/documents/corporate/transparency_229055456/health-agreements/bilateral-agreement-pan-canadian-virtual-care-priorities-covid-19/template-wf-report-eng.pdf Accessed 2023 Apr 14.

Office of the Auditor General of Ontario. Value-for-Money Audit Virtual Care: Use of Communication Technologies for Patient Care; 2020. https://www.auditor.on.ca/en/content/annualreports/arreports/en20/20VFM_08virtualcare.pdf Accessed 2023 Apr 14.

Previous CADTH Reports

Approaches to Evaluations of Virtual Care in Primary Care. Ottawa: CADTH; 2022. Approaches to Evaluations of Virtual Care in Primary Care (cadth.ca) Accessed 2023 Apr 14.

Virtual Care Rapid Scoping. Ottawa: CADTH; 2022. https://www.cadth.ca/virtual-care-rapid-scoping Accessed 2023 Apr 14.

Cooper-Jones B, Mason J, Kamel C, Mittmann N, Dunfield L. Virtual care and health technology assessment considerations. Healthcare Management Forum. 2022;35(3):127-129. PubMed

Evaluations of Virtual Care. Ottawa: CADTH; 2022. https://www.cadth.ca/evaluations-virtual-care Accessed 2023 Apr 14.

Virtual Care Use in Primary Care or Specialty Care Settings. Ottawa: CADTH; 2022. https://www.cadth.ca/virtual-care-use-primary-care-or-specialty-care-settings Accessed 2023 Apr 14.

An Overview of Direct-to-Patient Virtual Visits in Canada. Ottawa: CADTH; 2021. https://www.cadth.ca/sites/default/files/hs- Accessed 2023 Apr 14.eh/EH0091%20Virtual%20Visits%20Final.pdf Accessed 2023 Apr 14.

Direct to Patient Virtual Visits. Ottawa: CADTH; 2021. https://www.cadth.ca/direct-patient-virtual-visits Accessed 2023 Apr 14.

Virtual Health Care for Adults with Concurrent Disorders: Clinical Effectiveness and Guidelines. Ottawa: CADTH; 2020. https://www.cadth.ca/virtual-health-care-adults-concurrent-disorders-clinical-effectiveness-and-guidelines Accessed 2023 Apr 14.

Connected Devices to Support Remote Examination and Diagnosis in Primary Care and Specialty Care. Ottawa: CADTH; 2020. https://www.cadth.ca/connected-devices-support-remote-examination-and-diagnosis-primary-care-and-specialty-care Accessed 2023 Apr 14.