CADTH Health Technology Review

Automatic Stop Orders for Opioids

Rapid Review

Authors: Zahra Jafari, Sharon Bailey

Key Messages

Context and Policy Issues

Pain is one of the most common reasons people seek health care. Acute pain is a physiologic response to noxious stimuli that can become pathologic.1 Acute pain is often sudden and short-lived (lasting less than a month) and is typically caused by injury, trauma, or medical treatments such as surgical procedures.1 Unresolved acute or subacute pain (lasting for 1 to 3 months) can evolve into chronic pain.2 Opioids are effective drugs that play an imperative role in acute pain management.3 However, inappropriate prescribing and use of opioids can cause harm, including addiction, poisoning, and death.3 According to Health Canada, prescriptions written by health care professionals are a common source of opioids in Canada, and Canadians are the second highest users per capita of prescription opioids in the world.4 Because overprescribing of opioids could be a contributor to Canada’s opioid crisis, the Minister of Health issued a letter in June 2018 to Canadian manufacturers and distributors of opioids to immediately cease any marketing and advertising of opioids to health care professionals.4 On March 11, 2019, Health Canada considered further restricting the marketing and advertising of opioids by proposing additional terms and conditions for prescribing opioid-containing products.5,Since 2016, approximately half of all states in the US have passed legislation limiting initial opioid prescriptions for acute pain to a maximum 7-day supply. Similar policies are applied by many insurers, pharmacy benefit managers, and pharmacies.2 A limit on dispensing maximum opioid doses is also applied by several states.6

Despite the importance of minimizing unnecessary opioid exposure, opioid treatment for acute pain is not a one-size-fits-all situation.6 Although many patients do not require opioids, others may need at least a minimum of 3 days of treatment, especially when they have contraindications to nonopioid analgesics or they lack access to timely follow-up care outside the emergency department.6 In 2018, Health Quality Ontario released a quality standard offering guidance on prescribing opioids to treat people aged 15 years and older with acute pain in all care settings.7 According to this quality standard, people with acute pain first receive multimodal therapy consisting of nonopioid pharmacotherapy with physical and/or psychological interventions. If opioids are required, they are prescribed at the lowest effective dose of the least-potent immediate-release opioid for 3 days; a duration of more than 7 days is rarely indicated.7

Automated stop orders (ASOs) are policies or guidance that can support clinical decisions and actions regarding prescribing, dispensing, and administering medications. There are 2 types of ASOs: “soft stops” and “hard stops.”8 A soft stop provides information to the clinician about a potential drug safety or efficacy problem and alternative suggestions to consider, whereas a hard stop terminates the progress of prescribing, dispensing, or administering a potentially dangerous medication.8 Although soft stops might not be noticed by patients or clinicians for a variety of reasons (e.g., alert fatigue or improper format of the warning), hard stops are expected to be more effective in capturing the patient’s attention and navigating them to communicate with their clinicians to change the prescription.9 ASOs related to antimicrobial prescribing and dispensing have been implemented in some North American medical centres. Published studies from this context have reported that although well-organized hard stops may improve the safety and efficacy of the drug therapy and prevent adverse events, they may delay order completion or slow the dispensing and administration process for patients who still require treatment.9 In addition, some clinicians may find hard stops objectionable, noting that decision support should not replace the clinician’s responsibility for patients.9 Given the harms of opioid overprescribing and prolonged use, and the experience with antimicrobial ASOs, there is interest in considering ASOs for opioid analgesics in acute care settings. Therefore, the objectives of this report were to summarize and critically appraise current literature regarding the clinical effectiveness of ASOs for opioid prescriptions and the evidence-based guidelines regarding the use of ASOs for opioid prescriptions.

Research Questions

  1. What is the clinical effectiveness of automatic stop orders for opioid prescriptions?

  2. What are the evidence-based guidelines regarding the use of automatic stop orders for opioid prescriptions?

Methods

Literature Search Methods

An information specialist conducted a literature search on key resources including MEDLINE, Embase, the Cochrane Database of Systematic Reviews, the International HTA Database, the websites of Canadian and major international health technology agencies, as well as a focused internet search. The search approach was customized to retrieve a limited set of results, balancing comprehensiveness with relevancy. The search strategy comprised both controlled vocabulary, such as the National Library of Medicine’s MeSH (Medical Subject Headings), and keywords. Search concepts were developed based on the elements of the PICOS (population, intervention, comparison, outcome measures, study design) framework and research questions. The main search concepts were opioids and medication assessment alerts. No filters were applied to limit the retrieval by study type. The search was completed on January 19, 2023, and was limited to English-language documents published since January 1, 2008.

Selection Criteria and Methods

One reviewer screened citations and selected studies. In the first level of screening, titles and abstracts were reviewed, and potentially relevant articles were retrieved and assessed for inclusion. The final selection of full-text articles was based on the inclusion criteria presented in Table 1.

Table 1: Selection Criteria

Criteria

Description

Population

Patients who are prescribed opioid analgesics

Exclude: long-acting opioids, fentanyl patches, opioids used to treat substance use disorders (e.g., buprenorphine-naloxone, methadone); patients in the following settings: palliative care, hospice, end-of-life care

Intervention

Automatic stop orders

Comparator

Question 1: No automatic stop orders

Question 2: Not applicable

Outcomes

Question 1: Clinical benefits (e.g., opioid use; reduction in opioid dependence, misuse, tolerance; duration of therapy; discharge prescriptions; health-related quality of life) and harms (e.g., adverse events)

Question 2: Recommendations regarding the use of automatic stop orders for opioids or controlled drugs (e.g., best practices, contraindications)

Study designs

Question 1: Health technology assessments, systematic reviews, randomized controlled trials, nonrandomized studies

Question 2: Evidence-based guidelines

Exclusion Criteria

Articles were excluded if they did not meet the selection criteria outlined in Table 1, were duplicate publications, or were published before January 1, 2008.

Critical Appraisal of Individual Studies

The included publications met the selection criteria and were critically appraised by 1 reviewer using the Downs and Black checklist10 for randomized and nonrandomized studies. Summary scores were not calculated for the studies; each publication's strengths and limitations were described narratively Appendix 3

Summary of Evidence

Quantity of Research Available

A total of 522 citations were identified in the literature search. Following the screening of titles and abstracts, 518 citations were excluded and 4 potentially relevant articles from the electronic search were retrieved for full-text review. No potentially relevant publications were retrieved from the grey literature search for full-text review. Of these 4 potentially relevant articles, 2 were excluded for irrelevant populations; 2 publications11,12 met the selection criteria and were included in this report.

Appendix 1 presents the PRISMA13 flow chart of the study selection.

Additional references of potential interest are provided in Appendix 5.

Summary of Study Characteristics

Additional details regarding the characteristics of included publications are provided in Appendix 2

Study Design

Two observational, retrospective chart review studies were included in this report. The first study by Vyhmeister and Sierra (2022)12 aimed to describe changes in codeine prescribing practices in the pediatric population after the FDA soft-stop order in 2017. In the second publication, Brokenshire and colleagues (2021)11 studied the impact of oral and injectable opioid product restrictions on inpatient opioid use during a critical drug shortage period.

Country of Origin

The included studies were conducted by authors based in the US.11,12

Patient Population

In the retrospective study by Vyhmeister and Sierra (2022),12 data from pediatric patients younger than 12 years of age who were prescribed codeine-containing products between February 2017 and April 2018 were collected and analyzed before and after the FDA added boxed warnings for codeine-based products. The authors reviewed trends in prescribing codeine-based products from the records of 4 hospitals in an academic health system, which included inpatients in a dedicated children’s hospital, 3 outpatient surgery centres, and general and specialty clinics. Patients were excluded if codeine-containing prescriptions were discontinued before the end of the prescription period and therefore were not filled. In this study, the data were not reported in subgroups that could allow us to classify sample size and results for inpatients and outpatients.

In the retrospective study by Brokenshire and colleagues (2021),11 adult patients admitted to the University of Florida Health Shands Hospital were included in the study if they received at least 1 opioid product (either IV or oral route) outside of the operating room setting. Data for patients who underwent ambulatory surgery were only included if medications were received before entering the operating room.

Interventions and Comparators

In the retrospective study by Vyhmeister and Sierra (2022),12 the effect of the FDA boxed warning (intervention) on codeine prescribing practices between April 20, 2017, and April 30, 2018, was compared with a time frame before the FDA soft stop between February 1, 2017, and April 19, 2017 (comparator).

In the retrospective study by Brokenshire and colleagues (2021),11 the intervention plan consisted of 10 discrete phases of soft-stop or hard-stop medication restrictions within the study time span after a baseline period (comparator). A soft stop was defined as an automatic warning message meant to provide information about product availability including suggested therapeutic alternatives.14 A hard stop was defined as an automatic warning message that communicated essential information through the electronic health record and prevented the user from proceeding with the intended order. A restriction was classified as a hard stop if the medication was unable to be ordered or was only available if a patient met a predefined list of strict criteria for use.14 Table 2 describes the 10 discrete phases of medication restrictions in this study after the baseline period (phase 0) of 129 days starting on July 1, 2017. The phases were set for at least 7 days to allow for sufficient time to detect a change in usage and to minimize the heterogeneity of prescribing practices within each phase. The 10 phases were structured based on significant changes in the ability of providers to order IV hydromorphone, morphine, and fentanyl products.

Table 2: Intravenous Medication Ordering Restrictions Implemented in the Study by Brokenshire and Colleagues (2021)11

Study phases

Start date

Duration (days)

Description of stop

1

November 7, 2017

8

A hard stop for morphine and hydromorphone

Removal of default morphine and hydromorphone PRN orders by modification of PACU order sets

2

November 15, 2017

15

Hydromorphone transition from hard stop to soft stop

3

November 30, 2017

49

Hydromorphone transition from soft stop to no restriction

4

January 18, 2018

19

Soft-stop order for hydromorphone and ketamine infusion

5

February 6, 2018

15

Hydromorphone transition from soft stop to hard stop

Removal of hydromorphone from the emergency department override list

Soft-stop order for hydromorphone, fentanyl, and morphine PCA

6

February 21, 2018

20

Morphine transition from hard stop to soft stop

Soft-stop order for fentanyl

7

March 13, 2018

28

Implementation of strict criteria for PCA ordering

Changing fentanyl infusion to morphine infusion, unless clinically inappropriate

The transition of fentanyl from soft stop to hard stop with criteria for use

The transition of hydromorphone and fentanyl orders within PACU order set to meperidine; adding fentanyl soft stop to intraprocedural order set; maintaining hydromorphone on labour and delivery order set for acute pain due to concerns with meperidine use in this population

Removal of fentanyl from Omnicell override

8

April 10, 2018

31

Expansion of PCA criteria for use

Allowing short-dated fentanyl products in select units

Adding hydromorphone back to PACU order sets

Incorporating fentanyl infusion within “neurosurgery ICU sedation and analgesia” and “ICU sedation and analgesia” order sets

9

May 11, 2018

14

Fentanyl infusion transition from hard stop to soft stop

10

May 24, 2018

38

Morphine infusion transitioned from soft stop to hard stop

ICU = intensive care unit; PACU = postanesthesia care unit; PCA = patient-controlled analgesia.

Outcomes

In the retrospective study by Vyhmeister and Sierra (2022),12 the outcome measure was the rate of codeine-containing prescriptions, and the authors reported patient demographics, provider specialty, and indication for prescriptions.

In the retrospective study by Brokenshire and colleagues (2021),11 the outcome measures were changes in oral and injectable opioid product administration during 10 consecutive, discrete medication restrictions; changes in total institutional opioid administration; and changes in the utilization of individual restricted opioid drugs (i.e., morphine, fentanyl, and hydromorphone). The data were analyzed using an interrupted time series regression methodology for each of the 3 utilization measures of interest. This statistical technique allows for quantifying the effect of each restriction implemented during the study period, and was reported by the study authors to be the preferred methodology to study the effect of policies over time. The authors applied segmented linear regression models separately for each utilization measure (i.e., all 3 outcome measures). In this study, total opioid doses were converted to the corresponding morphine milligram equivalent (MME), and opioid consumption was computed as MME per patient per day to account for variations in the number of admitted patients during the study period. In addition, due to limitations in the documentation of total MMEs administered via continuous infusion and patient-controlled analgesia syringes, the authors used pharmacy dispensing records to quantify the total MMEs of opioid infusion products.

Summary of Critical Appraisal

In the retrospective study by Vyhmeister and Sierra (2022),12 the objective, the main outcome to be measured, the characteristics of the included patients, and the intervention of interest were clearly described. In the study design, the authors did not consider a similar time frame before (2 months and 19 days) or after (12 months and 10 days) the intervention (the FDA soft stop for codeine-based prescriptions), and the effect of a short preintervention period or a longer postintervention time period on the outcomes was unclear. The study design was limited to an academic health system, and no external institution was involved in the chart review. Thus, the roles of confounding factors, such as receiving codeine or other opioid prescriptions from providers at outside clinics and institutions or filling prescriptions at outside pharmacies, were not considered in the study design and their effects on findings were uncertain. For data presentation, the study’s findings were presented descriptively, and no statistical tests were applied to compare changes in codeine-containing prescriptions both before and after the FDA soft-stop order. Thus, it was unclear whether the outcomes reported were statistically significant. Likewise, the study findings were not reported by subgroups to allow for the classification of the results into inpatient and outpatient hospitals or clinics.

In the retrospective study by Brokenshire and colleagues (2021),11 the objective, the main outcomes to be measured, the characteristics of the included patients, and the interventions of interest were clearly described. An interrupted time series regression methodology was applied to analyze each outcome separately, which allowed for individually quantifying the effect of each restriction implemented during the study period. The study did not include previous training for the practice change and both restrictions and education of the multidisciplinary team involved in the practice change occurred simultaneously. It was unclear whether this limitation influenced the outcomes. The study was designed with a series of rapid successive medication restrictions implemented by the opioid workgroup. Given the heterogeneous nature of opioid use, the duration of each phase might not have been sufficient to allow for recording noticeable changes in opioid use. It was likely that nonsignificant changes after some interruptions were evidence of insufficient time to detect noticeable changes in the model. In addition, the study design was limited to a university hospital, and the list of opioid medications used was not all-inclusive. The study findings cannot provide a full picture of opioid use, and the effect of shortages and restrictions on opioid product use might differ in other health care systems and clinical settings. Thus, the findings of this study should be interpreted with caution, and their generalizability to other institutions is unclear.

Overall, the limitations of both studies based on chart reviews for data extraction (including inconsistency in administrative data recording and missing data) and the risk of selection bias should be acknowledged.15

Additional details regarding the strengths and limitations of the included publications are provided in Appendix 3

Summary of Findings

Appendix 4 presents the main outcomes of the included studies consisting of changes in codeine-containing prescriptions (Table 5) and opioid utilization (Table 6).

Changes in Codeine-Containing Prescriptions

In the retrospective study by Vyhmeister and Sierra (2022),12 the total number of codeine-containing prescriptions before (between February 1, 2017, and April 19, 2017, 2 months and 19 days) and after (between April 20, 2017, and April 30, 2018, 12 months and 10 days) the FDA soft stop were 205 and 210, respectively. After the FDA soft stop, there was a reduction in both the median number of prescriptions per month (73 vs. 14) and the median days of treatment (6.8 vs. 6.0) compared to the baseline period. Postintervention, percentages of total codeine prescriptions for postoperative pain and cough declined, whereas those for acute pain (new onset pain, not associated with recent surgery) and chronic pain (recurrent, associated with a chronic condition such as cancer or sickle cell disease) increased. Before the intervention, the orthopedic surgery department contributed to the largest share of codeine-containing prescriptions with 30 prescriptions per month, which reduced to 0 to 2 prescriptions per month within 3 months after the FDA warning. A similar trend of reduced prescriptions also was reported in the emergency medicine, plastic surgery, and pediatric surgery departments. However, in the hematology and oncology departments, the rate of codeine-containing prescriptions was relatively similar before and after the FDA soft-stop order. None of the measurements before and after the implementation of the soft stop were compared statistically.

Changes in Opioid Use

In the retrospective study by Brokenshire and colleagues (2022),11 changes in intermittent oral and injectable opioid product administration, total institutional opioid administration, and the utilization of individual restricted opioid drugs were reviewed.

Limitations

Current evidence on ASOs for opioid-containing prescriptions is limited. There were only 2 observational, retrospective studies11,12 that met the selection criteria regarding the impact of ASOs on some opioid prescriptions in acute care settings. There were no identified publications that reported on several outcomes of interest, including clinical benefits (e.g., reduction in opioid dependence, misuse, tolerance; duration of therapy; discharge prescriptions; health-related quality of life) and harms (e.g., adverse events) following ASOs for a list of common opioid medications. Because of the single-centre design of 1 of the included publications,11 the study populations might not be representative of all patients who receive opioid treatment. Thus, the findings reported should be interpreted with caution, and their generalizability to other institutions is uncertain. In addition, the 2 included studies were conducted in the US, so the findings’ generalizability to the Canadian population are uncertain. In response to the second research question, no evidence-based guidelines were identified that met the criteria for this review on the use of ASOs for opioids or controlled drugs.

Conclusions and Implications for Decision- or Policy-Making

We found 2 observational, retrospective studies on the clinical effectiveness of ASOs for opioid prescriptions in acute care settings. In 1 of the publications,12 the authors studied the effect of the FDA soft-stop order on codeine prescribing practices in an academic health system. The restriction was associated with reduced codeine prescriptions for pediatric patients younger than 12 years of age. The authors suggested using hard-stop alerts for medical departments with a slow response or no response to the soft stop. In the second publication,11 the authors studied the effect of a series of consecutive ASOs (a combination of soft-stop and hard-stop orders) on the institutional use of opioid medications during a critical drug shortage period. The 10 discrete phases of medication restrictions ordered in this study did not have a significant effect on the intermittent use of oral and injectable opioid products, the total institutional opioid administration, or the use of individual restricted opioid drugs (except for injectable hydromorphone). The design of the 2 included studies limited the settings to a hospital11 and a single health system12 and did not include a complete list of existing opioid-containing medications. In addition, we did not find any publications regarding a wide range of clinical benefits (e.g., reduction in opioid dependence, misuse, tolerance; duration of therapy; discharge prescriptions; health-related quality of life) and harms (e.g., adverse events) associated with opioid medication restrictions. Therefore, current evidence does not tell us how ASOs might work for a list of different opioid medications in other institutions and at the patient level. Taken together, given the limited number and quality of the studies, the absence of evidence-based guidelines regarding the use of ASOs for opioid prescriptions, and the lack of eligible studies on the Canadian population, it is difficult to draw any firm conclusions about the clinical effectiveness of ASOs for opioid prescriptions in the Canadian context. Additional research with rigorous methodological approaches is required to understand the clinical benefits and harms of ASOs for opioid prescriptions and to support stakeholders in decision-making.

References

1.Tighe P, Buckenmaier CC, 3rd, Boezaart AP, et al. Acute pain medicine in the United States: A status report. Pain Med. 2015;16(9):1806-1826. PubMed

2.Dowell D, Ragan K, Jones C, Baldwin G, Chou R. CDC clinical practice guideline for prescribing opioids for pain — United States, 2022. MMWR Recomm Rep. 2022;71(3):1-95. PubMed

3.Canadian Institute for Health Information. Opioid prescribing in Canada: How are practices changing? 2019; https://www.cihi.ca/sites/default/files/document/opioid-prescribing-canada-trends-en-web.pdf. Accessed 2023 Jan 31.

4.Health Canada. Restricting the marketing and advertising of opioids. 2021; https://www.canada.ca/en/health-canada/services/opioids/responding-canada-opioid-crisis/advertising-opioid-medications.html. Accessed 2023 Feb 2.

5.Health Canada. Notice to stakeholders: Further restrictions to the marketing and advertising of opioids. 2019; https://www.canada.ca/en/health-canada/services/drugs-health-products/drug-products/announcements/additional-restrictions-advertising-opioids.html. Accessed 2023 Feb 3.

6.Lowenstein M, Grande D, Delgado MK. Opioid prescribing limits for acute pain - striking the right balance. N Engl J Med. 2018;379(6):504-506. PubMed

7.Health Quality Ontario. Opioid prescribing for acute pain: Care for people 15 years of age and older. 2018; https://www.hqontario.ca/portals/0/documents/evidence/quality-standards/qs-opioid-acute-pain-clinician-guide-en.pdf. Accessed 2023 Jan 31.

8.Grissinger M. Small effort, big payoff: Automated maximum dose alerts with hard stops. P T. 2016;41(2):82-128. PubMed

9.Strom BL, Schinnar R, Aberra F, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010;170(17):1578-1583. PubMed

10.Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of Epidemiology and Community Health. 1998;52(6):377-384. PubMed

11.Brokenshire SA, Lemon SJ, Staley B, Voils A, Hincapie-Castillo JM. Impact of opioid restrictions during a critical drug shortage period: Interrupted time series for institutional opioid utilization. Pain Med. 2021;22(1):203-211. PubMed

12.Vyhmeister K, Sierra CM. Codeine prescribing practices before and after the 2017 FDA warning at an academic health system. J Am Pharm Assoc (2003). 2022;10(S1544-3191(22)00405-8).

13.Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of Clinical Epidemiology. 2009;62(10):e1-e34. PubMed

14.Powers EM, Shiffman RN, Melnick ER, Hickner A, Sharifi M. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: A systematic review. J Am Med Inform Assoc. 2018;25(11):1556-1566. PubMed

15.Prada-Ramallal G, Takkouche B, Figueiras A. Bias in pharmacoepidemiologic studies using secondary health care databases: A scoping review. BMC Med Res Methodol. 2019;19(1):53. PubMed

Appendix 1: Selection of Included Studies

Figure 1: Selection of Included Studies

A total of 522 citations were identified, 518 were excluded, and 4 electronic literature and 0 grey literature potentially relevant full-text reports were retrieved for scrutiny. In total, 2 reports were included in the review.

Appendix 2: Characteristics of Included Publications

Note that this appendix has not been copy-edited.

Table 3: Characteristics of Included Primary Clinical Studies

Study citation, country, objective, funding source

Study design, date of data collection

Population characteristics

Intervention and comparator(s)

Clinical outcomes, length of follow-up

Vyhmeister and Sierra (2022)12

Country: US

Objective: To describe changes in codeine prescribing practices after the FDA “soft stop” order in 2017

Funding source: No funding

A retrospective chart review of 4 hospitals in an academic health system

Date of data collection:

Before (between February 1, 2017, and April 19, 2017) and after (between April 20, 2017, and April 30, 2018) FDA “soft stop” order

Pediatric patients < 12 years of age with prescriptions for codeine-based products

Exclusion criterion:

Patients were excluded if codeine-containing prescriptions were discontinued before the end of the relevant encounter.

Intervention: The FDA “soft-stop” restriction on codeine-containing products

Comparator: Baseline period before the “soft stop” order

Outcome:

Rate of codeine prescriptions

Follow-up: NA

Brokenshire et al. (2021)11

Country: US

Objective: To evaluate the impact of oral and injectable opioid product restrictions on inpatient opioid utilization

Funding source: Not reported

A single-centre retrospective chart review

Date of data collection:

Between July 2017 and June 2018

44,401 adults, the mean age of 51 years (standard deviation = 17.6)

Patient subgroups at the initial encounter, n (%):

  • Inpatient = 23,786 (53.6)

  • Emergency = 12,402 (30.2)

  • Ambulatory surgery = 5,125 (11.5)

  • Observation = 2,088 (4.7)

Inclusion criterion: Receiving at least one opioid product (IV or oral route) outside of the operating room setting

Exclusion criterion: Those who were admitted to a pediatric medical service

Intervention: 10 discrete phases of “soft stop” or “hard stop” orders after a baseline time period

Comparator: Baseline period before ordering sequential restrictions

Outcomes:

  • Changes in intermittent oral and injectable opioid product administration

  • Changes in total institutional opioid administration

  • Changes in the use of individual restricted opioid medications

Follow-up: NA

NA = not applicable.

Appendix 3: Critical Appraisal of Included Publications

Note that this appendix has not been copy-edited.

Table 4: Strengths and Limitations of Clinical Studies Using the Downs and Black Checklist10

Strengths

Limitations

Vyhmeister and Sierra (2022)12

  • The objective, the main outcome to be measured, the characteristics of the included patients, and the intervention of interest were clearly described.

  • Evidence of data dredging was not observed.

  • A similar time frame was not considered before (almost 2 months and 19 days) and after (12 months and 10 days) the FDA “soft stop” order in April 2017. Thus, the impact of considering a longer time period postintervention on outcomes was unclear.

  • The authors reviewed the records of codeine-based product prescriptions of 4 hospitals in an academic health system, 1 inpatient and 3 outpatients. The outcomes were not reported in subgroups, allowing the classification of results into inpatient and outpatient hospitals.

  • The study design was limited to an academic health system, and the role of outside clinics, pharmacies, or other resources in providing and filling opioid product prescriptions after the FDA “soft stop” was unclear.

  • Given the study design, the included patients might not have been representative of the entire population eligible to be considered, and the generalizability of the findings was uncertain.

  • No statistical tests were used to compare changes in codeine-containing prescriptions before and after the FDA “soft stop” order, and the entire findings were reported descriptively. Thus, it was unclear whether the outcomes reported were statistically significant.

  • The limitations of studies framed based on chart reviews in data extraction (including inconsistency in administrative data recording and missing data) and the risk of selection bias should be considered.

Brokenshire et al. (2021)11

  • The objective, the main outcomes to be measured, the characteristics of the included patients, and the interventions of interest were clearly described.

  • The main findings, estimates of random variability for the main outcomes, and actual probability values were described.

  • The statistical tests used for data analysis were appropriate. For each outcome, an ITS regression methodology was used that allowed for quantifying the impact of each restriction implemented separately. ITS is less prone to biases and is the preferred methodology for evaluations of policies over time.

  • Evidence of data dredging in the main outcomes was not observed.

  • The study design consisted of the rapid succession of ordering 10 restrictions. Given the heterogeneous nature of opioid use, the short duration of each phase might not have been sufficient to detect a noticeable change and might have contributed to some nonsignificant increases and decreases after some phases.

  • Both opioid restrictions and education of the multidisciplinary team involved in the practice change occurred simultaneously, which might have influenced the outcomes.

  • The list of opioids prescribed for acute pain was not all-inclusive and may not well reflect the full picture of opioid use in other clinical settings. Namely, the impact of shortages and the effects of restrictions on opioid product use might differ in other health care systems, and the generalizability of the findings is uncertain.

  • The shortcomings of studies based on chart reviews for data extraction (e.g., inconsistency in administrative data recording, misclassification bias in outcome events, and missing data) and the risk of selection bias should be acknowledged.

ITS = interrupted time series.

Appendix 4: Main Study Findings

Note that this appendix has not been copy-edited.

Table 5: Summary of Findings by Outcome in the Study by Vyhmeister and Sierra (2022)12 — Codeine-Containing Prescriptions

Variable

Time point

Baseline

(2 months and 19 days)

Post “soft stop”

(12 months and 10 days)

Number of codeine-containing prescriptions

    Total

205

210

    Median (IQR)

73 (65 to 82)

14 (10 to 17)

Days of treatment, median (IQR)

6.8 (5.0 to 11.8)

6.0 (5.0 to 8.0)

Number of prescriptions with refills (%)

3 (1.5)

15 (7.1)

Total number of codeine-containing prescriptions by indication (%)

    Postoperative pain

142 (69.3)

108 (51.4)

    Cough

39 (19.0)

25 (11.9)

    Acute pain

14 (6.8)

24 (11.4)

    Chronic pain

10 (4.9)

53 (25.2)

Total number of codeine-containing prescriptions by prescriber specialty (% of total)

    Orthopedic surgery

83 (41.5)

53 (25.2)

    Emergency medicine

28 (13.7)

12 (5.7)

    Plastic surgery

25 (12.2)

4 (1.9)

    Pediatric surgery

18 (8.8)

6 (2.9)

    Hematology and oncology

12 (5.9)

49 (23.3)

    Family medicine

8 (3.9)

2 (1.0)

    Urology

1 (0.5)

2 (1.0)

    Dentistry, oral and maxillofacial surgery

1 (0.5)

16 (7.6)

    Podiatry

1 (0.5)

0 (0)

    Neurology and/or neurosurgery

0 (0)

6 (2.9)

    Gastroenterology

0 (0)

4 (1.9)

    Pain and/or palliative

0 (0)

2 (1.0)

    Rheumatology

0 (0)

2 (1.0)

    Obstetrics and gynecology

0 (0)

2 (1.0)

IQR = interquartile range.

Table 6: Summary of Findings by Outcome in the Study by Brokenshire et al. (2021)11 — Changes in Institutional Opioid Utilization

Outcome

Overall change, MME per patient per day (95% CI)

Study phases with significant change, MME per patient per day (95% CI)

Outcome 1a

−0.02 (−0.05 to 0.002)

Phase 3 = −6.96 (−12.78 to −1.15)

Phase 5 = 7.44 (0.31 to 14.56)

Outcome 2b

0.02 (−0.11 to 0.14)

Phase 4 = 29.87 (0.15 to 59.59)

Outcome 3c

Morphine = 0.0004 (−0.003 to 0.002)

Phase 1 = −0.84 (−1.68 to −0.004)

Phase 10 = −1.60 (−2.25 to −0.95)

Fentanyl = 0.004 (−0.003 to 0.01)

Phase 6 = 2.29 (0.39 to 4.19)

Hydromorphone = −0.007 (−0.01 to −0.002)

Phase 1 = −2.89 (−4.46 to −1.32)

Phase 5 = −1.99 (−3.37 to −0.61)

CI = confidence interval; MME = morphine milligram equivalent.

Note: A total of 258,221 unique medications were ordered for 44,401 patients.

aOutcome 1: Changes in intermittent oral and injectable opioid product administration.

bOutcome 2: Changes in total institutional opioid administration.

cOutcome 3: Changes in the utilization of individual restricted opioid drugs.

Appendix 5: References of Potential Interest

The following publications were identified because they may provide some information associated with this report, including nonrandomized studies on alternative populations or interventions, guidelines, and review articles.

Systematic Reviews

Powers EM, Shiffman RN, Melnick ER, Hickner A, Sharifi M. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: A systematic review. J Am Med Inform Assoc 2018;25(11):1556–66. PubMed

Nonrandomized Studies

Alternative Population (Outpatients)

Lowenstein M, Hossain E, Yang W, et al. Impact of a state opioid prescribing limit and electronic medical record alert on opioid prescriptions: A difference-in-differences analysis. J Gen Intern Med. 2020;35(3):662-671. PubMed

Kohler JE, Cartmill RS, Kalbfell E, Schumacher J. Continued prescribing of periprocedural codeine and tramadol to children after a black box warning. J Surg Res. 2020;256:131e135.

Alternative Intervention

Cowart MC, Miller D, Laham FR, Jordan-Villegas A. Implementation of an automatic 48-hour vancomycin hard-stop in a pediatric community hospital. J Pediatr Pharmacol Ther. 2022;27(2):147-150. PubMed

Guidelines

Fox ER, McLaughlin MM. ASHP guidelines on managing drug product shortages. Am J Health Syst Pharm. 2018;75 (21):1742–50. PubMed

Clark J, Fera T, Fortier C, et al. ASHP guidelines on preventing diversion of controlled substances. Am J Health Syst Pharm. 2022;79(24):2279-2306. PubMed

Review Articles

Report of the ASHP Opioid Task Force. Am J Health Syst Pharm. 2020;77(14):1158-1165. PubMed

Vo P, Sylvia DA, Milibari L, et al. Management of a parenteral opioid shortage using ASHP guidelines. Am J Health Syst Pharm. 2021;78(5):426-435. PubMed