Viewpoint

Development of an Evaluation Instrument on Artificial Intelligence Search Tools for Evidence Synthesis

Nicole Mittmann

In this issue, we share how Canada’s Drug Agency (CDA-AMC) evaluates Artificial Intelligence (AI) technologies that have the potential to enhance the production of our reports. The Research Information Services team at CDA-AMC created a tool to evaluate mandatory and desirable characteristics for suitable AI tools that could assist with information retrieval tasks. This work resulted in the development of an instrument than can help with evaluation and continuous monitoring in this rapidly evolving AI space.

During a period of exponential growth in AI technologies, we operationalized an approach to evaluate the utility and appropriateness of these tools for work by CDA-AMC. To further promote the judicious selection of AI tools for information retrieval, this report shares the evaluation instrument to assist all evidence synthesis producers interested in evaluating and making use of new technologies.