Efficacy of clinical decision support systems: methods and estimates

Clinical decision support (CDS) systems are the medical technologies that go through their life cycle. Evaluation of effectiveness and safety should be carried out at its various stages – at the development, in clinical trials, licensing, clinical and economic analysis, health technologies assessment. To date, the effectiveness and safety of CDS systems vary and are ambiguous – there are both successes and failures. Hundreds of clinical trials are carried out, and more than a hundred of systematic reviews are published. When evaluating the efficacy and safety of CDS systems, two types of outcomes are usually estimated: indicators of medical care (volume, time, costs, etc.), and patient outcomes (clinical and surrogate). A slight increase in physicians’ adherence to clinical guidelines has been observed, but it had very small influence on surrogate outcomes, and there is no effect on clinical patient outcomes. A slight increase in risk with respect to patient outcomes was found in only a few studies. However, the methodological quality of the evidence is very low. In this regard, a few products based on artificial intelligence have so far approached the licensing phase. The field of CDS systems is developing, but not yet sufficiently studied, and there is a long way to real successes ahead. Meanwhile, there is a wide gap between the postulated and empirically demonstrated benefits of CDS systems.

1. Гусев А.В., Зарубина Т.В. Поддержка принятия врачебных решений в медицинских информационных системах медицинской организации // Врач и информационные технологии. 2017. №2. С. 60-72. [Gusev A.V., Zarubina T.V. Clinical Decisions Support in medical information systems of a medical organization // Vrach i informacionnye tehnologii. 2017;(2):60-72. (In Russ.)]

2. Реброва О.Ю. Жизненный цикл систем поддержки принятия врачебных решений как медицинских технологий // Врач и информационные технологии. 2020. №1. С. 27-37. [Rebrova O.Yu. Life cycle of decision support systems as medical technologies // Vrach i informacionnye tehnologii. 2020;(1): 27-37. (In Russ.)]

3. Письмо Росздравнадзора “О программном обеспечении” от 13.02.2020. [Pis’mo Roszdravnadzora “O programmnom obespechenii” ot 13.02.2020. (In Russ.)] https://www.roszdravnadzor.ru/i/upload/images/ 2020/2/14/1581670651.93473-1-10822.pdf

4. Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1):e1000387. https://doi.org/10.1371/journal.pmed.1000387.

5. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems. A systematic review. Ann Intern Med. 2012;157:29-43. doi: 10.7326/0003-4819-157-1-201207030-00450.

6. Moja L, Kwag KH, Lytras T, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014; 104(12):e12–e22. https://doi.org/10.2105/AJPH.2014.302164.

7. Varghese J, Kleine M, Gessner SDI, et al. Effects of computerized decision support system implementations on patient outcomes in inpatient care: a systematic review. J Am Med Inform Assoc. 2018;25(5):593-602. https://doi.org/10.1093/jamia/ocx100.

8. Pombo N, Araújo P, Viana J. Knowledge discovery in clinical decision support systems for pain management: a systematic review. Artif Intell Med. 2014;60(1):1-11. https://doi.org/10.1016/j.artmed.2013.11.005.

9. Tomaselli Muensterman E, Tisdale JE. Predictive analytics for identification of patients at risk for QT interval prolongation: a systematic review. Pharmacotherapy. 2018;38(8):813-821. https://doi.org/10.1002/phar.2146.

10. Arani LA, Hosseini A, Asadi F, et al. Intelligent computer systems for multiple sclerosis diagnosis: a systematic review of reasoning techniques and methods. Acta Inform Med. 2018;26(4):258-264. https://doi.org/10.5455/aim.2018.26.258-264.

11. Cresswell K, Callaghan M, Khan S, et al. Investigating the use of data-driven artificial intelligence in computerised decision support systems for health and social care: a systematic review. Health Informatics J. 1460458219900452, 2020 Jan 22 [Online ahead of print]

12. Dissanayake PI, Colicchio TK, Cimino JJ. Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis. J Am Med Inform Assoc. 2020;27(1):159-174. https://doi.org/10.1093/jamia/ocz169.

13. Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. https://doi.org/10.1016/S2589-7500(19)30123-2.

14. Electronic health records vendor to pay largest criminal fine in Vermont history and a total of $145 million to resolve criminal and civil investigations. Department of Justice, U.S. Attorney’s Office, District of Vermont // https://www.justice.gov/usao-vt/pr/electronic-health-records-vendor-pay-largest-criminal-fine-vermont-history-and-total-145.

15. Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 346, f657, 2013 Feb 14. https://doi.org/10.1136/bmj.f657.

16. Государственный реестр медицинских изделий и организаций (индивидуальных предпринимателей), осуществляющих производство и изготовление медицинских изделий. [Gosudarstvennyj reestr medicinskih izdelij i organizacij (individual’nyh predprinimatelej), osushhestvljajushhih proizvodstvo i izgotovlenie medicinskih izdelij. (In Russ.)] https://www.roszdravnadzor.ru/services/misearch