Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study - PubMed

9 min read Original article ↗

Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study

Steindór Ellertsson et al. Ann Fam Med. 2023 May-Jun.

Abstract

Purpose: Respiratory symptoms are the most common presenting complaint in primary care. Often these symptoms are self resolving, but they can indicate a severe illness. With increasing physician workload and health care costs, triaging patients before in-person consultations would be helpful, possibly offering low-risk patients other means of communication. The objective of this study was to train a machine learning model to triage patients with respiratory symptoms before visiting a primary care clinic and examine patient outcomes in the context of the triage.

Methods: We trained a machine learning model, using clinical features only available before a medical visit. Clinical text notes were extracted from 1,500 records for patients that received 1 of 7 International Classification of Diseases 10th Revision codes (J00, J10, JII, J15, J20, J44, J45). All primary care clinics in the Reykjavík area of Iceland were included. The model scored patients in 2 extrinsic data sets and divided them into 10 risk groups (higher values having greater risk). We analyzed selected outcomes in each group.

Results: Risk groups 1 through 5 consisted of younger patients with lower C-reactive protein values, re-evaluation rates in primary and emergency care, antibiotic prescription rates, chest x-ray (CXR) referrals, and CXRs with signs of pneumonia, compared with groups 6 through 10. Groups 1 through 5 had no CXRs with signs of pneumonia or diagnosis of pneumonia by a physician.

Conclusions: The model triaged patients in line with expected outcomes. The model can reduce the number of CXR referrals by eliminating them in risk groups 1 through 5, thus decreasing clinically insignificant incidentaloma findings without input from clinicians.

Keywords: artificial intelligence; clinical decision support systems; primary care; respiratory symptoms; triage.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.

Study flowchart for clinical text note selection. CFEM = clinical feature extraction model; CTN = clinical text note; PC = primary care.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 2.
Figure 2.

The outcome distribution in the cross-validated data set. CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma. Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 3.
Figure 3.

The outcome distribution in test set 1. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Figure 4.
Figure 4.

The outcome distribution in test set 2. CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care. Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

References

    1. OECD . Health spending. OECD Data. Accessed Mar 21, 2022. https://data.oecd.org/healthres/health-spending.htm
    1. Svedahl ER, Pape K, Toch-Marquardt M, et al. . Increasing workload in Norwegian general practice - a qualitative study. BMC Fam Pract. 2019; 20(1): 68. 10.1186/s12875-019-0952-5 - DOI - PMC - PubMed
    1. Hobbs FDR, Bankhead C, Mukhtar T, et al. ; National Institute for Health Research School for Primary Care Research . Clinical workload in UK primary care: a retrospective analysis of 100 million consultations in England, 2007-14. Lancet. 2016; 387(10035): 2323-2330. 10.1016/S0140-6736(16)00620-6 - DOI - PMC - PubMed
    1. NHS . Long Term Conditions Compendium of Information: third edition. Published May 30, 2012. Accessed Mar 21, 2022. https://assets.publishing.service.gov.uk/government/uploads/system/uploa...
    1. Deloitte . Under pressure: the funding of patient care in general practice. Published Apr 2, 2014. Accessed Mar 21, 2022. https://www.queensroadpartnership.co.uk/mf.ashx?ID=406a083a-144f-457d-b1...

Publication types

MeSH terms