Pubmed
Crossref
Development and validation of a predictive scoring system for in-hospital mortality in COVID-19 Egyptian patients: a retrospective study
Abstract
SARS-CoV-2 virus has rapidly spread worldwide since December 2019, causing COVID-19 disease. In-hospital mortality is a common indicator for evaluating treatment outcomes. Therefore, the developing and validating a simple score system from observational data could assist in modulating the management procedures. A retrospective cohort study included all data records of patients with positive PCR for SARS-CoV-2. The factors that associated with mortality were analyzed, then allocation of potential predictors of mortality was executed using different logistic regression modeling, subsequently scoring system was developed from the most weighted predictors. The mortality rate of patients with COVID-19 pneumonia was 28.5% and 28.74%, respectively. The most significant factors that affected in-hospital mortality were old age (> 60 years), delay in hospital admission (> 4 days), high neutrophil/lymphocyte ratio “NLR” (> 3); higher computed tomography severity score; and CT-SS (> 20), in addition to using remdesivir and tocilizumab in the treatment protocol (P < 0.001 for all). The validity of the newly performed score was significant; the AUC was 85%, P < 0.001, and its prognostic utility was good; the AUC was 75%, P < 0.001. The prognostic utility of newly developed score system (EGY.Score) was excellent and could be used to adjust the treatment strategy of highly at-risk patients with COVID-19 pneumonia.
Introduction
COVID-19 is a current pandemic disease caused by SARS-CoV-2 which is associated with serious health complications worldwide1. As of October 28, 2022, there were 626,337,158 confirmed cases of COVID-19, including 6,566,610 deaths, reported to the WHO2. The severity of COVID-19 infection ranges from asymptomatic manifestations to flu-like symptoms such as fever, headache, dry cough, dyspnea, mayalgia, and joint pain, eventually leading to fatal acute respiratory distress syndrome (ARDS)3,4,5. According to previous studies, the mortality rate for the most serious SARS-CoV-2 infections that require admission to the intensive care unit (ICU) ranges from 8.1 to 30% for hospitalized patients with SARS-CoV-2 pneumonia and up to 16% to 78% for patients who require admission to an ICU with critical care4,6,7,8.
According to previous studies, the COVID-19 outbreak is not uniform across nations, with notable variations in the proportion of serious diseases and case fatality rates9. Multicenter reports emphasize that patient-specific characteristics are important predictors of the presentation and consequences of COVID-19, even though the quality of healthcare services may be a factor in such variations10. Since the WHO officially declared a global pandemic in March 2020, there have been significant efforts to identify prognosticators that clinicians use to evaluate the risk at the early stage of the illness. This has helped to better tailor management strategies, assist decision-making, and promote health for COVID-19 patients by raising the therapeutic response, increasing the diagnostic accuracy, and lowering the case fatality rate11.
Various studies found that advanced age12,13,14,15, male gender13, and co-morbidities13,14,15 such as diabetes mellitus (DM), obesity, systemic hypertension (HTN), renal diseases, coronary artery disease15, and malignancy were risk factors for COVID-19 mortality. Beyond manifestations such as fever, cough16, haemoptysis12, dyspnoea12,16, unconsciousness12, clinical parameters such as elevated neutrophil/lymphocyte ratio12,17, and high levels of creatinine18, elevated lactate dehydrogenase (LDH)12,14,15,18, direct bilirubin12, and alanine amino-transferase (AST)18, which indicate early evidences to the severity of disease, an increased plasma level of biomarkers like d-dimer15,18,19,20, C-Reactive Protein (CRP)21,22, serum ferritin18, Interleukin-6 (IL-6)18, and procalcitonin (PCT) strengthens these findings18,19,20. The efficacy of particular antiviral and targeted immuno-modulatory therapies is still uncertain, therefore risk categorization and mortality prediction offer a logical strategy for allocating medical resources23.
The value of developing a score system for predicting the upcoming prognosis as mortality is vastly important for health care providers, especially doctors, to be more reliable and objective in evaluating their patients rather than being subjective24,25. Additionally, the recent publications in the medical field and beyond focus on the quality of the scoring system model and the methods of formation either through using strict and dedicated factors during statistical analysis steps or using machine learning programs25,26,27,28. Therefore, in order to estimate the probability of mortality in patients with COVID-19 pneumonia, forming a simple custom score could enhance the objective decision-making around treatment selection, as well as teach from trial and error in dealing with such a new disease. Our aim of the study was to develop and validate a simple scoring system to predict in-hospital mortality among patients with COVID-19 pneumonia within the first days of hospitalization, which was based mainly on different presenting symptoms, comorbidities, vital signs, and some laboratory data in addition to some local and affordable treatment options entered into the regression model. The multiple logistic regression model approaches were applied to our set of data to find the best equation that predicts the death probability. Furthermore, to measure the validity of that equation, a receiver operating characteristic curve was applied. Hence, the area under the curve expresses the goodness of that model in the prediction of mortality.
Material and methods
Study design and patient selection
A retrospective cohort study included all data records of patients with positive PCR for SARS-CoV-2 who was admitted to Almaza Military Hospital from January 2020 to the end of December 2021. A flow diagram showed the criteria of selected patients (Fig. 1). Hence, only 1535 patients had been considered as suggested candidates for primary study criteria. All of them had positive oropharangeal PCR tests in addition to CT chest. Those below 18 years old and those with free CT findings were excluded from the study. As well, cases with incomplete data records were also excluded. After informed written consent from all participants for publication of their data, the study has been approved by the Research Ethics Committee of the Faculty of Pharmacy, Tanta University (REC-TP code: TP/RE/012-21P-005) and the ethical committee office of the Medical Military Academy in agreement with the Helsinki Declaration Roles.
Data collection
The data was extracted from the patient's files and included basic socio-demographic data in addition to the presenting symptoms. Only laboratory data that was conducted in the first 24 h after admission was considered. The evaluation of CT chest and quantifying CT-SS as regarding Yang et al.29, which was performed by two independent radiologists for reliability. The data about treatment protocol as regarding local guidelines was also included.
Outcome
The outcome of desire was in-hospital mortality from COVID-19 pneumonia, which is defined as death during a period of admission to a hospital as a consequence of COVID-19 disease. Expectation of a short-term death after discharge from the hospital is also considered in-hospital mortality30.
Statistical analysis
Building the predictive model and development of scoring system
All the data was collected and coded on an Excel sheet. The normality of data was examined using the Shapiro–Wilk test using SigmaPlot for Windows version 12.5. 0.38 (Systat Software, Inc., UK, 2011). The descriptive statistics were performed using Minitab 17.1.0.0 for Windows (Minitab Inc., 2013, Pennsylvania, USA). In the first step, all data was subjected to univariate analysis. Hence, the comparison between two means was done using an independent t-test, while the frequency comparison was made using the chi-square test. Regarding the Neyman-Pearson theory of classical statistics31,32, the critical value of a significant number was selected by the observer to guarantee that the type II errors were minimized as much as possible and circumvent the interrelationship between type I and type II errors. Additionally, Park, 201333 mentioned that all variables could be implicated to test their correlation with the final outcome, provided that the log equation of fitness was acceptable; however, some authors preferred to select the variable of desire to be enrolled in the logistic equation after one step of simple hypothesis testing (univariate analysis)34,35, consuming that the margin of type II error was not so far away to accept the null hypothesis36. Therefore, in the current models, all factors with a p-value ≤ 0.35 were subjected to the second step, multivariate analysis. During building the logistic regression models, and to avoid multicollinearity, factors with VIF > 5 were eliminated. The goodness of fit for the regression model was performed using the Hosmer and Lemeshow test. Forward selection and backward elimination techniques were applied to select the best predictors for mortality. The Wald statistics numbers for each factor of interest were multiplied by the coefficient and divided by a constant value, and the result was rounded to the nearest integer number, taking into account the sign of the regression coefficient. According to the literature, all numerical data are transformed into categorical groups based on the distribution histogram and cumulative frequency or normal reference. The total score is then calculated from the submission of each individual score.
Score validation
The data for all factors implicated in the scoring system (SOM) was collected from the validation cohort, while the individual score and total score for every participant were calculated automatically in an Excel sheet. The performance of the total score was assessed using ROC curve analysis. The AUC above 0.6 was considered acceptable. The logistic regression analysis was finally performed to calculate the equation of death probability from total SOM. All tests were two-sided and a p-value of less than 0.05 was considered significant.
Institutional review board statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by after approval by the Research Ethics Committee of Faculty of Pharmacy, Tanta University (REC-TP code: TP/RE/012-21P-005) and the ethical committee office of the Medical Military Academy.
Informed consent statement
Informed consent was obtained from all subjects involved in the study.
Results
Patients' characteristics
The total number of data points was 316, and the mortality rate of patients with COVID-19 pneumonia was 28.5%. As shown in Table 1, the basic criteria of both the survival and mortality groups were presented. One third of cases were male, and more than half of them were older than 60 years old. The most frequent comorbidities were DM and HTN, and nearly all patients were presented with cough and dyspnea. However, fever was a significant cardinal sign that correlated with mortality. The duration of complaints in the mortality group was significantly shorter than the survival one, as more than half of them came with a complaint history of less than 4 days. In the mortality group, the CT-SS was significantly higher and more than 2/3 of cases had severe and very severe degrees of CT scores (> 20). Moreover, neutrophil (%), NLR and PLR, as well as AST, urea, creatinine and IL-6 were significantly higher in the mortality group.
Table 1 Socio-demographic data and clinical characteristics.
Predictors of mortality
After selection of all factors in univariate analysis with a p-value ≤ 0.35, multivariate analysis with different approaches was applied to select the most significant factors that affected in-hospital mortality. As shown in Table 2, being elderly (> 60 years), having a shorter duration of complaint, having a high NLR, and having a higher CT-SS (> 20) were all significant independent predictors of mortality (P < 0.05 for all).
Table 2 Predictors of mortality using multivariate logistic regression model with different technique.
Score model formation and its performance
After collecting the predictors from the previous step, a multiple logistic regression analysis were applied to weight the factors and forming the scoring number based on both WS and Coefficient number. As shown in Table 3, two different scores could be used; the first one (SOM-1) including tocilizumab, and the second one (SOM-2) without it. However, both of them showed insignificant differences in discriminating in-hospital mortality (Fig. 2). Because the AUC was 85% and 84%, respectively, with P = 0.001 for both, SOM-1 was chosen for further validation.
Table 3 Development of scoring system based on best predictors for mortality.
Assessment of the new score: validation and prognostic utility
About 327 patients were subjected to score validation; the descriptive statistics of the validation cohort were summarized in Table 4, in which the mortality rate was 28.74%.
Table 4 Characters of validity cohort.
Moreover, Fig. 3a showed the median (IQR) value of SOM-1 was significantly higher in the mortality group; (13.5 (3–18) than the survival one; 2 (− 0.5–7)), P < 0.001. The prognostic utility of SOM-1 was so good; the AUC was 75%, P < 0.001 (Fig. 3b), and at cutoff values of above 5 and 16.5, the sensitivity and specificity were above 90%, respectively (Table 5). The probability of mortality increased with every unit increase in the SOM-1 (Fig. 3c).
Table 5 Performance of SOM-1 in predicting in-hospital mortality.
Discussion
The current study used the observational data from patients with COVID-19 pneumonia to develop a simple predictive model for further building a scoring system that easily calculates the probability of in-hospital mortality. The algorithms of the predictive models were applied to 316 patients and included simply collected historical, clinical, and laboratory data on the day of admission, which gave our model of prediction the capability to be applied in other medical sectors with an affordable set of data collection and less sophisticated investigation. However, so many scoring systems have been developed since the start of the pandemics to predict different outcomes related to COVID-19 disease. Their sensitivity and specificity ranged between 70 and 100%37,38,39, and the most effective study recorded a model with excellent validity (AUC = 93.8%)20.
The present score, besides its easy and manual calculation from simple data, had the capability for prognosis prediction; the utility of the score was very good enough to be accepted, hence the AUC of 85% in the training cohort and still good when applied to the validating cohort (AUC of 75%), which indicates how much the stability of the score in predicting the disease prognosis made the treatment strategy more powerful if it was applied as early as possible for a much better outcome. Moreover, during building the model, we entered medication that was used in local treatment protocol in response to laboratory results to estimate if these factors were implicated in the final outcome or not. We reported that remdesivir and tocilizumab were significantly correlated with in-hospital mortality. Therefore, two scoring systems were calculated with tocilizumab (SOM-1) and without tocilizumab (SOM-2), with an insignificant difference between both of them. The accuracy was 85 and 84%, respectively (Fig. 2). For that reason, SOM-1 was the choice for further external validity. Another concern that made our newly formed score more stable was that we depended on both the coefficient number and Wald statistic number from the logistic regression model to weight the predictors.
The attendance results did not find any surprising factors that affected the mortality except the duration before hospital admission. Hence, we found that patients with a shorter duration of less than 4 days before admission died, which made the explanation more difficult, and it could be related to the state of denial that the patients caught the disease and suffered from silent prolonged hypoxia. Surprisingly, only one study includes this historical item in the predictive model of COVID-19-related mortality. Henderson et al. reported that a shorter time from symptom onset to hospitalization is associated with a more serious disease and higher mortality40. On the other hand, old age was correlated with bad prognosis, which came in consistence with other reports41,42,43,44,45,46.
Despite the fact that many studies have pointed to the importance of male sex and associated comorbidity in poor prognosis45,46,47,48,49, our study did not find that link after multiple filtration of logistic regression modeling, and a recent study supported that50. Furthermore, the severity level of lesion in HRCT was one of the most reliable predictors; thus, it was associated with a poor prognosis, despite the fact that a few studies used that factor to predict in-hospital mortality outcome50,51,52.
Our study introduced some treatment medications like iverzine53,54, sofosbuvir/ledipasvir55, remdesivir, and tocilizumab to be estimated as predictive factors for mortality in a regression model. iverzine was used following our national COVID-19 management guidelines in its old versions53,54, however it is deleted from the recent version56. Only remdesivir and tocilizumab continued to be linked with mortality after multiple filtrations of the predictors. Hence, remdesivir was found to be protective against a bad prognosis, and early administration could inhibit the replication of viruses and decrease the viral load; it could also counter the process of pathology inside the lung parenchyma that finally led to improvement in the lung lesion57.
On the other hand, our data showed that tocilizumab increased the likelihood of mortality two times more, which could be due to limiting the use of that medication in patients with severe disease, so the risk of its link to a bad prognosis became much higher. Additionally, tocilizumab had been prescribed in COVID-19 patients with an established higher level of IL-6, denoting that a cascade of cytokines storm had been started58,59,60. Although a pooled analysis of systematic reviews on tocilizumab and mortality outcomes found that it was not only protective against bad outcomes, it was also significantly linked to post-drug infection, which led to a poor prognosis from super infection61. That fact could explain the present finding and draw attention to the limited power of using biological therapy in treatment. Nevertheless, several newly published randomized controlled trials (RCTs)62,63,64,65,66,67,68,69,70 and meta-analyses71,72,73 of RCTs have investigated the effects of TCZ as an adjunctive therapy in patients with COVID-19 but have reported inconsistent results. Moreover, there are increasing number of newly available studies regarding TCZ treatment for COVID-19. Therefore, there are still limited real-world data about the effect of TCZ on inflammatory activity in COVID-19 patients74.
Limitations
Even though our study was the first of its kind that was proposed in Egypt, which is a developing country with limited resources, it highlighted the ability of simple data to predict the outcome of COVID-19 patients. However, the study showed some limitations; the first was the types of study design; hence, the main issue with retrospective cohort type studies was the effect of confounders, which could influence the final outcome in an unpredicted way. The second limitation was the single-centre study, which increased the demand for further external validation from other centers. Additionally, the third point was that the study excluded younger individuals < 18 years old during recruitment, which may be an area of future interest. Moreover, the score was constructed for only COVID-19 patients, which made the comparison with other alternative scores much more difficult to apply.
Conclusion
The constructed score (EGY.Score) from the observational data could predict the prognosis of patients with COVID-19 pneumonia, which may possibly be used to adjust the management intervention for further gain of a desirable outcome.
Data availability
All data generated or analyzed during this study are included in this published article and supplementary material.
References
Driggin, E. et al. Cardiovascular considerations for patients, health care workers, and health systems during the COVID-19 pandemic. J. Am. Coll. Cardiol. 75, 2352–2371. https://doi.org/10.1016/j.jacc.2020.03.031 (2020).
WHO. Coronavirus (COVID-19) Dashboard, https://covid19.who.int/ (Accessed 28 Oct 2022).
Shaweno, T. et al. V-2 antibody among individuals aged above 15 years and residing in congregate settings in Dire Dawa city administration, Ethiopia. Trop. Med. Health 49, 55. https://doi.org/10.1186/s41182-021-00347-7 (2021).
Esfahanian, F., Seyedalinaghi, S., Janfaza, N. & Tantuoyir, M. M. Predictors of hospital mortality among patients with COVID-19 in Tehran, Iran. SAGE Open Med. 9, 20503121211051572. https://doi.org/10.1177/20503121211051573 (2021).
Kox, H. L. M. Relative infectuousness of asymptomatic andsymptomatic COVID-19 infections—analytical time table. MPRA Munich Personal RePEc Archive, https://mpra.ub.uni-muenchen.de/108781/1/MPRA_paper_108781.pdf (2021).
Olivas-Martınez, A. et al. In-hospital mortality from severe COVID-19 in a tertiary care center in Mexico City; causes of death, risk factors and the impact of hospital saturation. PLoS ONE 16, e0245772. https://doi.org/10.1371/journal.pone.0245772 (2021).
Oliveira, E. et al. ICU outcomes and survival in patients with severe COVID-19 in the largest health care system in central Florida. PLoS ONE 16, e0249038. https://doi.org/10.1371/journal.pone.0249038 (2021).
Shi, C. et al. Predictors of mortality in patients with coronavirus disease 2019: A systematic review and meta analysis. BMC Infect. Dis. 21, 663 (2021).
European Centre for Disease Prevention and Control. Situation Update Worldwide, as of Week 1, https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases (2021).
Cao, Y. et al. Comparative genetic analysis of the novel coronavirus (2019-nCoV/SARS-CoV-2) receptor ACE2 in different populations. Cell Discov. 6, 11. https://doi.org/10.1038/s41421-020-0147-1 (2020).
Assal, H. H. et al. Predictors of severity and mortality in COVID-19 patients. Egypt. J. Bronchol. 16, 18. https://doi.org/10.1186/s43168-022-00122-0 (2022).
Liang, W. et al. Development and validation of a clinical RiskScore to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern. Med. 180, 1081–1089. https://doi.org/10.1001/jamainternmed.2020.2033 (2020).
Ahmad, S. et al. Epidemiological, clinical, and laboratory predictors of in-hospital mortality among COVID-19 patients admitted in a tertiary COVID dedicated hospital, Northern India: A retrospective observational study. J. Prim. Care Community Health 12, 21501327211041490. https://doi.org/10.1177/21501327211041486 (2021).
Steinberg, E., Balakrishna, A., Habboushe, J., Shawl, A. & Lee, J. Calculated decisions: COVID-19 calculators during extreme resource-limited situations. Emerg. Med. Pract. 22(4 Suppl), CD1–CD5 (2020).
Chauhan, N. K. et al. Predictors of clinical outcomes in adult COVID-19 patients admitted to a tertiary care hospital in India: An analytical cross-sectional study. Acta Biomed. 92, e2021024. https://doi.org/10.23750/abm.v92i3.10630 (2021).
Salunke, A. A. et al. A proposed ABCD scoring system for patient’s self assessment and at emergency department with symptoms of COVID-19. Diabetes Metab. Syndr. 14, 1495–1501. https://doi.org/10.1016/j.dsx.2020.07.053 (2020).
Rai, D., Ranjan, A., Ameet, H. & Pandey, S. Clinical and laboratory predictors of mortality in COVID-19 infection: A retrospective observational study in a Tertiary Care Hospital of Eastern India. Cureus 13, e17660. https://doi.org/10.7759/cureus.17660 (2021).
Zhou, F. et al. Clinical course and risk factors for mortality of adult in patients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 395, 1054–1062. https://doi.org/10.1016/S0140-6736(20)30566-3 (2020).
Li, X. et al. Risk factors for severity and mortality in adult COVID-19 inpatients in Wuhan. J. Allergy Clin. Immunol. 146, 110–118. https://doi.org/10.1016/j.jaci.2020.04.006 (2020).
Shang, Y. et al. Scoring systems for predicting mortality for severe patients with COVID-19. E. Clin. Med. 24, 100426. https://doi.org/10.1016/j.eclinm.2020.100426 (2020).
Knight, S. R. et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: Development and validation of the 4C Mortality Score. BMJ 370, m3339. https://doi.org/10.1136/bmj.m3339 (2020).
Galloway, J. B. et al. A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: An observational cohort study. J. Infect. 81, 282–288. https://doi.org/10.1016/j.jinf.2020.05.064 (2020).
Gopalan, N. et al. Predictors of mortality among hospitalized COVID-19 patients and risk score formulation for prioritizing tertiary care—An experience from South India. PLoS ONE 17, e0263471. https://doi.org/10.1371/journal.pone.0263471 (2022).
Barbini, P., Cevenini, G., Furini, S. & Barbini, E. A naïve approach for deriving scoring systems to support clinical decision making. J. Eval. Clin. Pract. 20(1), 1–6. https://doi.org/10.1111/jep.12064 (2014).
Biagioli, B. et al. A multivariate Bayesian model for assessing morbidity after coronary artery surgery. Crit. Care. 10(3), R94. https://doi.org/10.1186/cc4951 (2006).
Demichelis, F., Magni, P., Piergiorgi, P., Rubin, M. A. & Bellazzi, R. A hierarchical Naive Bayes Model for handling sample heterogeneity in classification problems: An application to tissue microarrays. BMC Bioinform. 7(1), 514. https://doi.org/10.1186/1471-2105-7-514 (2006).
Lasko, T. A., Bhagwat, J. G., Zou, K. H. & Ohno-Machado, L. The use of receiver operating characteristic curves in biomedical informatics. J. Biomed. Inform. 38(5), 404–415. https://doi.org/10.1016/j.jbi.2005.02.008 (2005).
Kuzniewicz, M. W. et al. Variation in ICU risk-adjusted mortality: Impact of methods of assessment and potential confounders. Chest 133(6), 1319–1327. https://doi.org/10.1378/chest.07-3061 (2008).
Yang, R. et al. Chest CT severity score: An imaging tool for assessing severe COVID-19. Radiol. Cardiothorac. Imaging 2, e200047. https://doi.org/10.1148/ryct.2020200047 (2020).
Jeong, J. et al. Development and validation of a scoring system for mortality prediction and application of standardized W statistics to assess the performance of emergency departments. BMC Emerg. Med. 21, 71. https://doi.org/10.1186/s12873-021-00466-8 (2021).
Clark, C. A. & Chapter, I. Hypothesis testing in relation to statistical methodology. Rev. Educ. Res. 33(5), 455–473 (1963).
Nix, T. W. & Barnette, J. J. The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing. Res. Sch. 5(2), 3–14 (1998).
Park, H. A. An introduction to logistic regression: From basic concepts to interpretation with particular attention to nursing domain. J. Korean Acad. Nurs. 43(2), 154–164. https://doi.org/10.4040/jkan.2013.43.2.154 (2013).
Wang, S. et al. Development and validation of a novel scoring system developed from a nomogram to identify malignant pleural effusion. EBioMedicine 58, 102924. https://doi.org/10.1016/j.ebiom.2020.102924 (2020).
Soto, A. et al. Development of a clinical scoring system for the diagnosis of smear-negative pulmonary tuberculosis. Braz. J. Infect. Dis. 12(2), 128–132. https://doi.org/10.1590/s1413-86702008000200006 (2008).
Hartmann, K., Krois, J. & Waske, B. E-Learning Project SOGA: Statistics and Geospatial Data Analysis 33 (Department of Earth Sciences, Freie Universität Berlin, 2018).
Zhang, C. et al. A novel scoring system for prediction of disease severity in COVID-19. Front. Cell. Infect. Microbiol. 10, 318. https://doi.org/10.3389/fcimb.2020.00318 (2020).
Qin, L. et al. A predictive model and scoring system combining clinical and CT characteristics for the diagnosis of COVID-19. Eur. Radiol. 30, 6797–6807. https://doi.org/10.1007/s00330-020-07022-1 (2020).
Liang, M. et al. Novel risk scoring system for predicting acute respiratory distress syndrome among hospitalized patients with coronavirus disease 2019 in Wuhan, China. BMC Infect. Dis. 20, 960. https://doi.org/10.1186/s12879-020-05561-y (2020).
Henderson, B. E. Shorter time from COVID-19 symptom onset to hospitalization linked to disease severity, death. News-Medical Life sciences https://www.news-medical.net/news/20200926/Shorter-time-from-COVID-19-symptom-onset-to-hospitalization-linked-to-disease-severity-death.aspx (2020).
Verity, R. et al. Estimates of the severity of coronavirus disease 2019: A model-based analysis. Lancet Infect. Dis. 20, 669–677. https://doi.org/10.1016/S1473-3099(20)30243-7 (2020).
Espinosa, O. A. et al. Prevalence of comorbidities in patients and mortality cases affected by SARS-CoV2: A systematic review and meta-analysis. Rev. Inst. Med. Trop. Sao Paulo 62, e43. https://doi.org/10.1590/S1678-9946202062043 (2020).
Karimi, A. et al. Development of a novel scoring system for comprehensive COVID-19 patient evaluation: An evidence-based systematic review. Arch. Pediatr. Infect. Dis. https://doi.org/10.5812/pedinfect.110201 (2021).
Guan, W. J. et al. Clinical characteristics of coronavirus disease 2019 in China. N. Engl. J. Med. 382, 1708–1720. https://doi.org/10.1056/NEJMoa2002032 (2020).
Chen, Z., Russo, N. W., Miller, M. M., Murphy, R. X. & Burmeister, D. B. An observational study to develop a scoring system and model to detect risk of hospital admission due to COVID-19. J. Am. Coll. Emerg. Physicians Open 2, e12406. https://doi.org/10.1002/emp2.12406 (2021).
Haji Aghajani, M. H., Sistanizad, M., Pourhoseingholi, A., Asadpoordezaki, Z. & Taherpour, N. Development of a scoring system for the prediction of in-hospital mortality among COVID-19 patients. Clin. Epidemiol. Glob. Health 12, 100871. https://doi.org/10.1016/j.cegh.2021.100871 (2021).
Zheng, Z. et al. Risk factors of critical & mortal COVID-19 cases: A systematic literature review and meta-analysis. J. Infect. 81, e16–e25. https://doi.org/10.1016/j.jinf.2020.04.021 (2020).
Kumar, A. et al. Clinical Features of COVID-19 and Factors Associated with Severe Clinical Course: A Systematic Review and Meta-analysis (Social Science Research Network, 2020).
Tian, W. et al. Predictors of mortality in hospitalized COVID-19 patients: A systematic review and meta-analysis. J. Med. Virol. 92, 1875–1883. https://doi.org/10.1002/jmv.26050 (2020).
Boari, G. E. M. et al. Prognostic factors and predictors of outcome in patients with COVID-19 and related pneumonia: A retrospective cohort study. Biosci. Rep. https://doi.org/10.1042/BSR20203455 (2020).
Borghesi, A. & Maroldi, R. COVID-19 outbreak in Italy: Experimental chest X-ray scoring system for quantifying and monitoring disease progression. Radiol. Med. 125, 509–513. https://doi.org/10.1007/s11547-020-01200-3 (2020).
Borghesi, A. et al. Chest X-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: A study of 302 patients from Italy. Int. J. Infect. Dis. 96, 291–293. https://doi.org/10.1016/j.ijid.2020.05.021 (2020).
Ministry of Health and Population (MOHP); Coronavirus Disease COVID 19, SARS COV 2, Management guide; Version 1 (2020).
Ministry of Health and Population (MOHP), Egypt. Coronavirus Disease COVID 19, SARS COV 2, Management guide; Version 1.4 (2020).
Elgohary, M. et al. Efficacy of Sofosbuvir plus Ledipasvir in Egyptian patients with COVID-19 compared to standard treatment: A randomized controlled trial. J. Med. Life. 15(3), 350–358. https://doi.org/10.25122/jml-2021-0175 (2022).
Ministry of Health and Population (MOHP); Coronavirus Disease COVID 19, SARS COV 2, Management guide. Version 1.8 (2022).
Frediansyah, A., Nainu, F., Dhama, K., Mudatsir, M. & Harapan, H. Remdesivir and its antiviral activity against COVID-19: A systematic review. Clin. Epidemiol. Glob. Health 9, 123–127. https://doi.org/10.1016/j.cegh.2020.07.011 (2021).
Soy, M. et al. Cytokine storm in COVID-19: Pathogenesis and overview of anti-inflammatory agents used in treatment. Clin. Rheumatol. 39, 2085–2094. https://doi.org/10.1007/s10067-020-05190-5 (2020).
Wang, J., Jiang, M., Chen, X. & Montaner, L. J. Cytokine storm and leukocyte changes in mild versus severe SARS-CoV-2 infection: Review of 3939 COVID-19 patients in China and emerging pathogenesis and therapy concepts. J. Leukoc. Biol. 108, 17–41. https://doi.org/10.1002/JLB.3COVR0520-272R (2020).
Tsai, A., Diawara, O., Nahass, R. G. & Brunetti, L. Impact of tocilizumab administration on mortality in severe COVID-19. Sci. Rep. 10, 19131. https://doi.org/10.1038/s41598-020-76187-y (2020).
Kotak, S. et al. Use of tocilizumab in COVID-19: A systematic review and meta-analysis of current evidence. Cureus 12, e10869. https://doi.org/10.7759/cureus.10869 (2020).
FDA news release. Tocilizumab emergency use authorization (EUA). U.S. Food and Drug Administration. http://www.fda.gov/newsevents/press-announcements/coronavirus-covid-19-update-fdaauthorizesdrug-treatment-covid-19 (2021).
Chen, C. P. et al. A multicenter, randomized, open-label, controlled trial to evaluate the efficacy and tolerability of hydroxychloroquine and a retrospective study in adult patients with mild to moderate coronavirus disease 2019 (COVID-19). PLoS ONE 15(12), e0242763. https://doi.org/10.1371/journal.pone.0242763 (2020).
Ghosn, L. et al. Interleukin-6 blocking agents for treating COVID-19: A living systematic review. Cochrane Database Syst. Revpmcid. https://doi.org/10.1002/14651858.CD013881 (2021).
REMAP-CAP Investigators et al. Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N. Engl. J. Med. 384(16), 1491–1502. https://doi.org/10.1056/NEJMoa2100433 (2021).
Guaraldi, G. et al. Tocilizumab in patients with severe COVID-19: A retrospective cohort study. Lancet Rheumatol. 2(8), e474–e484. https://doi.org/10.1016/S2665-9913(20) (2020).
Guimarães, P. O. et al. Tofacitinib in patients hospitalized with Covid-19 pneumonia. N. Engl. J. Med. 385(5), 406–415. https://doi.org/10.1056/NEJMoa2101643 (2021).
Gupta, S. et al. Association between early treatment with tocilizumab and mortality among critically ill patients with COVID-19. JAMA Intern. Med. 181(1), 41–51. https://doi.org/10.1001/jamainternmed.2020.6252 (2021).
Yong-gang, Z., et al. Aberrant pathogenic GM-CSF+ T cells and inflammatory CD14+CD16+ monocytes in severe pulmonary syndrome patients of a new coronavirus. Biol. Res. Xiv (2020).
Hermine, O. et al. Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: A randomized clinical trial. JAMA Intern. Med. 181(1), 32–40. https://doi.org/10.1001/jamainternmed.2020.6820 (2021).
Herold, T. et al. Elevated levels of IL-6 and CRP predict the need for mechanical ventilation in COVID-19. J. Allergy Clin. Immunol. 146(1), 128-136.e4. https://doi.org/10.1016/j.jaci.2020.05.008 (2020).
Hirano, T. & Murakami, M. COVID-19: A new virus, but a familiar receptor and cytokine release syndrome. Immunity 52(5), 731–733. https://doi.org/10.1016/j.immuni.2020.04.003 (2020).
Hoffmann, M. et al. SARS-CoV-2 cell entry depends on ACE2 and TMPRSS2 and is blocked by a clinically proven protease inhibitor. Cell 181(2), 271-280.e8. https://doi.org/10.1016/j.cell.2020.02.052 (2020).
Zhang, J., Chen, C., Yang, Y. & Yang, J. Effectiveness of tocilizumab in the treatment of hospitalized adults COVID-19. A systematic review and meta-analysis. Medicine 101, 9 (2022).
Funding
Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).
Author information
Author notes
These authors contributed equally: Mohamed AbdelSalam Elgohary and Asmaa Ali.
Authors and Affiliations
Department of Tropical Medicine, Military Medical Academy, Cairo, Egypt
Mohamed AbdelSalam Elgohary
Department of Pulmonary Medicine, Abbassia Chest Hospital, MOH, Cairo, Egypt
Asmaa Ali
Department of Laboratory Medicine, School of Medicine, Jiangsu University, Zhenjiang, 212013, P. R. China
Asmaa Ali
Department of Pharmacology and Toxicology, Faculty of Pharmacy, Tanta University, Tanta, Egypt
Thanaa A. El-Masry
Department of Microbiology, Faculty of Medicine, Umm Al-Qura University, Makkah, Saudi Arabia
Hani Faidah & Farkad Bantun
Department of Tropical Medicine, Almaza Military Fever Hospital, Cairo, Egypt
Ahmad M. Elkholy, Nabila N. Elgamal & Mohamed Emam Mohamed
Department of Microbiology, Almaza Military Fever Hospital, Cairo, Egypt
Jaklin S. Fahim
Biological Prevention Department, Ministry of Defense, Cairo, Egypt
Mohamed G. Seadawy
Department of Public Health, Almaza Military Fever Hospital, Cairo, Egypt
Amro M. Helal
Smartox Biotechnology, 6 rue des Platanes, 38120, Saint-Egrève, France
Michel De Waard
L’institut du Thorax, INSERM, CNRS, Univ Nantes, F-44007, Nantes, France
Michel De Waard
Université de Nice Sophia-Antipolis, LabEx “Ion Channels, Science & Therapeutics”, F-06560, Valbonne, France
Michel De Waard
Egyptian Military Medical Services Administration, Cairo, Egypt
Hesham M. Shishtawy
Department of Pharmaceutical Microbiology, Faculty of Pharmacy, Tanta University, Tanta, Egypt
Maisra M. El-Bouseary
Authors
Mohamed AbdelSalam Elgohary
Asmaa Ali
Thanaa A. El-Masry
Hani Faidah
Farkad Bantun
Ahmad M. Elkholy
Jaklin S. Fahim
Nabila N. Elgamal
Mohamed Emam Mohamed
Mohamed G. Seadawy
Amro M. Helal
Michel De Waard
Hesham M. Shishtawy
Maisra M. El-Bouseary
Contributions
Conceptualization, M.A.E. and A.A.; Data curation, M.A.E., A.A., A.M.E., J.S.F., N.N.E., A.M.H., H.M.S., M.E.M. and M.M.E.; Formal analysis, M.A.E., A.A., H.F., F.B., T.A.E., M.E.M., M.D.W. and M.M.E.; Investigation, M.A.E., A.A., H.F., A.M.E., J.S.F., M.G.S., N.N.E. and M.M.E.; Methodology, A.E., A.A., F.B., A.M.H., M.G.S., M.M.E. and H.M.S.; Visualization, M.A.E., A.A., T.A.E., M.D.W. and M.M.E.; Writing—original draft, M.A.E. and A.A.; Writing—review and editing, M.A.E., A.A. and M.M.E.
Corresponding authors
Correspondence to
Mohamed AbdelSalam Elgohary or Maisra M. El-Bouseary.
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
SARS-CoV-2 virus has rapidly spread worldwide since December 2019, causing COVID-19 disease. In-hospital mortality is a common indicator for evaluating treatment outcomes. Therefore, the developing and validating a simple score system from observational data could assist in modulating the management procedures. A retrospective cohort study included all data records of patients with positive PCR for SARS-CoV-2. The factors that associated with mortality were analyzed, then allocation of potential predictors of mortality was executed using different logistic regression modeling, subsequently scoring system was developed from the most weighted predictors. The mortality rate of patients with COVID-19 pneumonia was 28.5% and 28.74%, respectively. The most significant factors that affected in-hospital mortality were old age (> 60 years), delay in hospital admission (> 4 days), high neutrophil/lymphocyte ratio “NLR” (> 3); higher computed tomography severity score; and CT-SS (> 20), in addition to using remdesivir and tocilizumab in the treatment protocol (P < 0.001 for all). The validity of the newly performed score was significant; the AUC was 85%, P < 0.001, and its prognostic utility was good; the AUC was 75%, P < 0.001. The prognostic utility of newly developed score system (EGY.Score) was excellent and could be used to adjust the treatment strategy of highly at-risk patients with COVID-19 pneumonia.
Introduction
COVID-19 is a current pandemic disease caused by SARS-CoV-2 which is associated with serious health complications worldwide1. As of October 28, 2022, there were 626,337,158 confirmed cases of COVID-19, including 6,566,610 deaths, reported to the WHO2. The severity of COVID-19 infection ranges from asymptomatic manifestations to flu-like symptoms such as fever, headache, dry cough, dyspnea, mayalgia, and joint pain, eventually leading to fatal acute respiratory distress syndrome (ARDS)3,4,5. According to previous studies, the mortality rate for the most serious SARS-CoV-2 infections that require admission to the intensive care unit (ICU) ranges from 8.1 to 30% for hospitalized patients with SARS-CoV-2 pneumonia and up to 16% to 78% for patients who require admission to an ICU with critical care4,6,7,8.
According to previous studies, the COVID-19 outbreak is not uniform across nations, with notable variations in the proportion of serious diseases and case fatality rates9. Multicenter reports emphasize that patient-specific characteristics are important predictors of the presentation and consequences of COVID-19, even though the quality of healthcare services may be a factor in such variations10. Since the WHO officially declared a global pandemic in March 2020, there have been significant efforts to identify prognosticators that clinicians use to evaluate the risk at the early stage of the illness. This has helped to better tailor management strategies, assist decision-making, and promote health for COVID-19 patients by raising the therapeutic response, increasing the diagnostic accuracy, and lowering the case fatality rate11.
Various studies found that advanced age12,13,14,15, male gender13, and co-morbidities13,14,15 such as diabetes mellitus (DM), obesity, systemic hypertension (HTN), renal diseases, coronary artery disease15, and malignancy were risk factors for COVID-19 mortality. Beyond manifestations such as fever, cough16, haemoptysis12, dyspnoea12,16, unconsciousness12, clinical parameters such as elevated neutrophil/lymphocyte ratio12,17, and high levels of creatinine18, elevated lactate dehydrogenase (LDH)12,14,15,18, direct bilirubin12, and alanine amino-transferase (AST)18, which indicate early evidences to the severity of disease, an increased plasma level of biomarkers like d-dimer15,18,19,20, C-Reactive Protein (CRP)21,22, serum ferritin18, Interleukin-6 (IL-6)18, and procalcitonin (PCT) strengthens these findings18,19,20. The efficacy of particular antiviral and targeted immuno-modulatory therapies is still uncertain, therefore risk categorization and mortality prediction offer a logical strategy for allocating medical resources23.
The value of developing a score system for predicting the upcoming prognosis as mortality is vastly important for health care providers, especially doctors, to be more reliable and objective in evaluating their patients rather than being subjective24,25. Additionally, the recent publications in the medical field and beyond focus on the quality of the scoring system model and the methods of formation either through using strict and dedicated factors during statistical analysis steps or using machine learning programs25,26,27,28. Therefore, in order to estimate the probability of mortality in patients with COVID-19 pneumonia, forming a simple custom score could enhance the objective decision-making around treatment selection, as well as teach from trial and error in dealing with such a new disease. Our aim of the study was to develop and validate a simple scoring system to predict in-hospital mortality among patients with COVID-19 pneumonia within the first days of hospitalization, which was based mainly on different presenting symptoms, comorbidities, vital signs, and some laboratory data in addition to some local and affordable treatment options entered into the regression model. The multiple logistic regression model approaches were applied to our set of data to find the best equation that predicts the death probability. Furthermore, to measure the validity of that equation, a receiver operating characteristic curve was applied. Hence, the area under the curve expresses the goodness of that model in the prediction of mortality.
Material and methods
Study design and patient selection
A retrospective cohort study included all data records of patients with positive PCR for SARS-CoV-2 who was admitted to Almaza Military Hospital from January 2020 to the end of December 2021. A flow diagram showed the criteria of selected patients (Fig. 1). Hence, only 1535 patients had been considered as suggested candidates for primary study criteria. All of them had positive oropharangeal PCR tests in addition to CT chest. Those below 18 years old and those with free CT findings were excluded from the study. As well, cases with incomplete data records were also excluded. After informed written consent from all participants for publication of their data, the study has been approved by the Research Ethics Committee of the Faculty of Pharmacy, Tanta University (REC-TP code: TP/RE/012-21P-005) and the ethical committee office of the Medical Military Academy in agreement with the Helsinki Declaration Roles.
Data collection
The data was extracted from the patient's files and included basic socio-demographic data in addition to the presenting symptoms. Only laboratory data that was conducted in the first 24 h after admission was considered. The evaluation of CT chest and quantifying CT-SS as regarding Yang et al.29, which was performed by two independent radiologists for reliability. The data about treatment protocol as regarding local guidelines was also included.
Outcome
The outcome of desire was in-hospital mortality from COVID-19 pneumonia, which is defined as death during a period of admission to a hospital as a consequence of COVID-19 disease. Expectation of a short-term death after discharge from the hospital is also considered in-hospital mortality30.
Statistical analysis
Building the predictive model and development of scoring system
All the data was collected and coded on an Excel sheet. The normality of data was examined using the Shapiro–Wilk test using SigmaPlot for Windows version 12.5. 0.38 (Systat Software, Inc., UK, 2011). The descriptive statistics were performed using Minitab 17.1.0.0 for Windows (Minitab Inc., 2013, Pennsylvania, USA). In the first step, all data was subjected to univariate analysis. Hence, the comparison between two means was done using an independent t-test, while the frequency comparison was made using the chi-square test. Regarding the Neyman-Pearson theory of classical statistics31,32, the critical value of a significant number was selected by the observer to guarantee that the type II errors were minimized as much as possible and circumvent the interrelationship between type I and type II errors. Additionally, Park, 201333 mentioned that all variables could be implicated to test their correlation with the final outcome, provided that the log equation of fitness was acceptable; however, some authors preferred to select the variable of desire to be enrolled in the logistic equation after one step of simple hypothesis testing (univariate analysis)34,35, consuming that the margin of type II error was not so far away to accept the null hypothesis36. Therefore, in the current models, all factors with a p-value ≤ 0.35 were subjected to the second step, multivariate analysis. During building the logistic regression models, and to avoid multicollinearity, factors with VIF > 5 were eliminated. The goodness of fit for the regression model was performed using the Hosmer and Lemeshow test. Forward selection and backward elimination techniques were applied to select the best predictors for mortality. The Wald statistics numbers for each factor of interest were multiplied by the coefficient and divided by a constant value, and the result was rounded to the nearest integer number, taking into account the sign of the regression coefficient. According to the literature, all numerical data are transformed into categorical groups based on the distribution histogram and cumulative frequency or normal reference. The total score is then calculated from the submission of each individual score.
Score validation
The data for all factors implicated in the scoring system (SOM) was collected from the validation cohort, while the individual score and total score for every participant were calculated automatically in an Excel sheet. The performance of the total score was assessed using ROC curve analysis. The AUC above 0.6 was considered acceptable. The logistic regression analysis was finally performed to calculate the equation of death probability from total SOM. All tests were two-sided and a p-value of less than 0.05 was considered significant.
Institutional review board statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by after approval by the Research Ethics Committee of Faculty of Pharmacy, Tanta University (REC-TP code: TP/RE/012-21P-005) and the ethical committee office of the Medical Military Academy.
Informed consent statement
Informed consent was obtained from all subjects involved in the study.
Results
Patients' characteristics
The total number of data points was 316, and the mortality rate of patients with COVID-19 pneumonia was 28.5%. As shown in Table 1, the basic criteria of both the survival and mortality groups were presented. One third of cases were male, and more than half of them were older than 60 years old. The most frequent comorbidities were DM and HTN, and nearly all patients were presented with cough and dyspnea. However, fever was a significant cardinal sign that correlated with mortality. The duration of complaints in the mortality group was significantly shorter than the survival one, as more than half of them came with a complaint history of less than 4 days. In the mortality group, the CT-SS was significantly higher and more than 2/3 of cases had severe and very severe degrees of CT scores (> 20). Moreover, neutrophil (%), NLR and PLR, as well as AST, urea, creatinine and IL-6 were significantly higher in the mortality group.
Table 1 Socio-demographic data and clinical characteristics.
Predictors of mortality
After selection of all factors in univariate analysis with a p-value ≤ 0.35, multivariate analysis with different approaches was applied to select the most significant factors that affected in-hospital mortality. As shown in Table 2, being elderly (> 60 years), having a shorter duration of complaint, having a high NLR, and having a higher CT-SS (> 20) were all significant independent predictors of mortality (P < 0.05 for all).
Table 2 Predictors of mortality using multivariate logistic regression model with different technique.
Score model formation and its performance
After collecting the predictors from the previous step, a multiple logistic regression analysis were applied to weight the factors and forming the scoring number based on both WS and Coefficient number. As shown in Table 3, two different scores could be used; the first one (SOM-1) including tocilizumab, and the second one (SOM-2) without it. However, both of them showed insignificant differences in discriminating in-hospital mortality (Fig. 2). Because the AUC was 85% and 84%, respectively, with P = 0.001 for both, SOM-1 was chosen for further validation.
Table 3 Development of scoring system based on best predictors for mortality.
Assessment of the new score: validation and prognostic utility
About 327 patients were subjected to score validation; the descriptive statistics of the validation cohort were summarized in Table 4, in which the mortality rate was 28.74%.
Table 4 Characters of validity cohort.
Moreover, Fig. 3a showed the median (IQR) value of SOM-1 was significantly higher in the mortality group; (13.5 (3–18) than the survival one; 2 (− 0.5–7)), P < 0.001. The prognostic utility of SOM-1 was so good; the AUC was 75%, P < 0.001 (Fig. 3b), and at cutoff values of above 5 and 16.5, the sensitivity and specificity were above 90%, respectively (Table 5). The probability of mortality increased with every unit increase in the SOM-1 (Fig. 3c).
Table 5 Performance of SOM-1 in predicting in-hospital mortality.
Discussion
The current study used the observational data from patients with COVID-19 pneumonia to develop a simple predictive model for further building a scoring system that easily calculates the probability of in-hospital mortality. The algorithms of the predictive models were applied to 316 patients and included simply collected historical, clinical, and laboratory data on the day of admission, which gave our model of prediction the capability to be applied in other medical sectors with an affordable set of data collection and less sophisticated investigation. However, so many scoring systems have been developed since the start of the pandemics to predict different outcomes related to COVID-19 disease. Their sensitivity and specificity ranged between 70 and 100%37,38,39, and the most effective study recorded a model with excellent validity (AUC = 93.8%)20.
The present score, besides its easy and manual calculation from simple data, had the capability for prognosis prediction; the utility of the score was very good enough to be accepted, hence the AUC of 85% in the training cohort and still good when applied to the validating cohort (AUC of 75%), which indicates how much the stability of the score in predicting the disease prognosis made the treatment strategy more powerful if it was applied as early as possible for a much better outcome. Moreover, during building the model, we entered medication that was used in local treatment protocol in response to laboratory results to estimate if these factors were implicated in the final outcome or not. We reported that remdesivir and tocilizumab were significantly correlated with in-hospital mortality. Therefore, two scoring systems were calculated with tocilizumab (SOM-1) and without tocilizumab (SOM-2), with an insignificant difference between both of them. The accuracy was 85 and 84%, respectively (Fig. 2). For that reason, SOM-1 was the choice for further external validity. Another concern that made our newly formed score more stable was that we depended on both the coefficient number and Wald statistic number from the logistic regression model to weight the predictors.
The attendance results did not find any surprising factors that affected the mortality except the duration before hospital admission. Hence, we found that patients with a shorter duration of less than 4 days before admission died, which made the explanation more difficult, and it could be related to the state of denial that the patients caught the disease and suffered from silent prolonged hypoxia. Surprisingly, only one study includes this historical item in the predictive model of COVID-19-related mortality. Henderson et al. reported that a shorter time from symptom onset to hospitalization is associated with a more serious disease and higher mortality40. On the other hand, old age was correlated with bad prognosis, which came in consistence with other reports41,42,43,44,45,46.
Despite the fact that many studies have pointed to the importance of male sex and associated comorbidity in poor prognosis45,46,47,48,49, our study did not find that link after multiple filtration of logistic regression modeling, and a recent study supported that50. Furthermore, the severity level of lesion in HRCT was one of the most reliable predictors; thus, it was associated with a poor prognosis, despite the fact that a few studies used that factor to predict in-hospital mortality outcome50,51,52.
Our study introduced some treatment medications like iverzine53,54, sofosbuvir/ledipasvir55, remdesivir, and tocilizumab to be estimated as predictive factors for mortality in a regression model. iverzine was used following our national COVID-19 management guidelines in its old versions53,54, however it is deleted from the recent version56. Only remdesivir and tocilizumab continued to be linked with mortality after multiple filtrations of the predictors. Hence, remdesivir was found to be protective against a bad prognosis, and early administration could inhibit the replication of viruses and decrease the viral load; it could also counter the process of pathology inside the lung parenchyma that finally led to improvement in the lung lesion57.
On the other hand, our data showed that tocilizumab increased the likelihood of mortality two times more, which could be due to limiting the use of that medication in patients with severe disease, so the risk of its link to a bad prognosis became much higher. Additionally, tocilizumab had been prescribed in COVID-19 patients with an established higher level of IL-6, denoting that a cascade of cytokines storm had been started58,59,60. Although a pooled analysis of systematic reviews on tocilizumab and mortality outcomes found that it was not only protective against bad outcomes, it was also significantly linked to post-drug infection, which led to a poor prognosis from super infection61. That fact could explain the present finding and draw attention to the limited power of using biological therapy in treatment. Nevertheless, several newly published randomized controlled trials (RCTs)62,63,64,65,66,67,68,69,70 and meta-analyses71,72,73 of RCTs have investigated the effects of TCZ as an adjunctive therapy in patients with COVID-19 but have reported inconsistent results. Moreover, there are increasing number of newly available studies regarding TCZ treatment for COVID-19. Therefore, there are still limited real-world data about the effect of TCZ on inflammatory activity in COVID-19 patients74.
Limitations
Even though our study was the first of its kind that was proposed in Egypt, which is a developing country with limited resources, it highlighted the ability of simple data to predict the outcome of COVID-19 patients. However, the study showed some limitations; the first was the types of study design; hence, the main issue with retrospective cohort type studies was the effect of confounders, which could influence the final outcome in an unpredicted way. The second limitation was the single-centre study, which increased the demand for further external validation from other centers. Additionally, the third point was that the study excluded younger individuals < 18 years old during recruitment, which may be an area of future interest. Moreover, the score was constructed for only COVID-19 patients, which made the comparison with other alternative scores much more difficult to apply.
Conclusion
The constructed score (EGY.Score) from the observational data could predict the prognosis of patients with COVID-19 pneumonia, which may possibly be used to adjust the management intervention for further gain of a desirable outcome.
Data availability
All data generated or analyzed during this study are included in this published article and supplementary material.
References
Driggin, E. et al. Cardiovascular considerations for patients, health care workers, and health systems during the COVID-19 pandemic. J. Am. Coll. Cardiol. 75, 2352–2371. https://doi.org/10.1016/j.jacc.2020.03.031 (2020).
WHO. Coronavirus (COVID-19) Dashboard, https://covid19.who.int/ (Accessed 28 Oct 2022).
Shaweno, T. et al. V-2 antibody among individuals aged above 15 years and residing in congregate settings in Dire Dawa city administration, Ethiopia. Trop. Med. Health 49, 55. https://doi.org/10.1186/s41182-021-00347-7 (2021).
Esfahanian, F., Seyedalinaghi, S., Janfaza, N. & Tantuoyir, M. M. Predictors of hospital mortality among patients with COVID-19 in Tehran, Iran. SAGE Open Med. 9, 20503121211051572. https://doi.org/10.1177/20503121211051573 (2021).
Kox, H. L. M. Relative infectuousness of asymptomatic andsymptomatic COVID-19 infections—analytical time table. MPRA Munich Personal RePEc Archive, https://mpra.ub.uni-muenchen.de/108781/1/MPRA_paper_108781.pdf (2021).
Olivas-Martınez, A. et al. In-hospital mortality from severe COVID-19 in a tertiary care center in Mexico City; causes of death, risk factors and the impact of hospital saturation. PLoS ONE 16, e0245772. https://doi.org/10.1371/journal.pone.0245772 (2021).
Oliveira, E. et al. ICU outcomes and survival in patients with severe COVID-19 in the largest health care system in central Florida. PLoS ONE 16, e0249038. https://doi.org/10.1371/journal.pone.0249038 (2021).
Shi, C. et al. Predictors of mortality in patients with coronavirus disease 2019: A systematic review and meta analysis. BMC Infect. Dis. 21, 663 (2021).
European Centre for Disease Prevention and Control. Situation Update Worldwide, as of Week 1, https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases (2021).
Cao, Y. et al. Comparative genetic analysis of the novel coronavirus (2019-nCoV/SARS-CoV-2) receptor ACE2 in different populations. Cell Discov. 6, 11. https://doi.org/10.1038/s41421-020-0147-1 (2020).
Assal, H. H. et al. Predictors of severity and mortality in COVID-19 patients. Egypt. J. Bronchol. 16, 18. https://doi.org/10.1186/s43168-022-00122-0 (2022).
Liang, W. et al. Development and validation of a clinical RiskScore to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern. Med. 180, 1081–1089. https://doi.org/10.1001/jamainternmed.2020.2033 (2020).
Ahmad, S. et al. Epidemiological, clinical, and laboratory predictors of in-hospital mortality among COVID-19 patients admitted in a tertiary COVID dedicated hospital, Northern India: A retrospective observational study. J. Prim. Care Community Health 12, 21501327211041490. https://doi.org/10.1177/21501327211041486 (2021).
Steinberg, E., Balakrishna, A., Habboushe, J., Shawl, A. & Lee, J. Calculated decisions: COVID-19 calculators during extreme resource-limited situations. Emerg. Med. Pract. 22(4 Suppl), CD1–CD5 (2020).
Chauhan, N. K. et al. Predictors of clinical outcomes in adult COVID-19 patients admitted to a tertiary care hospital in India: An analytical cross-sectional study. Acta Biomed. 92, e2021024. https://doi.org/10.23750/abm.v92i3.10630 (2021).
Salunke, A. A. et al. A proposed ABCD scoring system for patient’s self assessment and at emergency department with symptoms of COVID-19. Diabetes Metab. Syndr. 14, 1495–1501. https://doi.org/10.1016/j.dsx.2020.07.053 (2020).
Rai, D., Ranjan, A., Ameet, H. & Pandey, S. Clinical and laboratory predictors of mortality in COVID-19 infection: A retrospective observational study in a Tertiary Care Hospital of Eastern India. Cureus 13, e17660. https://doi.org/10.7759/cureus.17660 (2021).
Zhou, F. et al. Clinical course and risk factors for mortality of adult in patients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 395, 1054–1062. https://doi.org/10.1016/S0140-6736(20)30566-3 (2020).
Li, X. et al. Risk factors for severity and mortality in adult COVID-19 inpatients in Wuhan. J. Allergy Clin. Immunol. 146, 110–118. https://doi.org/10.1016/j.jaci.2020.04.006 (2020).
Shang, Y. et al. Scoring systems for predicting mortality for severe patients with COVID-19. E. Clin. Med. 24, 100426. https://doi.org/10.1016/j.eclinm.2020.100426 (2020).
Knight, S. R. et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: Development and validation of the 4C Mortality Score. BMJ 370, m3339. https://doi.org/10.1136/bmj.m3339 (2020).
Galloway, J. B. et al. A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: An observational cohort study. J. Infect. 81, 282–288. https://doi.org/10.1016/j.jinf.2020.05.064 (2020).
Gopalan, N. et al. Predictors of mortality among hospitalized COVID-19 patients and risk score formulation for prioritizing tertiary care—An experience from South India. PLoS ONE 17, e0263471. https://doi.org/10.1371/journal.pone.0263471 (2022).
Barbini, P., Cevenini, G., Furini, S. & Barbini, E. A naïve approach for deriving scoring systems to support clinical decision making. J. Eval. Clin. Pract. 20(1), 1–6. https://doi.org/10.1111/jep.12064 (2014).
Biagioli, B. et al. A multivariate Bayesian model for assessing morbidity after coronary artery surgery. Crit. Care. 10(3), R94. https://doi.org/10.1186/cc4951 (2006).
Demichelis, F., Magni, P., Piergiorgi, P., Rubin, M. A. & Bellazzi, R. A hierarchical Naive Bayes Model for handling sample heterogeneity in classification problems: An application to tissue microarrays. BMC Bioinform. 7(1), 514. https://doi.org/10.1186/1471-2105-7-514 (2006).
Lasko, T. A., Bhagwat, J. G., Zou, K. H. & Ohno-Machado, L. The use of receiver operating characteristic curves in biomedical informatics. J. Biomed. Inform. 38(5), 404–415. https://doi.org/10.1016/j.jbi.2005.02.008 (2005).
Kuzniewicz, M. W. et al. Variation in ICU risk-adjusted mortality: Impact of methods of assessment and potential confounders. Chest 133(6), 1319–1327. https://doi.org/10.1378/chest.07-3061 (2008).
Yang, R. et al. Chest CT severity score: An imaging tool for assessing severe COVID-19. Radiol. Cardiothorac. Imaging 2, e200047. https://doi.org/10.1148/ryct.2020200047 (2020).
Jeong, J. et al. Development and validation of a scoring system for mortality prediction and application of standardized W statistics to assess the performance of emergency departments. BMC Emerg. Med. 21, 71. https://doi.org/10.1186/s12873-021-00466-8 (2021).
Clark, C. A. & Chapter, I. Hypothesis testing in relation to statistical methodology. Rev. Educ. Res. 33(5), 455–473 (1963).
Nix, T. W. & Barnette, J. J. The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing. Res. Sch. 5(2), 3–14 (1998).
Park, H. A. An introduction to logistic regression: From basic concepts to interpretation with particular attention to nursing domain. J. Korean Acad. Nurs. 43(2), 154–164. https://doi.org/10.4040/jkan.2013.43.2.154 (2013).
Wang, S. et al. Development and validation of a novel scoring system developed from a nomogram to identify malignant pleural effusion. EBioMedicine 58, 102924. https://doi.org/10.1016/j.ebiom.2020.102924 (2020).
Soto, A. et al. Development of a clinical scoring system for the diagnosis of smear-negative pulmonary tuberculosis. Braz. J. Infect. Dis. 12(2), 128–132. https://doi.org/10.1590/s1413-86702008000200006 (2008).
Hartmann, K., Krois, J. & Waske, B. E-Learning Project SOGA: Statistics and Geospatial Data Analysis 33 (Department of Earth Sciences, Freie Universität Berlin, 2018).
Zhang, C. et al. A novel scoring system for prediction of disease severity in COVID-19. Front. Cell. Infect. Microbiol. 10, 318. https://doi.org/10.3389/fcimb.2020.00318 (2020).
Qin, L. et al. A predictive model and scoring system combining clinical and CT characteristics for the diagnosis of COVID-19. Eur. Radiol. 30, 6797–6807. https://doi.org/10.1007/s00330-020-07022-1 (2020).
Liang, M. et al. Novel risk scoring system for predicting acute respiratory distress syndrome among hospitalized patients with coronavirus disease 2019 in Wuhan, China. BMC Infect. Dis. 20, 960. https://doi.org/10.1186/s12879-020-05561-y (2020).
Henderson, B. E. Shorter time from COVID-19 symptom onset to hospitalization linked to disease severity, death. News-Medical Life sciences https://www.news-medical.net/news/20200926/Shorter-time-from-COVID-19-symptom-onset-to-hospitalization-linked-to-disease-severity-death.aspx (2020).
Verity, R. et al. Estimates of the severity of coronavirus disease 2019: A model-based analysis. Lancet Infect. Dis. 20, 669–677. https://doi.org/10.1016/S1473-3099(20)30243-7 (2020).
Espinosa, O. A. et al. Prevalence of comorbidities in patients and mortality cases affected by SARS-CoV2: A systematic review and meta-analysis. Rev. Inst. Med. Trop. Sao Paulo 62, e43. https://doi.org/10.1590/S1678-9946202062043 (2020).
Karimi, A. et al. Development of a novel scoring system for comprehensive COVID-19 patient evaluation: An evidence-based systematic review. Arch. Pediatr. Infect. Dis. https://doi.org/10.5812/pedinfect.110201 (2021).
Guan, W. J. et al. Clinical characteristics of coronavirus disease 2019 in China. N. Engl. J. Med. 382, 1708–1720. https://doi.org/10.1056/NEJMoa2002032 (2020).
Chen, Z., Russo, N. W., Miller, M. M., Murphy, R. X. & Burmeister, D. B. An observational study to develop a scoring system and model to detect risk of hospital admission due to COVID-19. J. Am. Coll. Emerg. Physicians Open 2, e12406. https://doi.org/10.1002/emp2.12406 (2021).
Haji Aghajani, M. H., Sistanizad, M., Pourhoseingholi, A., Asadpoordezaki, Z. & Taherpour, N. Development of a scoring system for the prediction of in-hospital mortality among COVID-19 patients. Clin. Epidemiol. Glob. Health 12, 100871. https://doi.org/10.1016/j.cegh.2021.100871 (2021).
Zheng, Z. et al. Risk factors of critical & mortal COVID-19 cases: A systematic literature review and meta-analysis. J. Infect. 81, e16–e25. https://doi.org/10.1016/j.jinf.2020.04.021 (2020).
Kumar, A. et al. Clinical Features of COVID-19 and Factors Associated with Severe Clinical Course: A Systematic Review and Meta-analysis (Social Science Research Network, 2020).
Tian, W. et al. Predictors of mortality in hospitalized COVID-19 patients: A systematic review and meta-analysis. J. Med. Virol. 92, 1875–1883. https://doi.org/10.1002/jmv.26050 (2020).
Boari, G. E. M. et al. Prognostic factors and predictors of outcome in patients with COVID-19 and related pneumonia: A retrospective cohort study. Biosci. Rep. https://doi.org/10.1042/BSR20203455 (2020).
Borghesi, A. & Maroldi, R. COVID-19 outbreak in Italy: Experimental chest X-ray scoring system for quantifying and monitoring disease progression. Radiol. Med. 125, 509–513. https://doi.org/10.1007/s11547-020-01200-3 (2020).
Borghesi, A. et al. Chest X-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: A study of 302 patients from Italy. Int. J. Infect. Dis. 96, 291–293. https://doi.org/10.1016/j.ijid.2020.05.021 (2020).
Ministry of Health and Population (MOHP); Coronavirus Disease COVID 19, SARS COV 2, Management guide; Version 1 (2020).
Ministry of Health and Population (MOHP), Egypt. Coronavirus Disease COVID 19, SARS COV 2, Management guide; Version 1.4 (2020).
Elgohary, M. et al. Efficacy of Sofosbuvir plus Ledipasvir in Egyptian patients with COVID-19 compared to standard treatment: A randomized controlled trial. J. Med. Life. 15(3), 350–358. https://doi.org/10.25122/jml-2021-0175 (2022).
Ministry of Health and Population (MOHP); Coronavirus Disease COVID 19, SARS COV 2, Management guide. Version 1.8 (2022).
Frediansyah, A., Nainu, F., Dhama, K., Mudatsir, M. & Harapan, H. Remdesivir and its antiviral activity against COVID-19: A systematic review. Clin. Epidemiol. Glob. Health 9, 123–127. https://doi.org/10.1016/j.cegh.2020.07.011 (2021).
Soy, M. et al. Cytokine storm in COVID-19: Pathogenesis and overview of anti-inflammatory agents used in treatment. Clin. Rheumatol. 39, 2085–2094. https://doi.org/10.1007/s10067-020-05190-5 (2020).
Wang, J., Jiang, M., Chen, X. & Montaner, L. J. Cytokine storm and leukocyte changes in mild versus severe SARS-CoV-2 infection: Review of 3939 COVID-19 patients in China and emerging pathogenesis and therapy concepts. J. Leukoc. Biol. 108, 17–41. https://doi.org/10.1002/JLB.3COVR0520-272R (2020).
Tsai, A., Diawara, O., Nahass, R. G. & Brunetti, L. Impact of tocilizumab administration on mortality in severe COVID-19. Sci. Rep. 10, 19131. https://doi.org/10.1038/s41598-020-76187-y (2020).
Kotak, S. et al. Use of tocilizumab in COVID-19: A systematic review and meta-analysis of current evidence. Cureus 12, e10869. https://doi.org/10.7759/cureus.10869 (2020).
FDA news release. Tocilizumab emergency use authorization (EUA). U.S. Food and Drug Administration. http://www.fda.gov/newsevents/press-announcements/coronavirus-covid-19-update-fdaauthorizesdrug-treatment-covid-19 (2021).
Chen, C. P. et al. A multicenter, randomized, open-label, controlled trial to evaluate the efficacy and tolerability of hydroxychloroquine and a retrospective study in adult patients with mild to moderate coronavirus disease 2019 (COVID-19). PLoS ONE 15(12), e0242763. https://doi.org/10.1371/journal.pone.0242763 (2020).
Ghosn, L. et al. Interleukin-6 blocking agents for treating COVID-19: A living systematic review. Cochrane Database Syst. Revpmcid. https://doi.org/10.1002/14651858.CD013881 (2021).
REMAP-CAP Investigators et al. Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N. Engl. J. Med. 384(16), 1491–1502. https://doi.org/10.1056/NEJMoa2100433 (2021).
Guaraldi, G. et al. Tocilizumab in patients with severe COVID-19: A retrospective cohort study. Lancet Rheumatol. 2(8), e474–e484. https://doi.org/10.1016/S2665-9913(20) (2020).
Guimarães, P. O. et al. Tofacitinib in patients hospitalized with Covid-19 pneumonia. N. Engl. J. Med. 385(5), 406–415. https://doi.org/10.1056/NEJMoa2101643 (2021).
Gupta, S. et al. Association between early treatment with tocilizumab and mortality among critically ill patients with COVID-19. JAMA Intern. Med. 181(1), 41–51. https://doi.org/10.1001/jamainternmed.2020.6252 (2021).
Yong-gang, Z., et al. Aberrant pathogenic GM-CSF+ T cells and inflammatory CD14+CD16+ monocytes in severe pulmonary syndrome patients of a new coronavirus. Biol. Res. Xiv (2020).
Hermine, O. et al. Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: A randomized clinical trial. JAMA Intern. Med. 181(1), 32–40. https://doi.org/10.1001/jamainternmed.2020.6820 (2021).
Herold, T. et al. Elevated levels of IL-6 and CRP predict the need for mechanical ventilation in COVID-19. J. Allergy Clin. Immunol. 146(1), 128-136.e4. https://doi.org/10.1016/j.jaci.2020.05.008 (2020).
Hirano, T. & Murakami, M. COVID-19: A new virus, but a familiar receptor and cytokine release syndrome. Immunity 52(5), 731–733. https://doi.org/10.1016/j.immuni.2020.04.003 (2020).
Hoffmann, M. et al. SARS-CoV-2 cell entry depends on ACE2 and TMPRSS2 and is blocked by a clinically proven protease inhibitor. Cell 181(2), 271-280.e8. https://doi.org/10.1016/j.cell.2020.02.052 (2020).
Zhang, J., Chen, C., Yang, Y. & Yang, J. Effectiveness of tocilizumab in the treatment of hospitalized adults COVID-19. A systematic review and meta-analysis. Medicine 101, 9 (2022).
Funding
Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).
Author information
Author notes
These authors contributed equally: Mohamed AbdelSalam Elgohary and Asmaa Ali.
Authors and Affiliations
Department of Tropical Medicine, Military Medical Academy, Cairo, Egypt
Mohamed AbdelSalam Elgohary
Department of Pulmonary Medicine, Abbassia Chest Hospital, MOH, Cairo, Egypt
Asmaa Ali
Department of Laboratory Medicine, School of Medicine, Jiangsu University, Zhenjiang, 212013, P. R. China
Asmaa Ali
Department of Pharmacology and Toxicology, Faculty of Pharmacy, Tanta University, Tanta, Egypt
Thanaa A. El-Masry
Department of Microbiology, Faculty of Medicine, Umm Al-Qura University, Makkah, Saudi Arabia
Hani Faidah & Farkad Bantun
Department of Tropical Medicine, Almaza Military Fever Hospital, Cairo, Egypt
Ahmad M. Elkholy, Nabila N. Elgamal & Mohamed Emam Mohamed
Department of Microbiology, Almaza Military Fever Hospital, Cairo, Egypt
Jaklin S. Fahim
Biological Prevention Department, Ministry of Defense, Cairo, Egypt
Mohamed G. Seadawy
Department of Public Health, Almaza Military Fever Hospital, Cairo, Egypt
Amro M. Helal
Smartox Biotechnology, 6 rue des Platanes, 38120, Saint-Egrève, France
Michel De Waard
L’institut du Thorax, INSERM, CNRS, Univ Nantes, F-44007, Nantes, France
Michel De Waard
Université de Nice Sophia-Antipolis, LabEx “Ion Channels, Science & Therapeutics”, F-06560, Valbonne, France
Michel De Waard
Egyptian Military Medical Services Administration, Cairo, Egypt
Hesham M. Shishtawy
Department of Pharmaceutical Microbiology, Faculty of Pharmacy, Tanta University, Tanta, Egypt
Maisra M. El-Bouseary
Authors
Mohamed AbdelSalam Elgohary
Asmaa Ali
Thanaa A. El-Masry
Hani Faidah
Farkad Bantun
Ahmad M. Elkholy
Jaklin S. Fahim
Nabila N. Elgamal
Mohamed Emam Mohamed
Mohamed G. Seadawy
Amro M. Helal
Michel De Waard
Hesham M. Shishtawy
Maisra M. El-Bouseary
Contributions
Conceptualization, M.A.E. and A.A.; Data curation, M.A.E., A.A., A.M.E., J.S.F., N.N.E., A.M.H., H.M.S., M.E.M. and M.M.E.; Formal analysis, M.A.E., A.A., H.F., F.B., T.A.E., M.E.M., M.D.W. and M.M.E.; Investigation, M.A.E., A.A., H.F., A.M.E., J.S.F., M.G.S., N.N.E. and M.M.E.; Methodology, A.E., A.A., F.B., A.M.H., M.G.S., M.M.E. and H.M.S.; Visualization, M.A.E., A.A., T.A.E., M.D.W. and M.M.E.; Writing—original draft, M.A.E. and A.A.; Writing—review and editing, M.A.E., A.A. and M.M.E.
Corresponding authors
Correspondence to
Mohamed AbdelSalam Elgohary or Maisra M. El-Bouseary.
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
Papers in This Article