2. Comprehensive evaluation index
This paper uses a random 10×2K-Folding cross-validation method to classify the samples based on the ARF-OOBEE ensemble model. Among them, after testing, the number of ensemble learning base classifiers is 70, the depth is 12, and it is compared with the prediction results of 5 common machine learning algorithms. According to the prediction index analysis in Table 3, compared with the other five algorithms, the ARF-OOBEE algorithm has improved the accuracy of G-Mean and AUC parameters by nearly 2%. It can be seen that for the AR samples with clinical imbalance characteristics, the ARF-OOBEE model has good generalization performance and comprehensive classification ability.
Precision, sensitivity, specificity, G-Mean= sqrt (Sensitivity×Specificity), F1-Score, area under ROC curve AUC and other parameters together were used as predictive evaluation indicators[22]. In Table 3 and Figure 4, 7 classification models are selected for comparison, covering probability model, tree model, linear model, ensemble model and neural network model.It comprehensively reflects the performance of the research objects in different classification models and the ensemble model has the best and most stable effect, in this paper. The comprehensive classification evaluation index is lower than the ensemble classification algorithms ARF-OOBEE and GCForest. The GCForest algorithm is composed of two RF and two extreme random tree(ERT) in parallel structure, and its multiple comprehensive evaluation indicators are better than the single structure RF algorithm, but the classification calculation is relatively large. The structure of the ARF-OOBEE model has adaptive characteristics, which can dynamically change the number of ensemble learning base classifiers, and train the component classifier model parameters separately. It has good comprehensive classification characteristics for massive large data and unbalanced samples.
Table 4 gives the independent classification evaluation indicators of the 8 types of rhinitis symptoms data for the original sample. Data analysis shows that the prediction accuracy of AR, RS, CS, SD, URI, AH, NAR and OTH for the binary classification of rhinitis is higher, while the classification of degree and types in multi-class rhinitis is lower. The reason is that the classification of the four binary classification rhinitis is based on data rebalancing and is determined by the dynamically ensemble RF weighted voting algorithm in the ARF model. Output prediction of AR classi­fication were estimated using an ERTensemble algorithm with multi-category classification. ARF-OOBEE ensemble model converts the compound label classification problem into a four-label classification problem as and two multi-class classification problems.Multi-label classification were used in classification of AR, RS, URI, OTH, and multi-category classification were used in classification of AR’sdegree and type respectively,and it can avoid two or more AR classification labels in the same patient at the same time
The evaluation method in this paper uses a calculation method based on sample weights. Sensitivity represents the model’s ability to identify patients with real illnesses, while specificity represents the model’s misdiagnosis rate, and the Hamming loss is a common way of evaluating multiple classifications. The data in the table uses weighted scores. Compared with evaluating the performance of the model itself, it more reflects its performance in actual use. Avoid the rare cases of diagnosis in reality that reduce the overall evaluation of the model. For the few cases of missed diagnosis in the auxiliary diagnosis model designed in this paper, it can be ruled out by the doctor’s secondary review and other methods.
Discussion
In recent years, the prevalence of AR has increased significantly, and its diagnosis is more based on symptom evaluation and allergen detection, but due to the lack of effective and reliable diagnostic tests, the diagnosis requires experts to verify the final results based on experience[23,24]. In order to help junior physicians and clinicians diagnose allergic diseases, this work uses AI methods to extract new information from previous data for training[25,26]. Through the dynamic verification of the rule base and rule inference method, make the clinical diagnosis support system more adaptable. By introducing meta-heuristic data preprocessing technology and ensemble classification method, the systemefficiency can be further improved. Therefore, junior clinicians can strengthen clinical decision-making by more accurately diagnosing allergic diseases, can diagnose and treat AR earlier, can control the appearance of patients’ symptoms to the greatest extent, and thus improve the quality of life of patients with AR.
The diagnosis of AR is mainly based on the symptoms and the detection of allergens[27]. However, due to the complex and variable nature of nasal inflammation, it is often combined with other diseases, such as rhinosinusitis and nasal tumors. Imaging examination helps to diagnose other diseases. Turbinate hypertrophy is also a characteristic change of AR. Our selected cases have also been found to have rhinosinusitis and nasal polyps. Therefore, the use of CT imaging can better assist the diagnosis of AR.
AI technology, without human intervention, can learn tasks from a series of training examples. Moreover, they aim to produce output that is simple enough to be easily understood by humans. The difference is that the characteristics of classical statistical methods are usually a clear probability model, and it is assumed that in most cases, they require expert intervention in variable selection and transformation of the problem and overall structure. The general method of data analysis usually includes four stages, namely (a) collecting and coding clinical data in an electronic form suitable for further processing; (b) Useing feature extraction and dimensionality reduction techniques (principal component analysis) for data processing to select the most predictive parameters; (c) Schema-model selection AI model; (d) Extract knowledge by evaluating accuracy, sensitivity and specificity[28]. At present, the most common calculation models include: artificial neural network (ANN), SVM, Bayesian network (BN) and fuzzy logic (FL),etc.
In recent years, ensemble learning can organically combine multiple prediction results obtained by multiple single learning models to obtain more accurate, stable and strong final results. For exampleensemble learning models such as Boosting, Bagging and RF have been proposed one after another and applied to various types of data sets[29,30]. In this study, through the deep learning of the ensemble learning model, six common machine learning classification algorithms have been selected for comparative experiments, including RF, multi-label naive Bayes (NB), and multi-label SVM (SVM), multi-label logistic regression (LR), GCForest. The single-classifier RF algorithm is a base classification evaluation standard, and also constitutes the base classifier component of other algorithms, with good classification specificity, but the comprehensive classification evaluation index is lower than the ensemble classification algorithms ARF-OOBEE, GCForest. The GCForest algorithm is composed of two RF and two ERT in parallel structure, and its multiple comprehensive evaluation indicators are better than the single structure RF algorithm, but the classification calculation is relatively large[31].
There are two types of output for AR diseases, degree and types, which belongs to the multi-class classification problem. This article uses the OOB (out-of-bag) EE ensemble classification algorithm and uses all samples as training data. And the Extra-Tree (ET) model is used as the base classifier to balance all training data to realize the prediction of unbalanced small samples. OOBEE extracts the data equal to the minority class from the majority class, and combines the reused minority class data to build a multi-group base classifier, and obtains the ensemble classifier through the weighted voting method to reduce the impact of sample data imbalance on classification. The structure of the ARF-OOBEE model has adaptive characteristics. It can dynamically change the number of ensemble RF and ERTbaseclassifiers, and train the component classifier model parameters separately. It has good comprehensive classification characteristics for massive large data and unbalanced samples. The results show that compared with the other five algorithms, the ARF-OOBEE algorithm has improved the accuracy of G-Mean and AUC parameters by nearly 2%. It can be seen that for the AR samples with clinical imbalance characteristics, the ARF-OOBEE model has good generalization performance and comprehensive classification ability.
There are some deficiencies in this study. First of all, the diagnosis of AR is mainly based on the symptom score and allergen detection, but some patients still have obvious symptoms while the test is negative, and need to be identified by such as nasal provocation test. However, this test cannot be widely used in the outpatient diagnosis and treatment, therefore, there will be individual cases of diagnostic errors. The artificial intelligence system is designed to help diagnosis, but it cannot completely replace the rhinologist. This study is a dual-center study conducted at Tongji Hospital of Tongji University and Anting Branch Hospital. There may be a selection bias. In the future, a multi-center study should be conducted to improve the database required for training artificial intelligence systems and improve their diagnostic capabilities. Finally, through the self-learning of the system, it can help junior doctors complete the diagnosis of AR and improve their diagnosis ability.
Ethical disclosures Confidentiality of data
The authors declare that no patient data appears in this article. Right to privacy and informed consent. The authors declare that no patient data appears in this article. Protection of human subjects and animals in research. The authors declare that no experiments were performed on humans or animals for this investigation.
Funding
This work was supported by the National Science Foundationof China (grant no. 8187040043,81973749), Western Medicine GuideProject of Shanghai City (grant no. 17411970500), ShenkangMedical Development Center Clinical Science and Technology Innovation Project of Shanghai City (grant no. SHDC12019X07), Health Commission Advanced Technology Promotion Project of Shanghai City (grant no. 2019SY071).
Conflicts of interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this manuscript.