ML model
|
Number of rounds
|
Number of clients
|
Train acc (F1)
|
Test acc (F1)
|
Random Forest
|
5
|
3
|
97.7
|
94.3
|
10
|
5
|
98.9
|
98.9
|
15
|
10
|
97.2
|
90.3
|
Logistic Regression
|
5
|
3
|
93.2
|
90.1
|
10
|
5
|
94
|
92.3
|
15
|
10
|
92.3
|
89.3
|
SGD
|
5
|
3
|
70.1
|
65.3
|
10
|
5
|
75.4
|
60.3
|
15
|
10
|
70.3
|
68.3
|
AdaBoost
|
5
|
3
|
97.6
|
90.3
|
10
|
5
|
97.4
|
92.3
|
15
|
10
|
90.1
|
87.2
|
Gaussian
|
5
|
3
|
80.1
|
80.3
|
10
|
5
|
89.7
|
78.3
|
15
|
10
|
82.3
|
74.3
|
T he results of this study indicate that the highest accuracy across all ML models was achieved when using 10 communication rounds and 5 clients in the FL environment. Figure 3 illustrates the experimental result of classification as this optimal combination was observed for Random Forest, Logistic Classifier, Stochastic Gradient Descent (SGD), Ada-Boost, and Gaussian Naïve Bayes in the context of clinical event classification.
Figure 3. Optimal performance achieved with 10 rounds and 5 clients for various machine learning models.
The results of this study demonstrate a significant improvement in classification accuracy compared to other research approaches in the field of clinical event classification, as shown in Table 6. The method used, incorporating FL, achieved an impressive 98.9% accuracy, outperforming all other methods investigated. This finding highlights the effectiveness and potential of FL in enhancing the performance of ML models for clinical event classification. The superior performance of the FL-based method can be attributed to its ability to leverage distributed datasets, maintain data privacy, and facilitate collaborative learning among multiple clients. This approach allows for the development of robust models that can generalize better and adapt to diverse data sources, leading to improved classification accuracy.
Table 6. Superior performance of federated learning-based method in clinical event classification.
|
Research in [29]
|
Research in [30]
|
Research in [31]
|
Our model
|
Number of fixtures
|
6
|
1
|
2
|
6
|
Vital signs
|
HR, BP, RR, SPO
|
BP
|
HR, BP
|
HR, BP, RR, SPO
|
Clinical event
|
Any
|
Any
|
Any
|
Any
|
Number of normal samples
|
1300
|
30
|
571
|
145085
|
Number of abnormal samples
|
130
|
30
|
116
|
117073
|
Accuracy
|
95.5 average
|
94%
|
ROC max 0.86
|
98.9
|
Federated learning
|
No
|
No
|
No
|
Yes
|
5. Conclusions
The classification of clinical events using vital signs data is crucial in healthcare, as it allows for the early detection and management of various medical conditions. This study employed FL to classify clinical events using vital signs data, utilizing datasets from multiple clients of X hospital, and employing cross-device ensemble ML classification models, such as Random Forest, AdaBoost, and SGD. Flower FL offered several advantages for clinic event classification, including privacy-preserving capabilities, enabling collaboration between multiple parties to train ML models, and safeguarding the privacy of each party's data. This happens because each party is only required to share encrypted model updates with other parties rather than sharing raw data. Furthermore, Flower FL is designed to scale to many participants, making it particularly suitable for clinic event classification problems where multiple hospitals or clinics may have data to contribute. By combining data and insights from multiple parties, Flower FL can help improve the ML model's performance for clinic event classification because the model can leverage the combined data and insights from various sources. By aggregating model updates from multiple parties, Flower FL can help make ML models for clinic event classification more robust and less susceptible to overfitting to a single party's data. Overall, this study demonstrated that using Flower FL for clinic event classification with ML classification can significantly improve the performance and robustness of the model while preserving the privacy of each party's data. This makes it an essential tool for addressing complex and sensitive healthcare problems.
This study achieved a high accuracy rate of 98.9% for clinic event classification on the MIMIC IV dataset. A client management system is planned to proactively address errors during training, such as data quality or communication issues. This system will ensure that each client's data is adequately incorporated into the ML model for clinical event classification using vital signs data, further improving accuracy and robustness. This study also aims to explore advanced techniques, such as deep learning and ensemble methods, to enhance model performance. These future developments will make this study’s approach even more valuable for healthcare providers and researchers.
Funding: This work was supported in part by the Commercialization’s Promotion Agency for R&D Outcome (COMPA) Grant funded by the Korean Government (MSIT) under Grant 2022-Future research service development support-1-SB4-1, and in part by the National Research Foundation of Korea (NRF) Grant funded by MSIT under Grant NRF-2022R1F1A1069069.
Dostları ilə paylaş: |