Type of the Paper Article
- Bu səhifədəki naviqasiya:
- ClientUpdate
|
|
Initialize | ||
|
|
for each round t = 1,2,… do | ||
|
|
|
m max(C · K, 1) | |
|
|
|
(random set of m clients) | |
|
|
|
for each round k ∈ in parallel do | |
|
|
|
|
ClientUpdate (k, ) |
|
|
|
| |
|
|
|
// Erratum4 | |
|
ClientUpdate(k, w): // Run on client k | |||
|
|
B (split into batches of size B) | ||
|
|
for each local epoch i from 1 to E do | ||
|
|
|
for batch b ∈ B do | |
|
|
|
|
w ← w − η∆ℓ(w; b) |
|
|
return w to server | ||
C is the fraction of clients (devices or servers with local data samples) that are randomly selected to participate in the computation during each round of training. E is the number of local epochs, or the number of times each client passes through its entire local dataset in each round. B is the size of the local minbatch that the client uses for its updates. If B = ∞, the entire local dataset is treated as a single batch. B = ꝏ (used in experiments) implies full local dataset is treated as the minibatch as given pseudo-code is given in Algorithm 1.
The main idea behind the Flower method is to ensure that the model parameters are updated evenly across all participants, regardless of the size and quality of their data. This is achieved by weighing the participants' contributions to the global model based on their data quality and the model performance on their local data. The Flower has several advantages compared to traditional federated learning methods as it ensures fairness in the model training by weighing the participants' contributions based on their data quality and the performance of the local model on their data. This helps to prevent the dominance of participants with larger and more diverse data, which can result in a suboptimal global model. Also, Flower incorporates differential privacy by adding noise to the aggregated gradients or weights before sending them to the central server. This helps to ensure the privacy of participants' data even if the central server is compromised.
One of its unique advantages is improved accuracy. By weighing the participants' contributions based on their data quality and the performance of the local model, Flower can improve the accuracy of the global model because the model parameters contributing the most to the global model are updated more frequently, resulting in a more accurate model. The Flower federated learning method provides a privacy-preserving and fair solution for training shared models on distributed data. It is beneficial in clinical settings where data are collected and stored in different hospitals or clinics.
Several hyperparameters in federated learning can impact the performance and convergence of the model. For example, the learning rate determines the size of the step taken toward the negative gradient during model parameter updates. Overshooting the optimal solution can occur with a high learning rate, while slow convergence can occur with a low learning rate. The number of communication rounds determines how often the model parameters are updated and aggregated between the participants and the central server. More communication rounds can result in better convergence as the local batch size also determines the number of examples each participant uses to calculate the gradients or weights for its local model. They follow regularization, which adds a penalty term to the loss function to prevent overfitting. This can help improve the model’s generalization performance, especially when dealing with insignificant amounts of data. The distribution of data across the participants can impact the performance and convergence of the model. A skewed distribution, where one participant has significantly more data than others, can result in suboptimal convergence. The last parameter of federated learning is that the heterogeneity of the data across the participants can impact the convergence and generalization performance of the model. This includes differences in the data's distribution, quality, and label balance.
Yüklə 331,48 Kb.
Dostları ilə paylaş:
