This repository is the official implementation of the paper “FedMHO: Heterogeneous One-Shot Federated Learning Towards Computing Resource-Constrained Clients”.
**Note: **
**We use PyTorch with MPI backend for a Master-Worker computation/communication topology. **
The code and environment are based on FedDF: https://github.com/epfml/federated-learning-public-code.
Train FedMHO-MD on EMNIST
python run.py --arch vgg9 --complex_arch master=vgg9,worker=vgg9:cvae,num_clients_per_model=5 --experiment heterogeneous --data emnist --pin_memory True --batch_size 64 --num_workers 8 --partition_data non_iid_dirichlet --non_iid_alpha 0.5 --train_data_ratio 1 --val_data_ratio 0 --n_clients 10 --participation_ratio 1 --n_comm_rounds 1 --local_n_epochs 200 --world_conf 0,0,1,1,100 --on_cuda True --fl_aggregate scheme=semi_supervised,clean_rate=0.8 --optimizer sgd --lr 1e-3 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 --lr_scheduler MultiStepLR --lr_decay 0.01 --weight_decay 0 --use_nesterov False --momentum_factor 0.9 --track_time True --display_tracked_time True --python_path /home/dzyao/anaconda3/envs/pytorch/bin/python --hostfile hostfile --manual_seed 2023 --pn_normalize False --same_seed_process True --use_hog_feature False --pin_memory False --latent_size 20 --num_synthesis_sample 12000 --is_md True
Train FedMHO-SD on EMNIST
python run.py --arch vgg9 --complex_arch master=vgg9,worker=vgg9:cvae,num_clients_per_model=5 --experiment heterogeneous --data emnist --pin_memory True --batch_size 64 --num_workers 8 --partition_data non_iid_dirichlet --non_iid_alpha 0.5 --train_data_ratio 1 --val_data_ratio 0 --n_clients 10 --participation_ratio 1 --n_comm_rounds 1 --local_n_epochs 200 --world_conf 0,0,1,1,100 --on_cuda True --fl_aggregate scheme=semi_supervised,clean_rate=0.8 --optimizer sgd --lr 1e-3 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 --lr_scheduler MultiStepLR --lr_decay 0.01 --weight_decay 0 --use_nesterov False --momentum_factor 0.9 --track_time True --display_tracked_time True --python_path /home/dzyao/anaconda3/envs/pytorch/bin/python --hostfile hostfile --manual_seed 2023 --pn_normalize False --same_seed_process True --use_hog_feature False --pin_memory False --latent_size 20 --num_synthesis_sample 12000 --is_sd True
Train FedAvg on EMNIST
python run.py --arch vgg9 --complex_arch master=vgg9,worker=vgg9:cnn,num_clients_per_model=5 --experiment heterogeneous --data emnist --pin_memory True --batch_size 64 --num_workers 8 --partition_data non_iid_dirichlet --non_iid_alpha 0.5 --train_data_ratio 1 --val_data_ratio 0 --n_clients 10 --participation_ratio 1 --n_comm_rounds 1 --local_n_epochs 200 --world_conf 0,0,1,1,100 --on_cuda True --fl_aggregate scheme=federated_average_ensemble --optimizer sgd --lr 1e-3 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 --lr_scheduler MultiStepLR --lr_decay 0.01 --weight_decay 0 --use_nesterov False --momentum_factor 0.9 --track_time True --display_tracked_time True --python_path /home/dzyao/anaconda3/envs/pytorch/bin/python3.8 --hostfile hostfile --manual_seed 2020 --pn_normalize False --same_seed_process True --use_hog_feature False --pin_memory False
Train FedDF on EMNIST
python run.py --arch vgg9 --complex_arch master=vgg9,worker=vgg9:cnn,num_clients_per_model=5 --experiment heterogeneous --data emnist --pin_memory True --batch_size 64 --num_workers 8 --partition_data non_iid_dirichlet --non_iid_alpha 0.5 --train_data_ratio 1 --val_data_ratio 0.1 --n_clients 10 --participation_ratio 1 --n_comm_rounds 200 --local_n_epochs 1 --world_conf 0,0,1,1,100 --on_cuda True --fl_aggregate scheme=noise_knowledge_transfer,update_student_scheme=avg_logits,data_source=other,data_type=train,data_scheme=random_sampling,data_name=mnist,data_percentage=1.0,total_n_server_pseudo_batches=6000,eval_batches_freq=200,early_stopping_server_batches=400 --optimizer sgd --lr 1e-3 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 --lr_scheduler MultiStepLR --lr_decay 0.01 --weight_decay 0 --use_nesterov False --momentum_factor 0.9 --track_time True --display_tracked_time True --python_path /home/dzyao/anaconda3/envs/pytorch/bin/python3.8 --hostfile hostfile --manual_seed 2023 --pn_normalize False --same_seed_process True --use_hog_feature False --pin_memory False
Besides, we are based on DENSE's official code with minor modifications to suit our experimental setup, and the code is provided in the DENSE-code folder.