abh15/fedml

(PyTorch > 1.0) A Research-Oriented Federated Learning Library. Supporting distributed computing, mobile/IoT on-device training, and standalone simulation

FedML: A Research Library and Benchmark for Federated Machine Learning

📄 https://arxiv.org/abs/2007.13518

News

2020-11-05 (System): Do you want to run federated learning on IoT devices? FedML architecture design can smoothly transplant the distributed computing code to the IoT platform. FedML can support edge training on two IoT devices: Raspberry Pi 4 and NVIDIA Jetson Nano!!! Please check it out here: https://github.com/FedML-AI/FedML/blob/master/fedml_iot/README.md

2020-10-28 (Algorithms) : We released more advanced federated optimization algorithms, more than just FedAvg! http://doc.fedml.ai/#/algorithm-reference-implementation

2020-10-26 (Publication) : V2 of our white paper is released. Please check out here: https://arxiv.org/pdf/2007.13518.pdf

2020-10-07 (Model and Dataset) : Datasets + Models ALL IN ONE!!! FedML supports comprehensive research-oriented FL datasets and models:

• cross-device CV: Federated EMNIST + CNN (2 conv layers)
• cross-device CV: CIFAR100 + ResNet18 (Group Normalization)
• cross-device NLP: shakespeare + RNN (bi-LSTM)
• cross-device NLP: stackoverflow (NWP) + RNN (bi-LSTM)
• cross-silo CV: CIFAR10, CIFAR100, CINIC10 + ResNet
• cross-silo CV: CIFAR10, CIFAR100, CINIC10 + MobileNet
• linear: MNIST + Logistic Regression

Please check create_model(args, model_name, output_dim) and load_data(args, dataset_name) at fedml_experiments/distributed/fedavg/main_fedavg.py for details.

We will support more advanced models and datasets, please stay tuned!

---

2020-09-30 (Publication): We maintained a comprehensive publication list of Federated Learning here: https://github.com/chaoyanghe/Awesome-Federated-Learning

---

2020-09-28 (Publication): Authors of FedML (https://fedml.ai) have 7 papers that got accepted to NeurIPS 2020. Big congratulations!!! Here is the publication list: https://github.com/FedML-AI/FedML/blob/master/publications.md. Highlighted ones are related to large-scale distributed learning and federated learning.

What is Federated Learning?

Please read this long vision paper Advances and Open Problems in Federated Learning.

This publication list is also helpful: https://github.com/chaoyanghe/Awesome-Federated-Learning

Introduction

Federated learning is a rapidly growing research field in the machine learning domain. Although considerable research efforts have been made, existing libraries cannot adequately support diverse algorithmic development (e.g., diverse topology and flexible message exchange), and inconsistent dataset and model usage in experiments make fair comparisons difficult. In this work, we introduce FedML, an open research library and benchmark that facilitates the development of new federated learning algorithms and fair performance comparisons. FedML supports three computing paradigms (distributed training, mobile on-device training, and standalone simulation) for users to conduct experiments in different system environments. FedML also promotes diverse algorithmic research with flexible and generic API design and reference baseline implementations. A curated and comprehensive benchmark dataset for the non-I.I.D setting aims at making a fair comparison. We believe FedML can provide an efficient and reproducible means of developing and evaluating algorithms for the federated learning research community. We maintain the source code, documents, and user community at https://FedML.ai.

For more details, please read our full paper: https://arxiv.org/abs/2007.13518

Usage

1. Research on FL algorithm or system
2. Teaching in a ML course
3. System prototype for industrial production.
4. Self-study FL: understanding code level details of FL algorithms.

Architecture

The functionality of each package is as follows:

fedml_core: The FedML low level API package. This package implements distributed computing by communication backend like MPI, and also support topology management. Other low-level APIs related to security and privacy are also supported.

fedml: The FedML high level API package. This package support different federated learning algorithm with only one line code. All algorithms are built based on the "fedml_core" package. Users can change this package to add more advanced algorithms.

fedml_mobile: This package is used to support on-device training using Android/iOS smartphones.

fedml_experiments: This package is used to test algorithms in "fedml" package by calling high level APIs.

benchmark: This package is used to run benchmark experiments.

applications: This package is a collection of applications based on FedML.

Join our Community

Please join our community. We will post updated features and answer questions on Slack.

Join fedml.slack.com (this is a link that never expires)

Contributing

We sincerely welcome contributors and believe in the power of the open source. We welcome expertise from two tracks, either research or engineering.

1. If you are a researcher who needs APIs that our library does not support yet, please send us your valuable suggestions.

2. If you are a researcher who has published FL-related algorithm or system-level optimization, we welcome you to submit your source code to FedML, which will then be maintained by our engineers and researchers.

3. If you are an engineer or student who is searching for interesting open source projects to broaden your career, FedML is perfect for you. Currently, we are developing the following urgent features.

i) transplanting more advanced FL algorithms to FedML. We will show you some important research publications once you are involved. For this role, we prefer engineers or students who have a basic understanding of machine learning.

ii) FedML-Mobiel service architecture: Flask + PyTorch + RabbitMQ

iii) upgrading our Android and iOS platform.

iv) building or applying more models in computer vision and NLP domains to FedML.

v) collecting realistic federated datasets by crowdsourcing.

Please email us for further information.

Citation

Please cite FedML in your publications if it helps your research:

@article{chaoyanghe2020fedml,
  Author = {He, Chaoyang and Li, Songze and So, Jinhyun and Zhang, Mi and Wang, Hongyi and Wang, Xiaoyang and Vepakomma, Praneeth and Singh, Abhishek and Qiu, Hang and Shen, Li and Zhao, Peilin and Kang, Yan and Liu, Yang and Raskar, Ramesh and Yang, Qiang and Annavaram, Murali and Avestimehr, Salman},
  Journal = {arXiv preprint arXiv:2007.13518},
  Title = {FedML: A Research Library and Benchmark for Federated Machine Learning},
  Year = {2020}
}

Contacts

The corresponding author is:

Chaoyang He
[email protected]
http://chaoyanghe.com