18520339/ml-distributed-training

Distributed training with Multi-worker & Parameter Server in TensorFlow 2

Hi, I'm Quan

• 👨‍🎓 Master student at UTS.
• 📚 I'm pursuing AI with the desire to develop most advanced solutions to solve real-life issues, make people's lives convenient & greatly contribute to my national language.
• 😭 I'm looking for help with Statistics.
• ⚡ Fun fact: I love Astronomy 🛰️.
• 🔔 Subscribe me on Youtube.

⚡ Technologies and Skills:

• Web Scraping (Python, JS).
• Web Development (MERN Stack).
• Machine Learning (TensorFlow).
• Network Engineering (CCNA).
• Microsoft Office (Excel, VBA).

💻 Notable projects:

🎬 Popular YouTube Videos (Click Here)

	Facebook crawling with Python Jul 7, 2020
	Phần mềm Quản lý học sinh THPT Feb 4, 2021
	VLSP 2018 - Aspect-based Sentiment Analysis for Vietnamese reviews Apr 15, 2020
	Chat Application with Pusher Chatkit Apr 7, 2020
	Distributed Database - ArangoDB Cluster Setup Jun 22, 2021
	Excel VBA - Phần mềm quản lý kho (Warehouse Management) Apr 8, 2020
	C# Winform - Tic tac toe (Game Caro) Apr 15, 2020

---

Visitor's count 👀

ml-distributed-training's Issues

Role of ps and worker

Hey there!
Just wanted to clarify something regarding the code for ps and worker. I've recently started working with such distributed training, so pardon my silly queries.

As much as I've come to know, ps serve parameters to the workers while the later one fetches them. Aside from the difference in the tf_config, I've noticed no code for fetching/serving of parameters particularly dedicated to only ps or only workers. Both share the same code.

I wanted to know how are they coordinating with one another?

A question about parameter server training

Hi, your code really helps! I have one question:
In the coordinator(train_dataset_fn), you use shard to split data to each worker, but the input param(input_context.input_pipeline_id) indicates which worker index is, so I think every worker should call the function(train_dataset_fn) to get his part of data. But your code show that only the coordinator use the train_dataset_fn function.
Can you explain to me how this param(input_context.input_pipeline_id) works
thx!!!