Comments (1)
How many GPUs on the machine do you have? Can you try the command from the readme (Benchmarking, Section 5 "The code now supports synchronous distributed training ..." and share the error message?
# for single node 8 gpus and nccl as backend on randomly generated dataset:
python -m torch.distributed.launch --nproc_per_node=8 dlrm_s_pytorch.py --arch-embedding-size="80000-80000-80000-80000-80000-80000-80000-80000" --arch-sparse-feature-size=64 --arch-mlp-bot="128-128-128-128" --arch-mlp-top="512-512-512-256-1" --max-ind-range=40000000 --data-generation=random --loss-function=bce --round-targets=True --learning-rate=1.0 --mini-batch-size=2048 --print-freq=2 --print-time --test-freq=2 --test-mini-batch-size=2048 --memory-map --use-gpu --num-batches=100 --dist-backend=nccl
from dlrm.
Related Issues (20)
- how to inference ./dlrm_s_criteo_kaggle.sh HOT 1
- Getting Nan loss when training dlrm with Kaggle Criteo dataset HOT 7
- Question regarding the pooling in QR trick HOT 1
- Issue with Activating UVM Function in torchrec_dlrm
- Size of embedding tables in MLPerf checkpoint
- Segmentation Fault on M1 Mac. HOT 1
- Help with installation. No module named caffe2
- Docker Build failing HOT 1
- Unable to preprocess Criteo Kaggle Display Advertising Challenge Dataset
- What's the purpose of torchrec_dlrm/?
- Correctly changing precision in DLRM HOT 1
- Multi-GPU training does not converge HOT 2
- Bind to cores for inference test
- mini-batch-size can impact inference-only test
- Why does embedding vectors could have various dimensions?
- Significant Variability in DLRM Benchmark Performance Metrics
- Accuracy of Quotient Remainder Method
- 'batched' function looks buggy HOT 1
- test.txt problem HOT 1
- torchrec run failed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dlrm.