Comments (4)
Hi,
Great to hear that you are considering using Distiller!
Your question is packed, so I'll take it step by step:
-
Sure, you can use Distiller with any network, not just the models that come from TorchVision. You can see some examples of image classification models that we added here. The code that adds them is here.
-
If you add YoLov2, you will be the first to add an object detection model to Distiller, which will help other people! This might mean a bit of extra work for you, because you might have to adjust the image classification sample application to peform object detection. Therefore, it will probably be easier for you to simply write a new sample application, specifically for object detection, based on your current YoLo application.
When I added a network that learns a language-model I wrote a new sample application (because this is very different from image classification) and I also wrote a tutorial. This tutorial should give you a lot of details on how to add a new application. Basically, to get started you don't need much: you just add some callbacks to your exiting Pytorch YoLo application.
If you need help and guidance, feel free to post more questions and we will try to help.
Cheers,
Neta
from distiller.
@nzmora
Dear Mr. nzmora:
First, thanks a lot for your detailed solution for how to prune YoLov2. I feel encouaged to do this, alothough with big fear to fail.
-
For the past week, I spend two days looking into your tutorial, focusing on the difference between the raw main.py from pytorch/examples/word_language_model/ and your main in this project. I get the main idea of how to integrate my application with your distiller library. But, I have no idea with NLP(I am into CV), which makes me feel confused sometimes.
-
I spend three days learning all the pruning alogrithms in your model zoo. I want to choose L1-Norm, followed by networking thinning. Now, i can print the sparsity table of the model. Then I write a yaml file for compression. When i use
distiller.config.file_config(model, None, args.compress)
to config it, an error happened.
Fatal error while parsing [section:lr_schedulers] [item:exp_finetuning_lr]
Exception: <class 'TypeError'> NoneType is not an Optimizer
Fatal while creating lr_schedulers
Exception: <class 'TypeError'> NoneType is not an Optimizer
Actually i copied this part from your yaml file for network thinning.
-
Then I want to debug the compress_classifier.py as https://github.com/NervanaSystems/distiller/blob/master/examples/pruning_filters_for_efficient_convnets/resnet56_cifar_filter_rank.yaml did, so that i can see the detailed steps about parsing the yaml file. Then an error happened when it runs model.cuda. I know the default gpus = None, and i dont have GPU in my PC, but it still runs model.cuda. Is there anything wrong with this? I want to make sure my code can run and then i move to a server with 2 GPUs to do the experiment with SSH.
-
Now I have my version of yaml with target sparsity level using "pruning_filters_for_efficient_convnets" algorithm and a pretrained weights file. Since I have a pretrained model, I want to "physically" remove the pruned filters from the network, then I set the epoch=0 in policy. and start to finetune for epoch 1 to epoch 100. Is that ok? Can i visualize the process of finetuning? And for the net_thinner class, how to change the arch: 'resnet56_cifar' and dataset: 'cifar10' for my applicaition?
-
I wanna say something about python. It really makes life easier i think. But, sometimes it is too abstract and encapsulate to many details with so many libs like Pytorch. I like C, I can know what is the programs doing step by step. It is painful for me to fit Python. But, i have to acquire it, for it will be very useful in AI.
Last, thanks again for your your time reading my comments. I will appreciate it if you reply.
best regards!
Youteng Sun.
from distiller.
Hi Sun,
I think the easiest way to proceed is if you opened a PR on a branch (e.g. call this branch "yolov2") so that I can see (and execute) your code.
(2) You can attach the YAML file for me to look at. But sharing all of the code is better, because the problem can be in the code, and not in the YAML.
(3) We currently don't support training on the CPU. We want to add this feature, but it is low priority (since training most models takes a long time on a CPU compared to a GPU). Again, if you share your code, I can run it on a GPU to see if it works (but note that I'm going to be in Beijing some of the time and won't be able to help).
(4) Using thinning in epoch=0 and then fine-tuning sounds good.
Regarding "And for the net_thinner class, how to change the arch: 'resnet56_cifar' and dataset: 'cifar10' for my application?" - this is a good question. Is the dataset COCO? I think here we might need to make some changes in the main code to support this (I need to see your code ;-).
Cheers,
Neta
from distiller.
@nzmora
Hello, friend:
Before I want to use this distiller, I have to get a training program for yolov2. I downloaded it from github, but there is something wrong with it,(maybe i am wrong). After debug the yolov2 pytorch version for a long time, I gave up. And I decided to write a program to implement the algorithm <> based on Darknet (C language) and prune the filter physically. And i realy made it yesterday. Thanks for the talking with me. I will go learning disitiller when i take part in the job next year I think. It is happy to learn so much from this open project.
Hope you had a good trip in Beijing. And i am in Shanghai.
Best regards
cheers,
Sun.
from distiller.
Related Issues (20)
- Could you provide the checkpoints for structural pruning experiments?
- checkpoints example in example jupyter-notebook download denied. HOT 1
- Support for PyTorch 1.7? HOT 3
- Can't install pyglet, even when i cloned it form github
- Higher than 8-bit Quantization not working properly!?
- yolo4 custom object detection deep compression
- Why can't I use multi-GPU training
- How can I use the distilled model in embedded device?
- Combining quantization and pruning in Distiller
- Issue running compress_classifier.py HOT 1
- Reduce the yolov3 model size of keras(.h5) or darknet(.weight)
- Quantization don't reduce the model file size
- How to train my original dataset in distiller? HOT 1
- Error running 'pip install mintapi' on Raspberry Pi
- --load-serialized will make model fail to prune HOT 1
- QAT for LSTM
- outdated requirements? HOT 2
- Sensitivity Analysis
- Does it support translation model?
- Load quantization aware model checkpoint (inference) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from distiller.