Randomly sample and save models

[shunlu91]

Randomly sample and save models

[ShunLu91]

Hi,
I think first you can load the weights into the supernet, and then inherit the weights from the supernet for each sampled architecture. Specifically, your can obtain the weights from the supernet for each selected operation per layer according to the predifined operation name.
Hope my answer will help you.

[abhishektyagi-t]

Thank you for your quick response.

I am actually a bit confused as to how I can implement the approach you mentioned above. Let me try to break it down as per my understanding:

1. First I will load the pre-trained weights of the supernet.
2. Once the weights for the supernet are loaded, I can generate a list (choice) that will select the random architecture

Now my questions are:

1. How do I do this: "Specifically, you can obtain the weights from the supernet for each selected operation per layer according to the predefined operation name."
2. Is it possible for me to save the random-selected model just like we can save any other model in PyTorch?

I appreciate your help!

[ShunLu91]

As for your two questions:

1. The supernet weights is a dict. Each selected operation per layer can be seen as a 'key' for this dict, so you can obtain its corresponding weights from this dict as long as you have the right operation name i.e. the 'key'. The operation name is usually pre-defined when you initialize the model.
2. It's not that convenient. You should first initialize a random-selected model, and then inherit weights for each operation in the random-selected model as above, and finally you can save this model with its weights as any other model in PyTorch.

If you have any questions, please feel free to ask.

[ShunLu91]

As there are no update recently, I will close this issue.
Feel free to re-open it if you have any question.