arcosin/doric

Doric is a column neural net and progressive neural net library built on pytorch.

---

Doric

---

Doric is a lightweight library for implementing and extending progressive neural networks (https://arxiv.org/pdf/1606.04671.pdf). It is a free and open-source library built atop PyTorch.

Doric utilizes user-made blocks to design column networks, which are then generated for new tasks in the prognet. We support inert blocks (blocks without laterals) and multi-channel blocks.

Thank you to Case Wright for help in developing Doric and writing all of our autoencoder tests, and to Molly Jacobson for the Doric logo.

Changes:

• 1/15/2021 -- 1.1.0: ProgRes update! Skip connections added. Changes made to multichannel blocks. Several utilities added.

---

doric's Issues

ProgDenseBlock, numLaterals and number of columns

Hi could you please explain a bit on what does the third argument of ProgDenseBlock: numLaterals mean? As I added more layers in generateColumn by defining more ProgDenseBlock as explained in https://towardsdatascience.com/progressive-neural-networks-explained-implemented-6f07366d714d under 'Simple Example' section, as long as it is more than one hidden layer I would get 'index out of range' error. Could you elaborate on how is numLaterals related to the number of columns? They are clearly not the same as I could run 30 columns with numLaterals = 0 with no error for 1 hidden layer case (in the sample code numLaterals is also 0 for two-column example). Thank you so much.

latLists or latList?

I haven't run this code yet but in the getData() methods you have a latLists empty list that you assign to data["lateral_states"] and a similar variable latList in the for loop. I'm not a Python dev but seems like these should be the same variable. Is this correct?

latLists = []
for lat in self.laterals:
    if lat is not None:   latList = [l.state_dict() for l in lat]
    else:                 latList.append(None)
data["lateral_states"] = latLists
return data

Doric/ProgBlocks.py

Line 341 in d1c0c35

for lat in self.laterals:

Adding Identity Module to the blocks.

Hello,
I love using your library, however, I find it missing a feature that I need for my implementation. Namely, the package does not support different sizes of consecutive columns and so in order to solve this I want to add identity layers as place holders so each column is of the same size.

Would you instruct me on how to do this?

Thanks!

Dependency problem via pip

With Python 3.10.6 i had a problem installing Doric via pip, on the dependency "pytorch".

I solved the problem downloading the .whl archive from pypi and changing the METADATA file at line 14, from:

Requires-Dist: pytorch

to:

Requires-Dist: torch

Then I recreated the .whl and installed with pip from local, and it works.

where is Doric?

cannot run

BackProp through Progressive nets

Hi, Thanks for putting in efforts to code the progressive nets. I have two questions:

1. I could not find the backpropagation method in the ProgColumn class. Do you have any plans of implementing it?
2. In the manuscript (Adapters section), the authors talked about multiplying the lateral activations with a learned scalar alpha before feeding them to another neural network (e.g., MLP or CNN). I do see that you are passing the lateral connections to the neural network but I could not find the learned scalar alpha anywhere.

save prognet

Hi, it seems the standard way of saving model does not work on prognet. Is there any specific way of doing so?

Thanks.

two column bug

Hi, when I'm running vae-cnn-two-network.py, the code will appear error Index 0 is out of range when I'm going to train the second column, and I found out the parameter for b16 should be len(parentCols) not 0 inorder to possess multi-column, Is this correct? Thanks for the repo, gain a lot!

l)