zju-vipa/kamalengine

Knowledge Amalgamation Engine

kamalengine's Issues

Can I update some new features?

Hello! I read the paper and want to implement the code, and I want to add some KA features on vit for image classification. My ideas are as below:
Provide a comprehensive and robust pipeline and a data preprocessing method that allows training with own datasets.
Implement KA on vision transformers to accomplish image classification tasks.(finetuned on several datasets such as Stanford Dogs)
Provide a multimodal transfer method to apply KA for audio classification.

mismatches of definitions

Is this model not finished or what? I was trying the examples and there's been many mismatches of definitions.
For example
KamalEngine/kamal/slim/prunning/strategy.py", line 23, in
_PRUNABLE_MODULES= tp.DependencyGraph.PRUNABLE_MODULES
AttributeError: type object 'DependencyGraph' has no attribute 'PRUNABLE_MODULES'

how to train each block?

Question about details of Task Branch

Hi, thank you for sharing such an awesome project. I have some questions about the details in your code, and I sincerely hope you can help me solve them.

1 In your paper, it is said that "We conduct knowledge amalgamation for each block of TargetNet" (Section 5.1.1), however, in the code, it seems that you only apply the knowledge amalgamation on the decoder blocks. This makes me feel confused that whether we need to apply it on the encoder blocks, as in the "layerwise amal".

2 In the task branch code, should the object of student be JointSegNet, instead of BranchySegNet? As in BranchySegNet, the network does not use teacher's blocks.
Since after running the code on NYUv2, I only got a target net with mIOU 0.2830 by merging the depth model (RMSE: 0.6772) and the seg model (MIOU: 0.5227) with BranchySegNet. When I replace it with JointSegNet, I can get a target net with mIOU 0.5233.

I would really appreciate it if you could help me. Look forward to your reply, thanks!

请问有没有该模型的详细说明？

请问有没有该模型使用步骤的详细说明？

Citation for Recombination KA Algorithm (#4)

Is there a link to the paper for the 4. Recombination algorithm (as seen in README)?