thu-ml/hide-prompt

Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality (NeurIPS 2023, Spotlight)

hide-prompt's Issues

Two quesions on training

Thanks so much for your wonderful open-source work!

Here, I have the following questions Could you please give me some hints?

Q1: TII task problem: In trainers.tii_trainer, the TII task here employs the class-level classification task along with the TAP task. Why do we not use a task-level classification task for TII?

Q2: Information leakage for two-stage training: The whole training process is two-stage training. In the first stage, train an FC layer with a frozen backbone on TII (use class classification) and TAP tasks. In the second stage, the previously trained model is used for task inference only. The model trained in the first stage has already met all tasks. Does this trained model bring information leakage, since it knows future task information?

Looking forward to your help.