Details on bone initialization, skinning function and training schedule

[facebookresearch]

Details on bone initialization, skinning function and training schedule

[gengshan-y]

Hi Hang, thanks for the questions/suggestions. I'm hoping to improve the readability of the codebase in the future but there's no concrete plan. I'll try to get to your questions and let me know if I missed anything.

1. Bone initialization: generate_bones(...) has not effect in the optimization. The bone initialization is actually done by nutils/geom_utils.py/reinit_bones(...), called at nnutils/train_utils.py:

   banmo/nnutils/train_utils.py

   Lines 1077 to 1081 in 12a19e5

   	if opts.lbs and opts.model_path=='' and \
   	(epoch==int(self.num_epochs*opts.reinit_bone_steps) or\
   	epoch==0 or\
   	epoch==int(self.num_epochs*opts.warmup_steps)//2):
   	reinit_bones(self.model.module, mesh_rest, opts.num_bones)

   For details of bone re-initialization, see "Sec. 4 Experiments - Optimization details" and "Fig. 15 ablation study" in the updated pdf. Some details/ablations were added to the original arxiv version.

2. Skinning: They are equivalent indeed. The scalars multiplied to mdis should be merged to scale in the future for readability.

3. Multiple stages: The multi-stage optimization will be described in the camera-ready version. In short, the result of the first stage is typically good enough. Stage 2-3 improve the limb articulations (e.g., to correctly articulate the crossing legs for cat-pikachiu) with coordinate descent. Stage 4 further improves details of the geometry. Bones are not re-initialized in stage 2-4.

[hangg7]

Thank you for your clarification!