Unable to export Phi-3-vision model to PyTorch exported program

[zewenli98]

Unable to export Phi-3-vision model to PyTorch exported program

[amyeroberts]

Hi @zewenli98, thanks for opening this issue!

I don't think this is a bug in either PyTorch or Hugging Face. When one uses torch.export, it's trying to trace the model/function/callable to produce a traced graph i.e. something which can effectively be compiled or serialized. Not all code is tracing compatible. In particular, things like variable shapes, unknown or changing input and output types and certain logic controls cannot be traced. In the error, we can see the tracing breaks at this line (implies variable tensor sizes and a logic condition).

As the modeling code is on the hub, it's the repo's authors who can update it. If you wish for the model to be exportable, I'd suggest opening a discussion on the checkpoint's community tab, requesting this feature.