Conservative reshaping

[WilliamTambellini]

Conservative reshaping

[dzarukin]

Hi @WilliamTambellini, please use DNNL_VERBOSE=all to see an explanation to the error you observe.
It says to me onednn_verbose,primitive,create:check,reorder,inconsistent src and dst mds,src/common/reorder.cpp:82.

Conservative reshape should be done through a dedicated function of memory::desc. Reorder does perform memory copy regardless of inputs since memory arguments are expected to be different.

[WilliamTambellini]

Tks @dzarukin
Though:

• as said in my initial post, desc::reshape does NOT do much except returning a new desc, it does NOT reshape a memory obj.
• if reorder is not the right pri to use, which one is it ?

That ticket is by definition not completed: you did not show evidence yet as today that a memcopy free reshape is doable with onednn 3.2.1. With 3.2.1, there is no way to change the desc of an existing memory. Creating a new mem object would mean memcopy the data.
Could you please reopened the ticket till resolved ?
Best regards
WT

[dzarukin]

it does NOT reshape a memory obj.

The memory object is not modified simply because oneDNN can't do any assumptions whether the original memory object is needed or not. It's always required to create a new object as the user is allowed to attach same handle over different memory objects.

The following piece should serve well.

    memory::desc md_orig(...);
    memory m_orig(md_orig, engine); // allocation happens here
    ... // filling/modifying m_orig.data()
    memory::desc md_reshaped(...);
    memory m_reshaped(md_reshaped, engine, m_orig.get_data_handle()); // Since md doesn't change data order, can re-apply the handle.

The piece above requires m_orig to be alive to keep the handle allocated. It's safer to provide a handle for m_orig though for juggling memory objects as needed.

[WilliamTambellini]

Tks @dzarukin
I was keeping that solution as last option because, as you said, many drawbacks, as imposing the user to keep alive the old original memory and of course the new created one.
Considering a conservative resize could be needed multiple times in the lifetime of a tensor, it would mean accumulating zombie memory objects just to keep the data alive.
I thought about fully refactoring our onednn plugin to never use onednn allocation but using the handle ctor but I hope you can see all the additional logic that it would require, plus being unable to take advantage of ondnn mem allocation system (gpu or cpu, best aligned for the running cpu, ...), having to reimplement a mem allocation system.
A cleaner, high perf and user friendly solution would be to simply be able to move the handle(s) from an existing memory object to another one, the original memory not owning the data anymore.
Is it possible with 3.2.1 ?
Best regards
W.

[WilliamTambellini]

Reminder: that issue is not closed: the algo is not doable today with 3.2.1.

[dzarukin]

If you want to migrate handles between memory objects, you must allocate memory on your end and manage it manually.
For convenience, you probably may insert a move ctor into dnnl namespace or make a lightweight function that will perform transition.
The memory allocated by the library can't be managed externally and there's no scenario where the library would detach the handle and make user de-allocate it.
Since the memory creation without handle is cheap, oneDNN team doesn't see a need to do anything else.