oneMKL `optimize_gemv` too slow for one-shot SpMV

[cwpearson]

oneMKL `optimize_gemv` too slow for one-shot SpMV

[gajanan-choudhary]

@cwpearson, you are right. oneapi::mkl::sparse::optimize_gemv() was a no-op for previous versions of oneMKL. Last few releases, though, it stopped being a no-op. In the current release of oneMKL (2024.0), if you expect to call oneapi::mkl::sparse::gemv() only a single time with a given matrix handle then, calling oneapi::mkl::sparse::optimize_gemv() would very likely be detrimental to performance, and you could remove that optimize_gemv() call. The cost of the call to optimize_gemv() is generally amortized over multiple calls to gemv() with the same matrix handle (e.g., you call optimize_gemv() once, followed by say, a hundred gemv() calls). The oneMKL team will try reducing the cost of optimize_gemv() in a future oneMKL release.

[cwpearson]

Thank you @gajanan-choudhary! We have a PR in progress to remove optimize_gemv for our Aurora users. We are considering a two-phase SpMV API, and would revisit this, but it does not exist yet.

[gajanan-choudhary]

Hi @cwpearson, just a quick update: I recently worked on trying to improve the performance of oneapi::mkl::sparse:;optimize_gemv(). I was able to significantly improve the performance (at least for "non-tiny" matrices) for the sycl::buffer APIs. Some bottleneck still exists in the USM APIs that we are still trying to figure out. These improvements will pop up in the oneMKL 2024.2 release (not the upcoming 2024.1 release) in a few months.

We will continue looking into improving the performance of both optimize_gemv and one-shot sparse::gemv() without optimize_gemv() in future releases.