Comparison with hidet

[mlc-ai]

Comparison with hidet

[junrushao]

Several of us are super familiar with Hidet developers. In fact, I believe @spectrometerHBH and @yaoyaoding are classmates :-)

Both TVM and Hidet support efficient code generation on CUDA targets (e.g. A100, Jetson Xavier), and both are as performant as existing solutions like cutlass/Triton. There is some difference, for example, older generation of TVM suffers from long tuning time, imperfect search space, lack of support for dynamic shape, etc, but fortunately, the new stack we are relying on doesn't.

On the other hand, MLC LLM aims to democratize local LLM computation, which means the scope is broader and we wanted to support AMD/Intel/Apple GPUs, x86/arm CPUs and other hardware platforms. On those non-NV targets, we found that TVM is the only existing solution that works out of the box. In fact, if you check out the binaries we release, Vulkan is good enough to support CUDA targets for personal use of LLMs.

[austinmw]

That’s very helpful, thanks for your response!