raise ValueError("No available memory for the cache blocks. " ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

[DAAworld]

raise ValueError("No available memory for the cache blocks. " ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

[JustinLin610]

Level up the --gpu-memory-utilization . vllm works differently as it will take a lot of memory of the device

[sparkfax]

Level up the --gpu-memory-utilization . vllm works differently as it will take a lot of memory of the device
already use --gpu-memory-utilization 0.95, how to level up?