simple-chat: error during loading params onto WebGPU, GPUPipelineError: A valid external Instance reference no longer exists

[lebron8dong]

simple-chat: error during loading params onto WebGPU, GPUPipelineError: A valid external Instance reference no longer exists

[CharlieFRuan]

Thanks for reporting the error. Could you show the log in console? Not sure if there is more info there. Besides, does this issue occur in all models? Could you perhaps try a smaller model like gemma 2b?

[lebron8dong]

Llama：log in console

WizardMath no error occurred；

[CharlieFRuan]

This is a bit strange. WizardMath has q4f16_1, could you also try Llama-2-7B-q4f16_1? Besides, how much RAM do you have? My guess is that it is some OOM issue. Looking at the field vram_required_MB in https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat/src/gh-config.js, Llama-2-7B-q4f32_1 requires roughly 2GB more vram than Llama-2-7B-q4f16_1 (hence also WizardMath).