I got empty result while using 7b-it model

[google]

I got empty result while using 7b-it model

[pengchongjin]

@egbertwong could you please provide the command you used to reproduce this?

[egbertwong]

@egbertwong could you please provide the command you used to reproduce this?

Hi, thanks for your reply! I just follow the steps in the README file. And I also write my commands down here:

VARIANT=7b
CKPT_PATH=/mnt/d/Code/gemma/gemma-7b-pytorch/gemma-7b-it-quant.ckpt

sudo usermod -aG docker $USER
newgrp docker

DOCKER_URI=gemma:${USER}

docker build -f docker/Dockerfile ./ -t ${DOCKER_URI}

PROMPT="The meaning of life is"

docker run -t --rm \
    --gpus all \
    -v ${CKPT_PATH}:/tmp/ckpt \
    ${DOCKER_URI} \
    python scripts/run.py \
    --device=cuda \
    --ckpt=/tmp/ckpt \
    --variant="${VARIANT}" \
    --prompt="${PROMPT}"

[pengchongjin]

It seems you are using quantized checkpoint, please make sure that you also add --quant in the command. I think it should work.

[egbertwong]

It seems you are using quantized checkpoint, please make sure that you also add --quant in the command. I think it should work.

Thanks, it worked!