The following `model_kwargs` are not used by the model: ['token_type_ids']

[dudu-upstream]

The following `model_kwargs` are not used by the model: ['token_type_ids']

[fedem96]

I had the same problem, I solved it by using --model_generation_type llama-text-generation

[calam1]

I ran %pip install transformers==4.31.0 according to https://huggingface.co/NumbersStation/nsql-llama-2-7B/discussions/1 and the following curl worked for me on my MBPro(intel). I think the person who opened the issue is also the same person in the huggingface discussion, but cross posting here for others.

curl --location 'http://127.0.0.1:5002/completions'
--header 'Content-Type: application/json'
--data '{
"prompt": "Hello World",
"max_tokens": 1024,
"temperature": 0.0,
"repetition_penalty": 1,
"top_k": 50,
"top_p": 10,
"do_sample": "True",
"n": 1,
"max_new_tokens": 1024
}'