xinference、qwen2的工具调用功能，对话中输出了思考过程

[JinCheng666]

xinference、qwen2的工具调用功能，对话中输出了思考过程

[zhanghx0905]

这个issue应该提到xinference下面，xinderence下个版本会支持，可以先试试qwen1.5
xorbitsai/inference#1642

[JinCheng666]

这个issue应该提到xinference下面，xinderence下个版本会支持，可以先试试qwen1.5 xorbitsai/inference#1642

感谢，尝试了qwen1.5-14b，确实可以正常使用。

刚升级了inference0.12.2，针对qwen2，确实不输出思考的过程了，但还是会多余地回复一些思考的中间值
@zhanghx0905

inference log

Question: 你是谁<|im_end|>
<|im_start|>assistant
Thought: I now know the final answer.
Final answer:  我是通义千问，由阿里云开发的AI助手。我被设计用来回答各种问题，提供信息和与用户进行对话。有什么我可以帮助你的吗？<|im_end|>
<|im_start|>user
Question: 获取当前时间<|im_end|>
<|im_start|>assistant
Thought: I can use m3VVjR.
Action: m3VVjR
Action Input: {}<|im_end|>
<|im_start|>function
Observation: {
  "time": "2024-06-21 19:08:26 Friday"
}<|im_end|>
<|im_start|>assistant
, generate config: {'temperature': 0.0, 'tool_choice': 'auto', 'stream': True, 'stop': ['<|endoftext|>', '<|im_start|>', '<|im_end|>'], 'stop_token_ids': [151643, 151644, 151645]}
2024-06-21 19:08:33,402 xinference.core.model 38993 DEBUG    After request chat, current serve request count: 0 for the model qwen:72b
2024-06-21 19:08:33,402 xinference.core.model 38993 DEBUG    Leave wrapped_func, elapsed time: 0 s
INFO 06-21 19:08:33 async_llm_engine.py:564] Received request 994ce71c-2fbe-11ef-b1f7-0cda411d272a: prompt: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nAnswer the following questions as best you can. You have access to the following APIs:\n\nm3VVjR: Call this tool to interact with the m3VVjR API. What is the m3VVjR API useful for? 获取用户当前时区的时间。 Parameters: [] Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [m3VVjR]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 你是谁<|im_end|>\n<|im_start|>assistant\nThought: I now know the final answer.\nFinal answer:  我是通义千问，由阿里云开发的AI助手。我被设计用来回答各种问题，提供信息和与用户进行对话。有什么我可以帮助你的吗？<|im_end|>\n<|im_start|>user\nQuestion: 获取当前时间<|im_end|>\n<|im_start|>assistant\nThought: I can use m3VVjR.\nAction: m3VVjR\nAction Input: {}<|im_end|>\n<|im_start|>function\nObservation: {\n  "time": "2024-06-21 19:08:26 Friday"\n}<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=['<|endoftext|>', '<|im_start|>', '<|im_end|>'], stop_token_ids=[151643, 151644, 151645], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1024, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.
INFO 06-21 19:08:36 async_llm_engine.py:133] Finished request 994ce71c-2fbe-11ef-b1f7-0cda411d272a.
2024-06-21 19:08:36,229 xinference.model.llm.utils 38993 DEBUG    Tool call content: 当前时间是2024年6月21日星期五19:08:26。, func: None, args: None

[JinCheng666]

xinference升级至0.12.3后，问题解决。感谢大家