The logprobs.token_logprobs section is described as "

this has been fixed <a class="user-mention notranslate" data-hovercard-type="user" dat

What exactly are logprobs in the API? about openapi HOT 1 CLOSED

johny-b commented on August 29, 2024

What exactly are logprobs in the API?

from openapi.

Comments (1)

orangetin commented on August 29, 2024

this has been fixed @johny-b !

first example:

>>> chat_completion = client.chat.completions.create(
...     messages=[{"role": "user", "content": "but why?"}],
...     model=model,
...     max_tokens=4,
...     logprobs=1,
...     temperature=1,
... )
>>> print(chat_completion.choices[0].logprobs)
ChoiceLogprobs(content=None, token_ids=[345, 2438, 2079, 1110], tokens=[' "', 'But', ' why', '?"'], token_logprobs=[-0.3569336, -0.037475586, -8.34465e-07, -0.009811401])

second example:

>>> chat_completion = client.chat.completions.create(
...     messages=[{"role": "user", "content": "but why?"}],
...     model=model,
...     max_tokens=2,
...     logprobs=1,
...     temperature=1,
... )
>>> print(chat_completion.choices[0].logprobs)
ChoiceLogprobs(content=None, token_ids=[345, 2438], tokens=[' "', 'But'], token_logprobs=[-0.3569336, -0.036346436])