Cannot load hugging face dataset for function calling

[sebastiangonsal]

Cannot load hugging face dataset for function calling

[CharlieJCJ]

Hi @sebastiangonsal,

We detailed dataset accessing instructions in this section of the README: https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard#prepare-evaluation-dataset. Please download the huggingface dataset through this procedure, which is fully tested and functional.

You can load the dataset through the helper function defined in this section: https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/eval_checker/eval_runner_helper.py#L409C1-L415C18. The dataset is organized in JSON files, where each file representing a test category. The helper function loads a JSON file into a list of dictionaries.

We are looking into ways to be compatible with the HuggingFace datasets package shortly to provide alternative ways to access the dataset. Thanks for raising this to our attention! In the meantime, please try out the way we outlined above. Thanks!

Best,
BFCL Team