daod/inters

This is the repository for our paper "INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning"

👋 About me

• I am currently a postdoctoral researcher at the Gaoling School of Artificial Intelligence, Renmin University of China. I earned my PhD from University of Montreal, where I was mentored by Prof. Jian-Yun Nie.
• I completed my master's (2019) and bachelor's (2016) degrees at Renmin University of China, under the guidance of Prof. Zhicheng Dou and Prof. Ji-Rong Wen, delving into various NLP challenges.
• Research interests: Retrieval-augmented generation, large language models for information retrieval, session-based document ranking

📎 Homepages

• Personal page: https://daod.github.io/
• Google scholar: https://scholar.google.com/citations?user=tBqVOWsAAAAJ
• DBLP: https://dblp.org/pid/71/9704-1.html

🔥 News

• 2024.5: We publish a new toolkit ⚡FlashRAG, which can help implement RAG methods quickly! See more details.
• 2024.5: Congrats! Our three papers have been accepted by ACL 2024!
• 2024.4: We write a new survey about generative information retrieval. See more details.
• 2024.1: We propose a new instruction tuning dataset (INTERS) for unlocking the power of LLMs on search tasks. See more details.
• 2023.11: We analyze the risk of data leakage in LLM pre-training and write a new paper to alert this problem. See more details.
• 2023.8: We write a new survey about applying large language models for information retrieval. See more details.
• 2023.8: We publish a new version of YuLan-Chat. It achieves better performance than the official LLaMA-2 and LLaMA-2-Chat on MMLU, C-Eval, and AGI-Gaokao benchmarks!

inters's Issues

Dataset availbility

Hey, are you planning to share the dataset?

How can I use this in practice?

This seems very interesting! How can I use it in practice? I've read the arxiv paper as well as the info here and on Hugging Face, and I am struggling to understand how I can use an INTERS fine tuned model to improve information retrieval in practice. Could you describe some of the intended use cases? For example,

1. Can I use it as an alternative to vector search - if so, how?
2. Is the intention that an INTERS model is used to reformulate a prompt/query? In what scenarios would I want my system to reformulate a user's prompt/query?

Code Requests for paper

Hello author, your work is written very well. I am very interested in your work. May I ask when the source code will be open? I can't wait to study it.

Embedding vs parametric knowledge

Work focused only on fine-tuning LLMs in order to make them perform better at IR task only using its parametric knowledge.

But usually, embeddings will be generated for query and documents and using some nearest neighbour algorithm, we will find the top relevant documents.

Would like to know your thought process behind this, as to why you went with the parametric knowledge as opposed to the resorting to embedding generation and approximate nearest neighbour search.

Load model through hugging face error

Hi,

I hope you are doing well!

I got an error when trying to load the model using the following code:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained('yutaozhu94/INTERS-Falcon-1b')

I am wondering if my way of loading the model is wrong? If so, I would appreciate if you can provide some code example of how
one can quickly use it through hugging face API! The error I got is as follows.

Thank you and have a nice day!

---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:532, in load_state_dict(checkpoint_file, is_quantized)
    [531](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:531)     weights_only_kwarg = {"weights_only": True} if is_torch_greater_or_equal_than_1_13 else {}
--> [532](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:532)     return torch.load(
    [533](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:533)         checkpoint_file,
    [534](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:534)         map_location=map_location,
    [535](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:535)         **weights_only_kwarg,
    [536](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:536)         **extra_args,
    [537](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:537)     )
    [538](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:538) except Exception as e:

File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1025, in load(f, map_location, pickle_module, weights_only, mmap, **pickle_load_args)
   [1024](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1024)     except RuntimeError as e:
-> [1025](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1025)         raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
   [1026](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1026) return _load(opened_zipfile,
   [1027](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1027)              map_location,
   [1028](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1028)              pickle_module,
   [1029](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1029)              overall_storage=overall_storage,
   [1030](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/torch/serialization.py:1030)              **pickle_load_args)

UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported class typing.OrderedDict

During handling of the above exception, another exception occurred:

UnicodeDecodeError                        Traceback (most recent call last)
File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:541, in load_state_dict(checkpoint_file, is_quantized)
    [540](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:540) with open(checkpoint_file) as f:
--> [541](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:541)     if f.read(7) == "version":
    [542](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:542)         raise OSError(
    [543](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:543)             "You seem to have cloned a repository without having git-lfs installed. Please install "
    [544](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:544)             "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
    [545](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:545)             "you cloned."
    [546](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:546)         )

File <frozen codecs>:322, in decode(self, input, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
Cell In[4], [line 2](vscode-notebook-cell:?execution_count=4&line=2)
      [1](vscode-notebook-cell:?execution_count=4&line=1) from transformers import AutoModelForCausalLM
----> [2](vscode-notebook-cell:?execution_count=4&line=2) model = AutoModelForCausalLM.from_pretrained('yutaozhu94/INTERS-Falcon-1b')

File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:563, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    [561](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:561) elif type(config) in cls._model_mapping.keys():
    [562](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:562)     model_class = _get_model_class(config, cls._model_mapping)
--> [563](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:563)     return model_class.from_pretrained(
    [564](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:564)         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    [565](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:565)     )
    [566](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:566) raise ValueError(
    [567](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:567)     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    [568](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:568)     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    [569](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:569) )

File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3531, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   [3522](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3522)     if dtype_orig is not None:
   [3523](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3523)         torch.set_default_dtype(dtype_orig)
   [3524](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3524)     (
   [3525](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3525)         model,
   [3526](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3526)         missing_keys,
   [3527](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3527)         unexpected_keys,
   [3528](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3528)         mismatched_keys,
   [3529](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3529)         offload_index,
   [3530](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3530)         error_msgs,
-> [3531](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3531)     ) = cls._load_pretrained_model(
   [3532](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3532)         model,
   [3533](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3533)         state_dict,
   [3534](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3534)         loaded_state_dict_keys,  # XXX: rename?
   [3535](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3535)         resolved_archive_file,
   [3536](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3536)         pretrained_model_name_or_path,
   [3537](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3537)         ignore_mismatched_sizes=ignore_mismatched_sizes,
   [3538](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3538)         sharded_metadata=sharded_metadata,
   [3539](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3539)         _fast_init=_fast_init,
   [3540](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3540)         low_cpu_mem_usage=low_cpu_mem_usage,
   [3541](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3541)         device_map=device_map,
   [3542](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3542)         offload_folder=offload_folder,
   [3543](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3543)         offload_state_dict=offload_state_dict,
   [3544](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3544)         dtype=torch_dtype,
   [3545](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3545)         hf_quantizer=hf_quantizer,
   [3546](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3546)         keep_in_fp32_modules=keep_in_fp32_modules,
   [3547](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3547)     )
   [3549](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3549) # make sure token embedding weights are still tied if needed
   [3550](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3550) model.tie_weights()

File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3938, in PreTrainedModel._load_pretrained_model(cls, model, state_dict, loaded_keys, resolved_archive_file, pretrained_model_name_or_path, ignore_mismatched_sizes, sharded_metadata, _fast_init, low_cpu_mem_usage, device_map, offload_folder, offload_state_dict, dtype, hf_quantizer, keep_in_fp32_modules)
   [3936](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3936) if shard_file in disk_only_shard_files:
   [3937](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3937)     continue
-> [3938](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3938) state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
   [3940](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3940) # Mistmatched keys contains tuples key/shape1/shape2 of weights in the checkpoint that have a shape not
   [3941](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3941) # matching the weights in the model.
   [3942](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3942) mismatched_keys += _find_mismatched_keys(
   [3943](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3943)     state_dict,
   [3944](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3944)     model_state_dict,
   (...)
   [3948](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3948)     ignore_mismatched_sizes,
   [3949](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:3949) )

File /mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:553, in load_state_dict(checkpoint_file, is_quantized)
    [548](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:548)             raise ValueError(
    [549](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:549)                 f"Unable to locate the file {checkpoint_file} which is necessary to load this pretrained "
    [550](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:550)                 "model. Make sure you have saved the model properly."
    [551](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:551)             ) from e
    [552](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:552) except (UnicodeDecodeError, ValueError):
--> [553](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:553)     raise OSError(
    [554](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:554)         f"Unable to load weights from pytorch checkpoint file for '{checkpoint_file}' "
    [555](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:555)         f"at '{checkpoint_file}'. "
    [556](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:556)         "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True."
    [557](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/d/yuchenxi/UDEM/diro/IFT6255/IR_LLM_user_intent_alignment/alignir_env/lib/python3.11/site-packages/transformers/modeling_utils.py:557)     )

OSError: Unable to load weights from pytorch checkpoint file for '/home/yuchen/.cache/huggingface/hub/models--yutaozhu94--INTERS-Falcon-1b/snapshots/381906a51044c1d79cf1d9066d85f35fd6bfe874/pytorch_model-00001.bin' at '/home/yuchen/.cache/huggingface/hub/models--yutaozhu94--INTERS-Falcon-1b/snapshots/381906a51044c1d79cf1d9066d85f35fd6bfe874/pytorch_model-00001.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

数据集还有模型啥时候开放呢

很棒的工作，主页的模型还有数据链接，现在都不能访问，请问这个大概啥时候会开放呢