Source LLMs

[eladlev]

Source LLMs

[Eladlev]

The system support running local LLMs with Langchain hugging face pipelines

We didn't test it in the paper, we definitely intend to expand the paper and provide more experiments

[jzyxn]

How to use local LLMs?

[Eladlev]

You should change in the relevant config file:

        llm:
            type: 'HuggingFacePipeline'
            name: <The name of the model>
            max_new_tokens: <max tokens>

If you want also that the optimizer will be local (highly not recommended), then you should change the meta prompts folder to:

meta_prompts:
    folder: 'prompts/meta_prompts_completion'

[jzyxn]

Thanks for the reply, I will try to see if it works.