Priority Task : start using trulens to evaluate Gemini

[tonic-ai]

Priority Task : start using trulens to evaluate Gemini

[Josephrp]

big thank you to 🏆😎 @MN-Noor for producing the first TruLens with gemini on RAG using open ai!

Open tasks :

• Make a notebook to test Gemini MultiModal (image inputs)
• Make a notebook to test more models against Gemini
• Make a notebook to test the "new features of Gemini" like the censorship level.

we'll all work on this together, normally if everyone does one, or at least contributes to a good one we will have secured this task.

[Zochory]

Est-ce que l'on ajouterait pas d'autres multimodal LLM ?
comme celui ci dans les evals ? https://huggingface.co/sshh12/Mistral-7B-LoRA-ImageBind-LLAVA

[Josephrp]

hey there @mie-h and @Zochory : https://github.com/Tonic-AI/DataTonic/tree/main/evaluation this is a folder where we will first start working on the trulens evaluations which are a hackathon requirement + good practice while building an app 🫡

[Josephrp]

hey there @mie-h & @Zochory : i added default prompts to the baseline prompts folder we can use those in a trulens evaluation.

[Josephrp]

consider using this to generate "system prompts" for gemini

[Josephrp]

added an incomplete example notebook : https://github.com/Tonic-AI/DataTonic/blob/main/evaluation/results/modelcomparision.ipynb