Comments (6)
I implement a LLava-llama3 Lora finetuning https://github.com/chuangchuangtan/LLaVA-NeXT-Image-Llama3-Lora
from llava-next.
I implement a LLava-llama3 Lora finetuning https://github.com/chuangchuangtan/LLaVA-NeXT-Image-Llama3-Lora
Great thank you! Does it also work with Llama3 70b? Btw, does it train only the bridger and language model, or does it also train the vision encoder (that we want to avoid)? Can we train without LoRA ?
It only trains the bridge and language model. We have set up to print the names of trainable parameters in the code, you can check them. We haven't tested it on 70b, but it should be work. You can set training commands to train without LoRA.
from llava-next.
That would be great to get the training scripts, as it was done in the original LLaVA repo :)
from llava-next.
I'd also love to use them for fine-tuning with several images, for few-shot image classification.
from llava-next.
what do you guys think of this ?
https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LLaVa/Fine_tune_LLaVa_on_a_custom_dataset_(with_PyTorch_Lightning).ipynb
By replacing llava by lava-next (processor and model)
from llava-next.
I implement a LLava-llama3 Lora finetuning https://github.com/chuangchuangtan/LLaVA-NeXT-Image-Llama3-Lora
Great thank you! Does it also work with Llama3 70b?
Btw, does it train only the bridger and language model, or does it also train the vision encoder (that we want to avoid)? Can we train without LoRA ?
from llava-next.
Related Issues (20)
- output of the demo code HOT 1
- videos of LLaVA-NeXT-interleave HOT 1
- When will mm_use_im_start_end be implemented in pre-training?
- LLaVA-NeXT-Interleave Training Details HOT 3
- how to get results? HOT 1
- Do we have some inference accelerate method for new llava-next-video models? HOT 1
- Eval results HOT 6
- How many A100s used for training? HOT 1
- Is LLaVA-NeXT-interleave 7B model availble? HOT 6
- Question about M4-Instruct datasets HOT 3
- Question regarding multi image inference - import vs demo HOT 3
- where is python3 llavavid/eval/eval_activitynet_qa.py? HOT 2
- question about the demo implementation HOT 2
- When will the training code be available? HOT 7
- Training dataset
- Requiremet File HOT 3
- Eval Results HOT 5
- Any plans to support vLLM?
- Can we add preprocessor_config.json for llava-next-interleave-qwen-7b model on Huggingface? HOT 1
- Chinese OCR Fine-tuning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llava-next.