Comments (5)
LLaVA-NeXT/playground/demo/video_demo.py
Line 160 in 6944062
try to change this line to
output_ids = model.generate(inputs=input_ids, images=video, attention_mask=attention_masks, modalities="video", do_sample=True, temperature=0.2, max_new_tokens=1024, use_cache=True)
from llava-next.
Hi, what is the version of your transformers?
from llava-next.
Thanks!
import transformers
print(transformers.version)
4.40.0.dev0
from llava-next.
My TF version is 4.39.0 and the conv model should be mistral_instruct for LLaVA-NeXT-Video-7B-32K.
BTW, there is "attention_dropout" in "https://huggingface.co/lmms-lab/LLaVA-NeXT-Video-7B-32K/blob/main/config.json"
from llava-next.
Thanks @ZhangYuanhan-AI !
-
What is the "attention_dropout" and how it solves the problem above "AttributeError: 'LlavaMistralConfig' object has no attribute 'attention_bias'"?
-
Additionally, I downgraded the TF from 4.40.0.dev0 to 4.39.0 and the same problem is still there.
-
what is the command you use to call LLaVA-NeXT-Video-7B-32K model?
(for others it is: bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B-DPO vicuna_v1 32 2 True ./data/llava_video/video-chatgpt/evaluation/Test_Videos/v_Lf_7RurLgp0.mp4) -
I added the following 3 lines for config, not sure whether they are correct or not:
setattr(cfg_pretrained, 'attention_bias', 0)
setattr(cfg_pretrained, 'rope_scaling', {"factor": 8.0, "type": "linear"})
setattr(cfg_pretrained, 'pretraining_tp', 1)
However, it is not giving any response
Time taken for inference: 2.013814687728882 seconds
Question: [INST]
Please provide a detailed description of the video, focusing on the main subjects, their actions, and the background scenes [/INST]
Response:
from llava-next.
Related Issues (20)
- output of the demo code HOT 1
- videos of LLaVA-NeXT-interleave HOT 1
- When will mm_use_im_start_end be implemented in pre-training?
- LLaVA-NeXT-Interleave Training Details HOT 3
- how to get results? HOT 1
- Do we have some inference accelerate method for new llava-next-video models? HOT 1
- Eval results HOT 6
- How many A100s used for training? HOT 1
- Is LLaVA-NeXT-interleave 7B model availble? HOT 6
- Question about M4-Instruct datasets HOT 3
- Question regarding multi image inference - import vs demo HOT 3
- where is python3 llavavid/eval/eval_activitynet_qa.py? HOT 2
- question about the demo implementation HOT 2
- When will the training code be available? HOT 7
- Training dataset
- Requiremet File HOT 3
- Eval Results HOT 5
- Any plans to support vLLM?
- Can we add preprocessor_config.json for llava-next-interleave-qwen-7b model on Huggingface? HOT 1
- Chinese OCR Fine-tuning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llava-next.