shan-mx/video-clip-indexer

Video-CLIP-Indexer

A GUI short-video 'clip' indexer in 60 lines.

Basic Usage

1. Install requirements

pip install -r requirements.txt

2. Run the streamlit GUI

streamlit run app.py

Parameters

Text Prompt

You can use a prompt to describe the scene you want to search for. The indexer will return several clips related to it.

Top N

The number of video clips you want to be returned.

Cut Sim

Approximately from 0.44 to 0.6. The smaller the number, the longer the video clips(with lower precise).

CLIP-as-service Server

The url of CLIP-as-service Server. The default value is a demo server loaded with ViT-L/14-336px provided by Jina.ai. You can also run your own CLIP-as-service server.

video-clip-indexer's Issues

搭建好服务后，发现报错如下

You can now view your Streamlit app in your browser.

Network URL: http://y.y.y.y:8501
External URL: http://x.x.x.x:8501

╭────────── 🆕 New version available! ───────────╮
│ You are using clip_client 0.4.20, but 0.8.2 is │
│ available. You may upgrade it via pip install │
│ -U clip_client. Read Changelog here. │
╰────────────────────────────────────────────────╯
UserWarning: failed to set default signal handler: ValueError('signal only works in main thread of the main interpreter')` (raised from /usr/local/lib/python3.9/dist-packages/jina/init.py:146)
E0821 11:14:27.951663108 1326 hpack_parser.cc:992] Error parsing 'content-type' metadata: invalid value
2023-08-21 11:14:27.960 Uncaught app exception
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/jina/clients/base/grpc.py", line 87, in _get_results
async for resp in stub.Call(
File "/usr/local/lib/python3.9/dist-packages/grpc/aio/_call.py", line 344, in _fetch_stream_responses
await self._raise_for_status()
File "/usr/local/lib/python3.9/dist-packages/grpc/aio/_call.py", line 251, in _raise_for_status
raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Stream removed"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2023-08-21T11:14:27.951837898+00:00", grpc_status:2, grpc_message:"Stream removed"}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/streamlit/scriptrunner/script_runner.py", line 557, in _run_script
exec(code, module.dict)
File "app.py", line 23, in
spams, ndarray, scores = search_frame(keyframe_data, text_prompt, int(topn_value), cas_url)
File "/home/seedkuny/Video-CLIP-Indexer/helper.py", line 29, in search_frame
r = client.rank([d])
File "/usr/local/lib/python3.9/dist-packages/clip_client/client.py", line 382, in rank
self._client.post(
File "/usr/local/lib/python3.9/dist-packages/jina/clients/mixin.py", line 184, in post
return run_async(
File "/usr/local/lib/python3.9/dist-packages/jina/helper.py", line 1330, in run_async
return asyncio.run(func(*args, **kwargs))
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/dist-packages/jina/clients/mixin.py", line 175, in _get_results
async for resp in c._get_results(*args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/jina/clients/base/grpc.py", line 138, in _get_results
raise BadClient(msg) from err
jina.excepts.BadClient: gRPC error: StatusCode.UNKNOWN Stream removed

网页如下：