simliai/radiodemo

Simli Radio Demo

This is a showcase of the Simli Lipsync API in action. The demo works by getting an mp3 radio stream, converting it to pcm16 using ffmpeg and sending the pcm16 frames to the Simli Lipsync API over websockets.

For the response spec, check Websockets.md

1. OpenCVRenderer.py: Uses OpenCV and PyAudio to render the video and audio on the screen.

Installation

0. Use python 3.11, anything lower is not guaranteed to work Fast enough.
1. Install FFMPEG
2. Create a virtual environment and install the requirements:

on Linux/MacOS

python3 -m venv .venv
source .venv/bin/activate

On Windows

python -m venv .venv
.venv\Scripts\activate

pip install -r requirements.txt

Usage

OpenCVRenderer.py

! Disclaimer: Moving the OpenCV window around can cause the audio to be out of sync with the video.

python OpenCVRenderer.py

To close just enter ctrl+c in the terminal.

radiodemo's Issues

Not working

Tried running it and it just crashes.

python OpenCVRenderer.py Received session: gAAAAABmgzkQcFrQKqS2z-9NZVspw4QY7jo65rWnz2-x1A1Y7VeB-ah_EA7obEoJzfUM4HTKwh6KesfMNgRJO9Mk5kfv56jWYj2RrKOHHDNwc7ImCY3zc6ysxCl2hdOk0CjL-oiqX-LON9NpPqZN2s4tgM-ZQvL1KFnPXtmGuekNPj7DQWchnEEFeEztJgUIo73acUyEs3jj3I6eHsyXZWZ4nFWezDz-AW1V3bPQfh-_AzVlFejYRjtudymuUWLUvR9VPYnNWJFQKjTCn7EgQuNpNo5LIXA_QHcj8iHp2WrMpgqjpLGB1gymJ-GRUpO4NEcIwgM2DnxiH-CllupQUCQU56oY4AfyNS-TU53V0sBem8ZHYnDdgKXszjsFODhC9Ztpulhkb3YiS3Uxc6JEm10UkU5nF8Z-DImatFEX3ThAR4CzAgDp5zsUFm4Q7UmHLEAQxngxqdft1xx6ZWbcfbGexi6w_Wh3BpITw-J59IvPmTjnUuel1xTAT7dpqdrWnWvtC21cuo-VqD-VmhEPx4ZpxiWGoHieCgh6yyHGKFzN1CYMroX08AOr9ukmCpJj5b1WCu0WY-efswSllTk4wcptIAKKBU0fDCl6hbJ2UvGZG07k6ci74Q_rUEmkgIcdWwDYqZujXxlC-XjfqNl7VEIFy_KYCt9QEvAbUNBbfDekhRdYS3yxJNWcyDpl9t_WZ3Jv6cRD4MvWQ_kr05319X_B7r_uuBBiKw== Sent session token ffmpeg -nostdin -v error -i https://radio.talksport.com/stream -f s16le -acodec pcm_s16le -ar 16000 -ac 1 pipe:1 FFMPEG STARTED sent metadata SENDING DISCONNECTED received 1005 (no status received [internal]); then sent 1005 (no status received [internal]) Exception in thread Thread-1 (start_asyncio_loop): Traceback (most recent call last): File "/opt/homebrew/Cellar/[email protected]/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045, in _bootstrap_inner self.run() File "/opt/homebrew/Cellar/[email protected]/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "/Users/larsvagnes/work/RadioDemo/OpenCVRenderer.py", line 196, in start_asyncio_loop asyncio.new_event_loop().run_until_complete(main(frames)) File "/opt/homebrew/Cellar/[email protected]/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/Users/larsvagnes/work/RadioDemo/OpenCVRenderer.py", line 189, in main await asyncio.gather( File "/Users/larsvagnes/work/RadioDemo/OpenCVRenderer.py", line 128, in send await websocket.send(data) File "/Users/larsvagnes/work/RadioDemo/.venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 635, in send await self.ensure_open() File "/Users/larsvagnes/work/RadioDemo/.venv/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 939, in ensure_open raise self.connection_closed_exc() websockets.exceptions.ConnectionClosedOK: received 1005 (no status received [internal]); then sent 1005 (no status received [internal]) ^CTraceback (most recent call last): File "/Users/larsvagnes/work/RadioDemo/OpenCVRenderer.py", line 202, in <module> time.sleep(0.001) KeyboardInterrupt av_interleaved_write_frame(): Broken pipe [out#0/s16le @ 0x13ee35740] Error muxing a packet [out#0/s16le @ 0x13ee35740] Error writing trailer: Broken pipe