bornfree/talking_avatar

A nice 3D avatar that can speak input text with facial expressions

Talking avatar

A ThreeJS-powered virtual human being that uses a set of neat Azure APIs to do some talking!

ta.webm

Add Chat GPT to the mix and maybe you can have for yourself a nice face to chat with. 🙂

Run

$ yarn install
$ yarn start

Backend

You'll also need https://github.com/bornfree/talking_avatar_backend for the text to speech conversion.

talking_avatar's Issues

less teeth?

I was wondering how to show less of teeth of the model ? (maybe just me but it feels too much )

2d Character option

Hi there, great initiative.

I was just wondering how to integrate 2d characters in the system?

Functions of Avatar

Hi, great project, I'd love to develop it with you, I have a brilliant idea on how to use it and I hope that with this option it will be possible!!!

A few questions needed for implementation:

1. How to integrate an avatar with chat gpt?

2. Is it possible to change the language in which the avatar speaks?

3. Is it possible to change the avatar's voice?

4. Is it possible to change the appearance of the avatar to a new one? If so, can I insert a real person here (based on photos/video) or one generated on another platform with a specific file extension such as .glb or .mhb?

5. Is it possible to change the avatar's voice?

6. What can I do to be able to talk to the avatar in real time like in the @bornfree film?

I would be very grateful for any help!
Best regards!

Example only shows the model's hair

Hi,

Thank you for a nice example on integrating the text to audio.

I can receive the audio and Viseme, but the model is not loading correctly.

Would you be so kind to show me how to display the entire model?

2d characters

The azure documentation mentions that 2d characters can work with this. Have you tried it?

cuustom character

hello
this is awesome project
how can I create my own character? I need Men , Women and child character
regards

—-

Not speaking in Mozila

It works very well in Microsoft Edge, and in Chrome the avatar take a little bit late than the sound. The wors in Mozila it is not speaking at all, just running.

THREE.WebGLRenderer: Context Lost.

The voice file has been produce but hit the error and the page become grey after click speak.
three.module.js:46363 Uncaught TypeError: Cannot read properties of undefined (reading 'uuid') at AnimationMixer.clipAction (three.module.js:46363:1) at App.js:262:1 at arrayEach (lodash.js:530:1) at Function.forEach (lodash.js:9410:1) at App.js:261:1 at commitHookEffectListMount (react-reconciler.development.js:14669:1) at commitPassiveMountOnFiber (react-reconciler.development.js:16531:1) at commitPassiveMountEffects_complete (react-reconciler.development.js:16495:1) at commitPassiveMountEffects_begin (react-reconciler.development.js:16482:1) at commitPassiveMountEffects (react-reconciler.development.js:16470:1) clipAction @ three.module.js:46363 (anonymous) @ App.js:262 arrayEach @ lodash.js:530 forEach @ lodash.js:9410 (anonymous) @ App.js:261 commitHookEffectListMount @ react-reconciler.development.js:14669 commitPassiveMountOnFiber @ react-reconciler.development.js:16531 commitPassiveMountEffects_complete @ react-reconciler.development.js:16495 commitPassiveMountEffects_begin @ react-reconciler.development.js:16482 commitPassiveMountEffects @ react-reconciler.development.js:16470 flushPassiveEffectsImpl @ react-reconciler.development.js:19142 flushPassiveEffects @ react-reconciler.development.js:19095 (anonymous) @ react-reconciler.development.js:18891 workLoop @ scheduler.development.js:266 flushWork @ scheduler.development.js:239 performWorkUntilDeadline @ scheduler.development.js:533

Feature Request : Hindi Language Support

Please tell me how to add hindi language support

Are there any completely open-source implementation?

Are there any open source implementation of the whole pipeline needed for such Avatar Video that can be run locally and/or deployed to different cloud?

Blend json - how to create?

Noticed you are using .json for animation?

what software did you use to create then export the .json animation?

I’ve tried a few software packages and cannot locate on to export the .json animation data

Speak button not working?

hi, how come when I click on the Speak button, it does not talk? the button flickers (I think it says running or a fraction of a second). see video. thanks

https://www.loom.com/share/4f3444181adb46d881dfc5e456ab8ef8?sid=e2b1650f-b922-4f42-9cbe-711a1c5735e5

Avatar is not speaking

Hi @bornfree ,

I have npm install and npm start the code. The avatar is not speaking is it require to put the Azure Speech Service Key and Region? If yes, where should I put? I couldn't find the KEY that I need to put.

Video Download option

Hi there, I am working on this project.

Could you please tell me how can I download the video that I create?

How to create character ?

Everythings looks fine but looking to change digital human character. Can you please provide documentation how can we create ?