"XR Animator" is our featured web app based on our System Animator Online. It uses the machine learning (ML) solution from MediaPipe and TensorFlow.js to detect the 3D poses from a live webcam video, which is then used to drive the 3D avatar (MMD/VRM model) as if you are controlling it with your body. It can be used for whatever XR purpose, and even for VTubing.
It has a variety of motion tracking options. You can choose to track the face, full body, or something in between (any combination of face/body/hands).
The online version works on all major web browsers both on desktop and smartphone. On browsers supporting both web worker and OffscreenCanvas (e.g. Chrome), it can achieve 60fps visual rendering and 30fps body pose detection on a medicore PC. On smartphones with limited processing power, you may want to use limit its usage on face tracking.
The Windows app version (powered by Electron) is also availabe for download, which provides a few extra features (e.g. VMC-protocol, transparent background) available only in a native-OS environment.
-
Support full-body AI motion tracking using a single webcam or media file (image/video)
-
Support using any MMD/VRM model as your 3D avatar
-
Record mocap motion and export it to VMD motion format
-
Support loading VMD/BVH/FBX format 3D motions
-
Export BVH/FBX motions to VMD format
-
Customize the background and 3D scene with 2D image/video , 3D panorama and 3D objects (.x/.glb format)
-
Support VMC-protocol to animate a 3D model elsewhere in other VMC-enabled applications such as VSeeFace, Unity and Unreal Engine (Electron mode only)
-
Support frameless window with transparent background on video capture apps such as OBS (Electron mode only) (*)
-
Support AR (Augmented Reality) on Android Chrome browser
Check out these YouTube video demos and watch XR Animator in action!
(*) - To capture the online version of XR Animator on OBS without the browser UI, you have to open XR Animator on OBS browser. A few extra command line parameters on OBS are required to allow camera access. Details
XR Animator and some other demos of System Animator Online support the "Augmented Reality" (AR) mode on mobile phones, which renders the 3D models that appear as if they exist in the real world. The AR mode requires mobile phones that support Google's ARCore technology, Chrome browser and the new WebXR API. Follow the steps below.
-
Check here for a list of ARCore-supported devices and see if your device is supported.
-
Install Google Play Services for AR (ARCore) on Google Play.
-
Install Chrome browser for Android.
Are you ready for the AR experience? Check out the online version of XR Animator on your Android Chrome browser!
After the page has been fully loaded, click on the little phone button on the top-left (or bottom-left) menu to activate the AR mode. Once the AR mode is enabled, you will see what your phone's camera is showing. Move your camera around the ground where you want to place the 3D model, and a white circle should apppear. Double-tap on the screen, and the 3D model will be placed over the white circle. Double-tab again to re-summon the white circle if you want to place the model elsewhere.
Check out these YouTube videos for demonstration.
XR Animator also supports the AR selfie feature, using both the frond camera (for AR) and the "selfie" camera, allowing you to take a selfie with the 3D avatar in AR mode.
Before you enter the AR mode, you need to click the Selfie icon and enable the Selfie mode first. Then you can toggle the Selfie mode at any time during the AR session. By default, Miku is always displayed in front of you in Selfie mode. Click the "BodyPix AI" icon to enable the AI mode, which detects your shape inside the camera, allowing Miku to blend into the background and appear to be behind you. Note that the AI mode is SLOW. You will probably need the latest Android phone to get a smooth frame rate.
You may also use the "BodyPix AI" directly and display the 3D model behind you without going through the AR mode.
Lastly, click the Snapshot icon and a still photo will be generated. This can be saved into your phone for whatever purpose you want, showing off to your friends perhaps!
Check out the following YouTube video for demonstration.
XR Animator inherits from System Animator, my desktop gadget project which have been active for many years. Lots of time, efforts and money have been spent to keep these projects free and running. If you like my works, please consider making a donation. Your financial support is needed to keep us going!🙏
-
🟡Donate via Bitcoin ( 1KkHVxgn4tusMhXNt1qFqSpiCiDRcqUh8p )
System Animator was originally a desktop gadget project, born more than 10 years ago. The latest version, System Animator Online, is a major version advancement with focus on working as a web app instead of being just a desktop gadget. It fully supports MikuMikuDance (MMD) models and motions, as well as the latest VRM models and FBX/BVH motions, to create an immersive 3D environment.
It's hard to describe what System Animator Online can do in a few words. From a simple animated CPU meter to an interactive 3D music visualizer, a simple AR gadget on your phone to a full-body motion tracking app on your PC, the possibility is endless.
For more information about the desktop gadget version of System Animator, please visit the following page. https://www.animetheme.com/sidebar/
System Animator was born more than 10 years ago as a personal and tiny 100-line-ish JavaScript desktop gadget project for Windows Vista which shows an animated rocket Anime girl as a CPU meter (the animation is still in XR Animator).
As time goes by, I decided to add more features, multi-purpose system meter, music visualizer, 3D/MMD support, animated wallpaper engine, RPG engine and eventually what you see in XR Animator. The codebase has grown exponentially while the core is still an Internet-Explorer-based JavaScript gadget, and things were becoming more and more clumsy, to a point when I had to decide whether to rewrite everything from scratch to match the modern coding standard (open source, module based, etc). However, I gave up and decided to carry on with what I have written, as a total restart would require too much time and efforts, probably not worthy as a personal project. Besides, as the rule of programming says, "If it works, don't touch it" LOL
Eventually, I decided to put the project on Github for my own convenience, but technically speaking you can consider it open source, though I have to admit that some of the codes are outdated, clumsy and confusing. Everything is fine if you are just an end-user of XR Animator/System Animator as an app, but if you want to build your own things from my codes, be warned that they can be pretty incomprehensible LOL
-
3D Miku The Dancer (drop any MP3 and she will dance for you)
-
3D Multiplayer RPG (up to 3 players)
All demos support the use of custom MMD (MikuMikuDance) model. Drop a zip of your favorite MMD model at the beginning, press the START button, and the demo will proceed with your model instead of the default one.
-
System Animator © Butz Yung/Anime Theme http://www.animetheme.com/sidebar/
- License (CC BY-NC-SA 4.0):
http://creativecommons.org/licenses/by-nc-sa/4.0/
- This license does NOT cover any third-party assets which may have incompatible licenses of their own.
- This license applies if you are adapting System Animator's source code for your own purpose, such as building another software or service.
- This license does not apply to content generated from the functionality of System Animator, such as video content generated from the motion capture feature of System Animator using your own assets. System Animator claims no right or responsibility over such content.
- Disclaimer: http://www.animetheme.com/system_animator_online/docs/disclaimer.txt
- License (CC BY-NC-SA 4.0):
http://creativecommons.org/licenses/by-nc-sa/4.0/
-
jThree v2 (NOTE: jThree has been discontinued. Its successor is known as "Grimoire.js")
-
ammo.js, a port of Bullet Physics to JavaScript, zlib licensed
-
JSZip (used under MIT license)
-
"Appearance Miku" MMD Model - Readme/License
-
Some texture/image/icon sources https://3dtextures.me/ https://opengameart.org/content/rpg-inventory https://opengameart.org/content/fantasy-icon-pack-by-ravenmore-0 https://opengameart.org/content/potion-bottles https://www.flaticon.com/ https://www.iconfinder.com/ https://icon-icons.com/en/pack/Social-Distancing/2274 https://github.com/icons8/flat-color-icons https://www.behance.net/gallery/41818673/FREE-SPORT-ICONS
-
Simple Explosion by Bleed https://remusprites.carbonmade.com/ https://opengameart.org/content/simple-explosion-bleeds-game-art
-
Various 3D background effects ported and modified from codes found on Shadertoy
-
For some other third-party programming libraries/3D data/assets used in System Animator, please refer to the corresponding script/readme for license and terms (can be found on the downloadable/Github version of System Animator).
-
もぐ式りょう/りく/りょく/りん by Mogg https://3d.nicovideo.jp/works/td55798 https://3d.nicovideo.jp/works/td55973 https://3d.nicovideo.jp/works/td56074 https://3d.nicovideo.jp/works/td56604
-
"Stranger Things" - A Remix ft. Michael Jobity https://soundcloud.com/foreignmachine/stranger-remix
-
Dragon Ball Super I Ultra Instinct OST I Clash of Gods Remix I Hip Hop Instrumental I @AndrezoWorks https://www.youtube.com/watch?v=KJ71dY4mkNo
-
Credits are given to the authors of any other image/media files used in System Animator.
-
Homepage: https://www.animetheme.com/sidebar/
-
Twitter: https://twitter.com/butz_yung
-
Email: [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent