m3-org/avatar-interop

avatars, wearables, and animation interop research

Avatar Interop Group

This meetup group is dedicated to avatar interoperability R&D. We experiment with interoperable file formats, wearables, and animation systems between game engines and virtual worlds. While testing we also aim to provide useful feedback to other open metaverse standards working groups, such as the Metaverse Standards Forum, OMI scripting group and the OMI glTF-extensions group.

Wiki — Discussions — Discord — Twitter — Calendar

---

Calendar

📆 View the monthly M3 calendar.

We meet weekly on Tuesdays at 8pm EST starting in the M3 discord. You can add this event to your calendar here or by using this calendar URL: https://calendar.google.com/calendar/ical/f9lsrdg825i8ivgifmjts8s4lo%40group.calendar.google.com/public/basic.ics

• Monday: Virtual production writer's room
• Tuesay:Avatar Interop Meetup
• Wednesday: Metaverse Standards Forum + OMI group community call
• Thursday: MetaFactory community call
• Friday: Neon Buidl
• Satuday: Clear

---

Current Projects

• M3taloot (3D loot wearables): https://twitter.com/dankvr/status/1507189063694131202
• Open source avatar builders: https://hackmd.io/@XR/avatarbuilders
• CloneX Experiments: https://hackmd.io/@XR/clonexperiments
• Webaverse MMO: https://hackmd.io/@xr/webaverse-mmo
• MetaFactory (digi-physical wearables): https://hackmd.io/@xr/mf_notes

---

Past Events

Note: Starting to combine this list with virtual production meetups because of how much the groups overlap See full list of past events here: https://github.com/m3-org/research#m3-20

• 2-8-22 Avatar Interop meetup 1
  • Video: https://www.twitch.tv/videos/1293166793
• 2-15-22 Avatar Interop meetup 2
  • Video: https://www.youtube.com/watch?v=VQgFZdjka2w
• 3-8-22 Avatar Interop meetup 3
  • Video: https://www.youtube.com/watch?v=bPiRNrFJsW8
• 3-15-22 Avatar Interop meetup 4
  • Videos: #8 (comment)
• 3-22-22 Avatar Interop meetup 5
  • Video: https://www.youtube.com/watch?v=lxV5GK8CJRU
• 4-7-22 M3 x Metaverse Crew event
  • Video: https://www.youtube.com/watch?v=ls21IwX94YE
• 4-8-22 Demo Day 1 w/ Sith, Jin, A
• 4-22-22 Demo Day 2 w/ M2, Jin, Robert
  • Jins Slides
  • Robert presenting Third Room: https://www.youtube.com/watch?v=zl_e2ySrRfw
• 4-21-22 - 5-28-22 MF VR Fashion Shoots
  • M3 grant for interoperable avatars + wearables and metaverse fashion shoots
• 7-19-22 Loot: Emergent Artwork @ M3 Avatar Interop Group
• 8-2-22 Avatar randomizer update + Interactive wearable overlay apps
• 8-9-22 Avatar Interop Group Weekly
  • What could NPCs do? by A
  • Pets in Anarchy Arcade by SM SithLord
  • Hyperfy NPCs and quests by Ash
• 8-16-22 VRM avatar workshop by jin
• 8-22-22 Avatar Interop meetup 8-22
  • Notes
• 8-29-22 Vprod meetup notes
• 8-30-22 Field trip into Hyperfy w/ VRMs
• 9-13-22 Avatar Interop
• 9-19-22 VPROD
• 9-20-22 Avatar Interop
• 10-3-22 VPROD
• 10-10-22 VPROD
• 10-17-22 VPROD

---

FAQ

How do I join M3?

I recommend reading this and starting with this guide: https://github.com/M3-org/git-gud

Can I present a project?

Yes, feel free to hop on the discord and talk about it first though. Out of courtesy, it's recommended to attend atleast 1 meetup before being a presentor if you're a guest.

When is the next event?

We have a weekly public event on Tuesdays 1pm EST where we discuss and show n' tell avatar interop projects. Occasionally we also host demo day events when there are atleast 3 demos to share.

Where can I learn more?

Read up on areas of research we've covered and are currently exploring. Dev logs contain ongoing documentation so expect some rough edges.

• https://hackmd.io/@XR/avatars - tons of general info about avatar interop and VRM, #1 result on google for "avatar interoperability"
• https://hackmd.io/@XR/avatarlod - ways to optimize custom avatars to increase performance and CCU
• https://hackmd.io/@XR/wearables1 - notes on interoperable wearables systems
• https://hackmd.io/@XR/mocap-interop - notes on interoperable animation / mocap between game engines and platforms
• https://hackmd.io/@xr/avatarbuilders - notes on avatar builder programs we're building and ones that exist

Excellent tutorials:

• https://hackmd.io/@metamike/CloneX-VRM
• https://hackmd.io/@metamike/CloneX-Virtual-World-Files
• https://hackmd.io/@metamike/fbx-vrm-webaverse
• https://docs.webaverse.com/docs/art/clonex-to-vrm

Where to get avatars?

Learn the basics of making your own custom avatar from this workshop by jin, some more links to check out:

• https://readyplayer.me/
• https://vroid.com/en/studio
• https://booth.pm/
• https://www.100avatars.com/
• https://vrm.dev/en/vrm_applications/#platform-for-3d-models
• https://gitlab.com/s-ilent/SCSS/-/wikis/Other/Avatar-Models
• https://hackmd.io/@XR/nftavatars

Can you help me bring my art to life in 3D?

Possibly, if you're looking for help to make 3D avatars, pets, or vehicles ping jin#6455 in the Discord.

avatar-interop's Issues

Exploring the Polygon avatar / wearables ecosystem

Many virtual world crypto projects are using polygon for avatars and wearables, boasting strong network effects along with cheaper / faster transactions and mint fees which are super useful for lower cost accessories. It's worth diving into this ecosystem to see how it can become more interoperable.

• Webaverse
• Cryptovoxels
• Decentraland
• Sandbox
• Mona
• Spatial
• OnCyber
• NFT Worlds (rebooting)
• House of Kibaa
• Immersed
• Cryptoavatars

And that's just the social layer, tons of games are on Polygon as well. Would be nice to find an updated ecosystem map.

add new sample files

Wearables

• add metaversefiles
• add clo file
• add blender example
• add JSON from hyperfy grabbable

READMEs

• add readme in different folders like wearables

Aligning with Webaverse on this

When working on avatar interop problems I always circle back to Webaverse solving the hard parts

Here's a handful of recent and related issues being worked on the main app

• webaverse/app#2533
• webaverse/app#2532
• webaverse/app#2524
• webaverse/app#2519
• webaverse/app#2456
• webaverse/app#2547

I think studying and documenting Webaverse implementation for avatars, wearables, and animation interop is a good path

3/22/22 Weekly meeting 5 Agenda

• Weekly news
  • Otherwise (yugalabs)
  • ENSavatar by Mekezzo
  • M3taloot teasers
• Interoperable avatars / wearables
  • Tubby pets
  • Super yetis
  • Mfers
  • Metatravelers
• Community infrastructure updates
  • Wiki updates
  • Call for POAPs
  • Gearing up with M3taloot

add a contributor guide

how can ppl contribute to the wiki / repo

Creating flowcharts

Flow charts can help us visualize this space better and areas that can be improved
It'd be interesting to map out what an ideal creator flow looks like for max self expression, freedom / interop

Avatars
Flowcharts on creation of avatars and a rough look on how to bring them into worlds / apps

Wearables
How to attach custom wearables to your avatar

Prefab of avatars in repo

Make avatar switcher prefab (or multiple) with all the characters in the wiki for testing in VRchat

cross-over m3taloot hero shot

When Yuga labs dropped the Otherwise video the frame that contained all the other NFT cross-overs became the most talked about. The power of cameos / cross-overs in metaverse media is huge (see Ready Player One). This issue is to talk about the M3 equivalent by taking it one step further with interoperable wearables (m3taloot)

---

M3taloot

Have 3D avatars wearing the loot and posing in video / photo shoots to create a hero shot of something like a fun game, similar to South Park The Stick of Truth.

Here is an older concept art of m3taloot. We are thinking 9 characters for the new shot.

• Orion
• VRoid
• Bayc
• Meebit w/ vox version wearbles
• Cryptovoxels mannequin
• Metaani / Tubby cat / Cryptoad
• 0N1Force
• Upstreet
• Mfer
• Super yeti

I know that's 10, so we need to cut it down to max 9 and save the rest for future previews.

Then we manually rig avatars with m3taloot

Art team will pose / create the final art.

To-do: NPCs dev log

Need to add more details, link demos, and open source code examples here: https://hackmd.io/pjXXP0WUTzyABdSsSBipXg

Add initial sample files

We need a shared library of assets to experiment with avatar interop together, lets gather a handful of avatars, wearables, and animation samples to prototype with or resources on where to get them:

• Avatars
  • Orion
  • Meebit
  • VRoid
  • Cryptovoxels
  • Upstreet
  • 0N1Force
  • Tubby cat
  • Y-bot
• Wearables
  • MetaFactory swag
  • M3taloot (glb model)
  • Sketchfab stuff (non-NFT good foil to have)
  • Cryptovoxels wearables (vox models)
• Animations
  • Mixamo
  • HEAT
  • PoseAI
  • Cryptovoxels dances
  • Webaverse

Feedback welcome / feel free to suggest more ideas

Wearable patches / pins for M3

Wanted to start a discussion to document adding patches / pins onto our avatars

Docs: create a glb npc with multiple animations

Repost from Ash in M3 discord, we should post this into a docs page somewhere and link some other guides (like this one published by polygonal mind: https://www.polygonalmind.com/blog-posts/export-a-gltfglb-with-several-animations)

• roughly following https://www.donmccurdy.com/2017/11/06/creating-animated-gltf-characters-with-mixamo-and-blender/
• export from mixamo without animations and T-Pose
• new blender
• import FBX, select "Manual Orientation" and "Automatic Bone Orientation"
• rename animation to TPose
• select all (armature + meshes) and apply all transforms
• if mesh is offset in pose mode
  • select all bones and clear all trasnfroms (option GRS)
  • in animation editor insert keyframes for both frames, to make sure pose is updated
• do optimisations, eg:
  • resize textures and option+s to repack
  • fix transparency issues by removing alpha and using Opaque instead of Alpha Blend
• at this point exporting to GLB should look good in Hyperfy and not have any frustum issues
• adding animations
  • export as FBX
  • import to mixamo, textures wont show but thats fine
  • for each animation, download without skin
  • import FBX animation back into blender, ensuring "Manual Orienation" and "Automatic Bone Orient"
  • rename the animation to some clear like "Idle" or "Talk"
  • rename base armature "Character"
  • now delete the animation armatures, this will still keep the animations around
  • go to animation tab, switch Dope Sheet to Action Editor, select Character and switch the animations to play them
  • each animation needs to be "Push Down" so that it exports with the GLB
  • export GLB with animations, test here: https://gltf-viewer.donmccurdy.com/
  • BOOM!

Document animation interop examples

Upload demos sharing mixamo animations between vrchat / webaverse
Create a documentation page on UGC animations in webaverse docs