cgtinker/blendartrack

blender add-on for importing augmented reality motion tracking data recorded with the mobile application blendartrack

blendartrack

The Blender add-on "blendartrack" may be used for importing .JSON and .zip data from the mobile app blendartrack. The main purpose of this add-on is to make augmented reality tracking available to the blender community.

Discontinued

I cannot activly maintain blendartrack anymore, I'll accept PR's and consider to transfer the ownership if someone plans to activly maintain it.

Capabilites

Importing face mesh

• importing animated face mesh geometry
• importing animated empties
  • use animated empties to drive a rigify face rig
  • update driver influences using sliders

Importing camera motion

• import camera motion tracking data
  • motion of the device camera
  • camera frustum for projection
  • custom adjustment data for video matching based on world space reference points
  • dot cloud data
  • reference points
  • movie clip

Features

• .zip / .json file import created with the mobile app BlendArTrack
• sets up a simple compositing tree for rendering

License

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

Copyright (C) cgtinker, cgtinker.com, [email protected]

If you want to support the development you can either donate at Gumroad or become a Patreon. Would be lovely, thanks!

blendartrack's Issues

Camera from face tracking

I'd like it if there was a feature to obtain a camera track (background composite and all) to achieve a camera track from the face track, so that selfie VFX videos can be based on the background and not the face track, it would allow for some cool videos in which the user is composited into a VFX scene as if they were taking a selfie

Generate Drive Rig - Error

Hi,
I tried to follow your tutorial, but something failed ... can you check it?

Blender run from Terminal (not mentioned, just avoid possible issue).
Data from mobile successfully downloaded and imported as Empties, Deleted Parent Empty, Generated Face Rig ... now when I want to Generate Driver Rig I got an Error. Rig is generated but is not driven by Face Empties.

Error: Python: Traceback (most recent call last):
  File "/Users/filip/Library/Application Support/Blender/3.1/scripts/addons/blendartrack-main/src/interface/Operators.py", line 56, in execute
    input_manager.generate_driver_rig()
  File "/Users/filip/Library/Application Support/Blender/3.1/scripts/addons/blendartrack-main/src/management/input_manager.py", line 68, in generate_driver_rig
    rig = armature.get_armature("rig")
  File "/Users/filip/Library/Application Support/Blender/3.1/scripts/addons/blendartrack-main/src/utils/blend/armature.py", line 10, in get_armature
    armature = bpy.data.objects[name]
KeyError: 'bpy_prop_collection[key]: key "rig" not found'

test_blendartrack_face_rig.blend.zip

Thank you for info and your hard work ... it looks awesome :)

Support for 3.1

Hi! First of all, thank you for all this work. Seams like a fantastic and "forward-looking" tool.
Seams the scrip it's still not ready for 3.1. Isn't that right?

l

Nice work

Just passed by to say that you are doing a great work.
All the effort necessary to create tha app for smartphone and the blender importer, that a great addition to the community 👏

Congratulations and thanks for the effort.

Error with cam json on Import Data

Getting an error opening the cam json file from blendartrack app. Using Windows 10, Blender 3.6. Don't seem to get errors with the json files intrinsic, screen or anchor when clicking Import Data for them.

Added image of the error:

My Android 10 device isn't supported

Dear team,

Thanks for building an awesome concept. I'd like to get started to test it but I cannot download from the Google Play store as it says my device is not supported.

It's a good phone, Huawei Nova 5T.

Do you have any plans to release for this model?

Best regards,

Yestin

Exception on generating a face rig in various versions of blender

I have tried b2.83.19, b2.93 and b3.4.

I am getting "None type" has no location parameter in all three versions of Blender, whenever I try to create a rig from a file generated by https://play.google.com/store/apps/details?id=com.cgtinker.Retargeter.

2023-03-25_13-26-54_face.zip

File imports fine into all three. Rig gets created but does not appear associated with the mesh.

I note a closed ticket that talks about a similar problem, but claims b2.93 is fine. It may be fine, dependent on some quirk of the files imported?

I tried installing blender 3.0, to see if the code would work on 3.0, but 3.0 was/is crap, and keeps crashing on start. (I had long not bothered with it).

Remid me where the android app puts the zip file?

Mobile App Is not working

the mobile app is not working for newer Android versions. And the app on play store is available to download

Issue when importing?

I'm not sure if this problem is because I'm new, but when I import I have 3 problems (not BIG problems, they have easy fix manually)

• Reference objects are not adjusted to the floor (I mean 0 at Z axis)
• I chose 1920x1080 30fps in the app, but the addon sets a 2400x1080 camera

I always have to fix them, but it may be possible to do it automatically.

To reproduce it, I just record with the app and this settings ON:

• general: 1st and 2nd
• camera tracknig: 1st
• point cloud density: default (5)
• resolution: 1920x1080 30fps
• max bitrate

Then, import with addon.

Change the licence

CC licences does not fit well for software.

Pick another one:
https://choosealicense.com/licenses/

I need some help here.

If I scale up the scene, reference points will be useless because they are not in their original position (obviously).

Any ideas about how to work with a bigger scene? I feel uncomfortable working with small pieces.

As you can see, its really small (the cube is a default cube).

blender 3.1 import data error

error info：

Python: Traceback (most recent call last):
File "C:\Users\xnuon\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\blendartrack-main\src\interface\cgt_operators.py", line 16, in execute
input_manager.file_to_load(bpy.path.abspath(user.data_path))
File "C:\Users\xnuon\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\blendartrack-main\src\management\input_manager.py", line 38, in file_to_load
manager.import_models(paths)
File "C:\Users\xnuon\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\blendartrack-main\src\management\execution_handler.py", line 12, in import_models
queue_manager = queue.QueueManager(paths)
NameError: name 'queue' is not defined

location: :-1

Support for blender 3.2

Hello, I am learning python and decided blender would be a great way to start. I planned to use your addon for a personal project. Thank you for making it btw. But I noticed that It doesn't show up on the sidebar on blender 3.2. I would like to help you do that so I'll start by looking at the docs for how addons are built. In the meantime, when you can, would you mind taking a look if it would be at least a lot of work?

I think it's an awesome add-on. If you can create a ticket and describe what needs to be done I can take a stab at it as a way to learn. I have some years of experience with java, kotlin. If you can point me towards good resources to learn this stuff that would be awesome.

Thanks for sharing your work!

support for Unity

Your repo is awesome. Do you have a plan to let user import the captured data into Unity?
So, we could also render the animation in Unity.

And your donation link is dead? How could I donate money to you for your work?