olexale/arkit_flutter_plugin

ARKit Flutter Plugin

Hi, my name is Oleksandr 👋

I'm a cross-platform mobile developer from Ukraine 🇺🇦

A few facts about me:

• 🐦 I'm a Google Developer Expert in Flutter and Dart
• 📱 I started developing mobile apps in times when iPhones had non-retina displays, Android was ugly, and Windows Phone was incredibly cool
• 🧪 I'm passionate about software quality and tests
• 🎤 I like to share the knowledge
• 🎸 I played bass guitar in a jazz band

arkit_flutter_plugin's Issues

Add support for GLTF

Hello,

it's possibile to add support for GLTF through the GLTFSceneForm?

Are face tracking and face geometry supported?

ARFaceAnchor, ARFaceGeometry and ARSCNFaceGeometry

as per https://www.raywenderlich.com/5491-ar-face-tracking-tutorial-for-ios-getting-started

Slow: Pan /Scale and Rotate

First of all thanks for this plugin, I could follow it very well, below is one issue which is bothering me.
In the manipulation sample, pinch and pan detection are quite slow... once you add the box try a few pinch... move your finger away from screen and try pinching again, I had to try atleast 5-6 times to grab the box. I tried in iPhoneXR as well as iPhone 6s, the behaviour is same in both. May be we need to add simultaneous recognition??

    func gestureRecognizer(_ gestureRecognizer: UIGestureRecognizer, shouldRecognizeSimultaneouslyWith otherGestureRecognizer: UIGestureRecognizer) -> Bool {
        return true
    }

Record camera question

Is there a way to record video? I'm currently using the [camera plugin](https://pub.dev/packages/camera) to record video but I want to be able to use arkit as well. Is this possible?

Load network image dynamically

Hi, this plugin is great!
I just wondering if the use of const keyword is compulsory for the tracking images array with a harcoded url, because every time I assign a variable on the 'name' property in ArKitReferenceImage it failed to track the reference image. Please take a look the screenshot below,

Thank you.
Best regards, Ifan.

Check if TrueDepth camera is supported

I know we can use device_info plugin, but there is this function you can incorporate to check easily if the camera is supported.
Using device_info I have to check manually "machine" string and compare with the supported ones and update the function each time a new model comes out.
Thanks

How to use ARPlaneDetection.horizontalAndVertical

I tried to change plane detection to ARPlaneDetection.horizontalAndVertical but nothing happen

Feature request - camera node

@olexale I am trying to build a "Floor Target" that moves around horizontal planes based on the cameras position. I see how I can get distances off of hit test using the camera world alignment, but is there a way I can get orientational information of the phone?

[Question] Change Material of SCNNode

Hi Oleksandr Leuschenko,

Thank you for the amazing plugin,
I am trying to change the material when the user tap the plane,
my approach is to get the previous geometry, using _updateSingleProperty,
and rewrite the materials using new material, is it the right approach?

Best Regards
Steve

Measure distance & angle between detected image and device camera

Is it possible to obtain the distance and angles for a detected image using the plugin?

Any hint is highly appreciated.
Thanks

How to move object with PanRecognizer?

Hi,

I try to implement _onPanHandler to move plane object with this code but not working.
Could you please give me some advice?

void _onPanHandler(List<ARKitNodePanResult> pan) { for (var panNode in pan) { final node = nodes.firstWhere((n) => n.name == panNode.nodeName); if (node != null) { final old = node.position.value; final newMoveX = panNode.translation.x; final newMoveY = panNode.translation.y; node.position.value = vector.Vector3(newMoveX, newMoveY, old.z); } } }

Version 0.5.2 is not on https://pub.dev/

I would like to use the referenceImageName that you fixed in the 0.5.2 version (https://github.com/olexale/arkit_flutter_plugin/blob/master/CHANGELOG.md)

Do you plan to publish the new version ?
Is possible to use the version from github (master) ?

Thank you.

[Question] How to add multi-Image Detection?

Hello, I want to know how to add multi-Image Detection?

because I plan to use it in my senior project to teach children about numeric.

Thank you very much!

Is animation supported?

I didn't see any example about animation....Is it supported or do you fave plan to support it?
Thanks

Face mask

Thanks for this awesome library, it works flwlessy.
The AR Face demo with wireframe is awesome.
I know it is not a bug, but I want to add texture on the face, using a canonical image like this one:
https://developers.google.com/ar/images/canonical-face-texture-photoshop.jpg
Is it possbile? Thank you!

Can I make a double sided sphere?

Hi, thanks for the awesome plugin.
I wonder if can make a double sided sphere, to simulate a 360 view of a photo.
Just like this example:
https://stackoverflow.com/questions/46423415/how-to-apply-360-video-as-a-texture-in-arkit

Or if you happen to know how I can make this somehow I would be infinitely grateful.

Thank you.

face detection demo not work

could you tell me Why does the face detection demo work failed on my iPhone?
Camera failed to use? Display white screen。

Support for Android

Is it possible to use this plugin for Android apps built using Flutter?

[Request] Possibility to add ARKit with coreML ?

Hello,

I wanted to integrate ARKit with coreML in my senior project
for example like this https://github.com/hanleyweng/CoreML-in-ARKit
I would like to know if you can add this feature.

Thank you very much!

How to check ARKit supported device

Hi,

I want to show alert box when user's devices are not supported this ARKit plugin.
How can check?

Thank you very much for an awesome plugin.

Remove anchor from scene

Hi,
Thanks for help with identifying which dynamically loaded image was recognized. It works great.
I have another question. Is it possible to remove anchor from scene?

Feature request - childs of a node

1. I would love to access the nodes of a ARKitController in an array

2. I want to access the childs of a node
   I have a .dae with several nodes and if I add it to my ARKitController and use the TapRecognizer I can access the names of the childs, but I would love to change their material.

targetPlatform

Support for reading QR Codes and barcodes

Is it possible to detect and read an QR-Code or an barcode with the plugin?

ARKitPosition isn't defined

Hi!

In your function which we can see in your usage

void onARKitViewCreated(ARKitController arkitController) {
    this.arkitController = arkitController;
    this.arkitController.addSphere(ARKitSphere(
      position: ARKitPosition(0, 0, -0.5),
      radius: 0.1,
    ));
  }

So, to resolve this issue, you juste have to replace
position: ARKitPosition(0, 0, -0.5),
to
position: ARKitVector3(0, 0, -0.5),

I know it's a minor issue but I hope it's can help anyone who is beginner.

Thanks for your work!

Take a screenshot of the SceneView with Widget projection

I followed the example code for flutter's widget projection into a an ARKitSceneView. I can visualize correctly in screen. Then I tried to take a screenshot of what I saw in the screen, several questions in Swift/Objective-C points to the snapshot() function of the SceneView but I couldn't find it inside this plugin.

I also tried to use Screenshot plugin and write my own code using RepaintBoundary class and toImage() function. Both cases the screenshot is taken but only the projected flutter widget is seen and the background is entirely white. The camera scene is missing from the image.

Is there any way to do that?

Screen is black

I followed your instructions and testing your code of arkit_flutter_plugin/example/lib. When app enters to hello_world.dart after run

void onARKitViewCreated(ARKitController arkitController) {
    this.arkitController = arkitController;

    this.arkitController.add(_createSphere());
    this.arkitController.add(_createPlane());
    this.arkitController.add(_createText());
    this.arkitController.add(_createBox());
    this.arkitController.add(_createCylinder());
    this.arkitController.add(_createCone());
    this.arkitController.add(_createPyramid());
    this.arkitController.add(_createTube());
    this.arkitController.add(_createTorus());
    this.arkitController.add(_createCapsule());
 }

Console displays:
_platformCallHandler call onError Error
And the screen becomes black (only appbar is visible).

flutter doctor:
`[✓] Flutter (Channel beta, v1.6.3, on Mac OS X 10.14.5 18F132, locale es-MX)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.0)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.2.1)
[✓] Android Studio (version 3.4)
[!] VS Code (version 1.35.1)
✗ Flutter extension not installed; install from
https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[✓] Connected device (1 available)

! Doctor found issues in 1 category.
`
I'll really appreciate your help on this.

Node not rotating on init

I'm trying to add a node to my scene at a hit test but at a fixed angle. I am I've tried to adjust the Euler angles and Rotation properties of the ARKitNode, but with no luck. Oddly enough, I can animate the Euler angles just fine, but can't initialize them at an angle.

void onARKitViewCreated(ARKitController arkitController) {
this.arkitController = arkitController;
this.arkitController.onAddNodeForAnchor = _handleAddAnchor;
this.arkitController.onUpdateNodeForAnchor = _handleUpdateAnchor;
this.arkitController.onARTap = (ar){
print("Tapping Screen");
final point = ar.firstWhere((o) => o.type == ARKitHitTestResultType.existingPlane);
if(point != null){
print("Tapped plane");
_onARTapHandler(point);
_addFloorTarget(point);
}
};
}

void _addFloorTarget(ARKitTestResult point) {
final position = VectorMath.Vector3(
point.worldTransform.getColumn(3).x,
point.worldTransform.getColumn(3).y,
point.worldTransform.getColumn(3).z,
);

final material = ARKitMaterial(
  diffuse: ARKitMaterialProperty(color: Colors.green),
  lightingModelName: ARKitLightingModel.constant,
);

final plane = ARKitTube(
  innerRadius: 0.0,
  outerRadius: 0.001,
  height: 0.1,
  materials: [material]
);

final floorTarget = ARKitNode(
  position: position,
  geometry: plane, 
  eulerAngles: VectorMath.Vector3.(0.0, pi, 0.0),
);

this.arkitController.add(floorTarget);

Plz make panorama load from url

App rejection by TrueDepth API

Can I choose to use the TrueDepth API?

I'm sorry to hear that you recently received a decline in the review process for your app using this plugin.

The app was rejected because of the code in the TrueDepth API included in arkit_plugin. Since the app uses only the AR View feature, the TrueDepth API is not required, but it appears to be rejected because it is included in the plugin.

Can't we update the TrueDepth API so that we can pick up the answer from the Apple Review Team?

---

Guideline 2.5.1-Performance-Software Requirements

During review, we found that your app includes TrueDepth APIs. However, we were not able to locate any features in your app that use TrueDepth APIs.

Next steps

If your app does not include any features that use TrueDepth APIs, please remove them from your app.

If your app uses the Unity ARKit plugin, it would be appropriate to update to the latest version (https://forum.unity.com/threads/submitting-arkit-apps-to-appstore-without-face-tracking.504572/ # post-3297235), which includes a setting that allows you to exclude TrueDepth APIs.

If your app does include features that use TrueDepth APIs, please reply to this message in Resolution Center to provide information on how to locate them.

Please see attached screenshots for details.

Cloud anchors support

Hello,

is it possible to add support for Google's cloud anchors: https://developers.google.com/ar/develop/ios/cloud-anchors/developer-guide

or some hints how to do it?

Thanks!

Add custom object

Hello,

Thank you very much for your library. I'm not used to develop for iOs, so having the ability to use ARKit with flutter is awesome.
I need to add custom objects in the scene. I maybe missed something, but I didn't find any sample with custom objects. Its seems that's it's possible with ARKit (importing dae scenes).

Is it possible with the current implementation or is it something to come?

Best regards,

JM

Loading image assets dynamically

Hello,
First, thanks for this awesome plugin !

I had a question, is it possible to load dynamically image assets in the app ?
To be more clear, I want to use the widget projection & Image detection features but I can't load the asset directly in the bundle.
Indeed, I want to detect images downloaded directly from the app, so images are not know at advance and can't be added to the project...

Is there already possible ?

Thanks in advance.

[Request] Possibility to add measure feature ?

Hi!

I'm new in Flutter and I wanted to integrate ARKit in my project. Thanks to you it's possible!

I'm very interested by this measure feature, and I would like to know if you can add this feature.

Thank you in advance!

Request: Implement UIRotationGestureRecognizer

It would be great to have UIRotationGestureRecognizer also implemented.

Object tracking and anchors

Hi, first of all, thanks for share your work!!!...

currently i have a tensorflow model for foot detection, and i need is to attach a 3d model to the foot detected, or how can i set an anchor to a detected foot, this is because anchor use a vector3 and the tf model give me top and left position of the detected foot.

is this possible to achieve with this library?

regards.

[Question] Image Targets and tracking

Hi there,
Can we do something like this with this plugin ?
https://library.vuforia.com/articles/Training/Image-Target-Guide

Identify detected image which was dynamically loaded

Hi.
I have a question. I'm loading images to detect from firebase. I need to show different things to different images. Is any way to identify which image was just detected? I checked referenceImageName from anchor but it was null.
Thanks in advance for your help

Where to put assets file?

I've successfully loaded the earth example, but the sphere is white (no texture I guess).
The earth.jpg image on the repo is located in the Runner directory, so I've put the same image using Finder in the ios/Runner directory.
The sphere is still white...Am I missing something?
Thanks

EDIT: Nevermind, I had to drag and drop the image on Assets.xcassets in Xcode

Question, display a flutter view over arkit view ?

That sounds like something very difficult to do but I have to ask. Would it be possible for example to track an image, and to display over this image a stretched flutter widget (strechted to fit the tracked image) ?

NoSuchMethodError on Matrix4.decompose

Hi Oleksandr

when using your camera_distance_page.dart example I added some code to obtain quaternions of the detected image anchor (see changes to your code below):

 void _handleAddAnchor(ARKitAnchor anchor) {
    if (anchor is ARKitImageAnchor) {
      final imagePosition = anchor.transform.getColumn(3);
      final vector.Matrix4 anchorClone = anchor.transform.clone();

      anchorId = anchor.identifier;
      vector.Vector3 translation, scale;
      vector.Quaternion rotation;
      anchorClone.decompose(translation, rotation, scale);
    _updateDistance();
    }
  }

When I try to decompose the transformation matrix the following error get's thrown in platform_channel.dart's _handleAsMethodCall method:

Debug Output:

_NoSuchMethodError: The getter '_v3storage' was called on null.
Receiver: null
Tried calling: v3storage

MethodCall(didAddNodeForAnchor, {node_name: B547B598-A1F5-4D1C-AC39-D5521560733E, transform: 0.015258 0.999745 -0.016619 0.00000

Is this a bug, or just bad code on my side?
Thanks for looking into this.

Cheers

How to detect two or more images (same image) on a screen?

Sorry, I don't know about native IOS.
Can you add function to return array ARKitAnchor?

Thank you very much.

Question obtain camera position

Hi!

Flutter newb here. I am trying to figure out how to get the phone or camera position, in order to determine the distance away a point is from it. I so far have set worldAlignment: ARWorldAlignment.camera in ARKitSceneView, and took the distance from the point and the vector (0,0,0). Is it possible to obtain this information with this library? If so, is this a correct approach?

Thank you!

Why is this only iOS!?

Can you add geo point of interest AR

add support for android TargetPlatform

i tried running the code with android device. it shows
TargetPlatform.android is not supported by this plugin

please resolve this soon, make it available in andriod devices too.

Custom Object: Gesturerecognition doesnt seem to work

When I try to pan or scale on a custom object, the gesture recognizer handlers are not triggered nothing happens. I followed the box sample and instead of box sample I added a custom object to the plane.

Motion capture

Hi, Do you have any example of human motion capture?

Video with transparent background

Hello,

is posible to render a video with transparent background in a fixed position?

thank you in advance.

Remote Object

Hi Oleksandr Leuschenko,

Once again, thank you very much for your amazing plugin!
I would like to ask about the remote object capability,
is it possible to use a downloaded .dae file?
or maybe a downloaded .dae file saved in the local directory?

Best Regards
Steve