hanleyweng/coreml-in-arkit

Simple project to detect objects and display 3D labels above them in AR. This serves as a basic Template for an ARKit project to use CoreML.

coreml-in-arkit's Issues

adding textview to storyboard

Hi mate, how did you manage to add a textview to the storyboard without deleting the scene?
Really curious about that.

Licence

Hey Hanley,
the code is currently under no open-source license.
Would be great, if you could specify one.

Thanks
Michael

Use of unresolved identifier 'Inceptionv3' Error

dyld: Library not loaded - Error while running on iphone x

Xcode Version: 9.2 (9C40b)
Phone: iphone x

I am getting the following error message while running. Followed a few instructions on StackOverflow with no help. I have (1) my developer certificate set to always trust and (2) "Always embed swift standard libraries" set to Yes in Build Settings as suggested by some threads on StackOverflow. Also, tried running after cleaning build folder.

Error message:

dyld: Library not loaded: @rpath/libswiftARKit.dylib
  Referenced from: /var/containers/Bundle/Application/5CE8F604-58A4-48E3-8172-7C615161D3ED/CoreML in ARKit.app/CoreML in ARKit
  Reason: no suitable image found.  Did find:
	/private/var/containers/Bundle/Application/5CE8F604-58A4-48E3-8172-7C615161D3ED/CoreML in ARKit.app/Frameworks/libswiftARKit.dylib: code signing blocked mmap() of '/private/var/containers/Bundle/Application/5CE8F604-58A4-48E3-8172-7C615161D3ED/CoreML in ARKit.app/Frameworks/libswiftARKit.dylib'

Screenshot:

Incorrect distance on object placement

I was using an iPhone 12. Sometimes, when I tap the screen, the AR dot and text gets placed right at the camera instead of in the object.

Any idea why?

ClassificationRequest.imageCropAndScaleOption = VNImageCropAndScleOption.centerCrop

Can't compile code.
I get Type 'VNImageCropandScaleOption' has no member 'centerCrop' I added the inceptionV3Model model

**Question** Get depth without tap

Is it possible to get depth (z position) of the detected object without user tap ?

200mb+ CoreML Models causes ARKit hiccups

Hi @hanleyweng
I'm doing a same thing as your project and I'm dealing with a big performance problem when running with a bigger coreml model 200mb. Whenever the coreml perform its request the arkit camera previewer get a flick. It seems like the coreml performing is blocked the rendering for a few milliseconds. I tried to run the bigger model on your project as well and there was.the same issues ! do you have any idea ?

something wrong with inceptionv3.mlmodel

I already copy the inceptionv3.mlmodel into project.but when i click Inceptionv3.mlmodel,it shows " Interface generation only available with valid target",which dont like the gif of yours.

How is the Label move as i move the object?

I tried running the project in iPhone 7 plus and it is working fine. I noticed as i move the object slowly, even the label moves. But in source code position updation of the node is not happening. Then how come, the text node is moving as i move the object slowly?

Image Confidence?

Hi, how could I get the percentage of confidence or accuracy of the prediction?

Thanks

this code creates a leak

I realised that using the continuous vision request creates a leak when the arviewcontroller is dismissed, it stays in the heap.

CoreML + ARKit