googlesamples/arcore-ml-sample

ARCore ML sample

An ARCore sample demonstrating how to use camera images as an input for machine learning algorithms, and how to use the results of the inference model to create anchors in the AR scene.

This sample uses ML Kit's Object Detection and (optionally) Google's Cloud Vision API to infer object labels from camera images.

Getting Started

To try this app, you'll need the following:

• An ARCore compatible device running Google Play Services for AR (ARCore) 1.24 or later
• Android Studio 4.1 or later

Configure ML Kit's classification model

By default, this sample uses ML Kit's built-in coarse classifier, which is only built for five categories and provides limited information about the detected objects.

For better classification results:

1. Read Label images with a custom model on Android on ML Kit's documentation website.
2. Modify MLKitObjectAnalyzer.kt in app/src/main/java/com/google/ar/core/examples/java/ml/classification/ to specify a custom model.

[Optional] Configure Google Cloud Vision API

This sample also supports results from the Google Cloud Vision API for even more information on detected objects.

To configure Google Cloud Vision APIs:

1. Follow steps for configuring a Google Cloud project, enabling billing, enabling the API, and enabling a service account on Set up the Vision API documentation.
2. Save the resulting service account key file to app/src/main/res/raw/credentials.json.

License

Copyright 2021 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

arcore-ml-sample's Issues

custome model doesn't work

I have import a custome model and edited MLKitObjectDetector for custom model . but after i run and scan it crash the app .

the error

2023-07-27 15:08:15.121 10224-10392/com.google.ar.core.examples.java.ml E/AndroidRuntime: FATAL EXCEPTION: DefaultDispatcher-worker-1
    Process: com.google.ar.core.examples.java.ml, PID: 10224
    com.google.mlkit.common.MlKitException: Failed to initialize detector. Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
        at com.google.mlkit.vision.vkp.PipelineManager.start(com.google.mlkit:vision-internal-vkp@@18.0.0:63)
        at com.google.mlkit.vision.objects.custom.internal.zzg.tryLoad(com.google.mlkit:object-detection-custom@@16.3.1:3)
        at com.google.mlkit.common.sdkinternal.model.CustomModelLoader.load(com.google.mlkit:common@@17.1.1:3)
        at com.google.mlkit.vision.objects.custom.internal.zzh.load(com.google.mlkit:object-detection-custom@@16.3.1:2)
        at com.google.mlkit.common.sdkinternal.ModelResource.zza(Unknown Source:18)
        at com.google.mlkit.common.sdkinternal.zzn.run(Unknown Source:10)
        at com.google.mlkit.common.sdkinternal.zzp.run(com.google.mlkit:common@@17.1.1:2)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zze(com.google.mlkit:common@@17.1.1:4)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzc(Unknown Source:8)
        at com.google.mlkit.common.sdkinternal.zzj.run(Unknown Source:2)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzd(Unknown Source:10)
        at com.google.mlkit.common.sdkinternal.zzk.run(Unknown Source:2)
        at java.lang.Thread.run(Thread.java:1012)

Blowsome sample project!!!

Hi @devbridie,

Sorry to put this in the Issues section but the Discussion tab is not enable.

Just a quick message to tell you that you are doing an awesome work and that this sample is another demonstration of your clean/pro coding.

I will use it mostly as it is for my personal work.

Thanks a lot

why can't i get hitpoint after detect object?

i have writen my code in java . it dected the object due to the custom model . I want show a 3d object on the detected object . at the movement i can show it . but it appear far away from the detectedobject. and wehen i check logcat i have noticed an error that "E0000 00:00:1691133633.112591 14707 hit_test.cc:426] INTERNAL: No point hit."

Adds PlaneRenderer example

Hi,
I'm using this project to start a my personal project and I would show planes.
The problem:

• there isn't a PlaneRenderer and the PlaneRenderer.java class that i found in others ARCore examples uses GLSurface Renderer, so it is incompatible with this project.
  Can You give me a way to implement it?

Thanks

Not able to configure the Credential.json

I created the credentials.json in put it in the raw folder but still, the Cloud vision toggle is disabled. please let me know the steps.

trying to use STREAM_MODE

Hello there,
I would like to know if you have a version of this app that uses stream_mode?
if not i would like to ask how may i do such thing if its possible, and why if its not.
kinde regards,
a7med

Unable to use custom object detection model

Hello! When I try to use my custom object detection model, the app crashes and I obtain this error. Would really appreciate it if you could help. Thank you so much!

Cannot use on custom model

Tried to use my custom model, But when i try to hit the Scan button the app crashes. Do you have a detailed instructions on how to use custom model ?

is there arcore ml project in java

Modify MLKitObjectAnalyzer.kt

src/main/java/com/google/ar/core/examples/java/ml/mlkit/ no such file

credentials.json also not found