erfansn/ar-touch

🔮 Obtain the power of touchless interaction with display screens

ar-touch's Issues

Improve overall UX

• Prevent lock and off the device screen
• Handle configuration changes related to the rotating screen
• Improve the touching experience

Detect user hand landmarks

• Create a separate library module with the name hand-detection
• Implement Mediapipe hand landmarks detector
• Create a view to show visual version of landmarks
• Add Hand Landmarker as Image Analysis Usecase to CameraX
• Show hand landmarks analysis result in front of view

Dispatch touch event on specified position to target system

• Register the device BLE as a Peripheral HID device with the specification of ArTouch
• Advertise the device as a peripheral to discoverable by other devices
• Obtain bonded Bluetooth devices for connecting
• Connect and send the touch event to the central device

Extract touch placement in screen frame

• Create JNI function to calculate touch position with perspective transform to range 0..1
• Combine markers and hand detector results to expose touch signal with closed index and thumb fingers together
• Normalize detected touch position
• Add TextView to show instantly position value

Prepare configuration screen to initial

• Request and handle necessary permissions on-demon in various scenarios
• Able to enable Bluetooth if not enabled before
• Show a selectable list of bonded Bluetooth devices
• Able to disable show debugging stuff

Write necessary tests

• For :app module:
  • Integration test for TouchFragment and its ViewModel
    • Local Robolectric with Espresso
  • UI Test for ConfigurationScreen
    • Local Robolectric with Compose Test
  • Unit Test for rememberPermissionsRequestLauncher
    • Instrumented with Ui Automator
• For :producer module
  • Unit Test for ArUcoMarkerDetector
    • Instrumented with AndroidJUnit
  • Unit Test for TouchPositionExtractor
    • Instrumented with AndroidJUnit
  • Unit Test for DefaultTouchEventProducer
    • Local with JUnit
  • Performance Test for ArUcoMarkerDetector
    • Instrumented with Microbenchmark
  • Performance Test for HandLandmarkerDetector
    • Instrumented with Microbenchmark
  • Performance Test for DefaultImageRotationHelper
    • Instrumented with Microbenchmark

Show device camera preview to user

• Request and handle camera runtime permission
• Implement CameraX to show the preview of it

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update kotlin related stuffs (androidx.compose.compiler:compiler, org.jetbrains.kotlin.jvm, org.jetbrains.kotlin.android)
• Update all dependencies (androidx.emoji2:emoji2-views-helper, org.robolectric:robolectric, androidx.benchmark:benchmark-junit4, androidx.fragment:fragment-testing, androidx.fragment:fragment-ktx, io.insert-koin:koin-test-junit4, io.insert-koin:koin-test, io.insert-koin:koin-androidx-compose, io.insert-koin:koin-android, androidx.activity:activity-compose, com.google.accompanist:accompanist-themeadapter-material3, androidx.compose:compose-bom, com.google.mediapipe:tasks-vision, com.google.android.material:material, androidx.camera:camera-lifecycle, androidx.camera:camera-view, androidx.camera:camera-camera2, androidx.camera:camera-core, androidx.navigation:navigation-ui-ktx, androidx.navigation:navigation-fragment-ktx, androidx.lifecycle:lifecycle-viewmodel-compose, androidx.lifecycle:lifecycle-runtime-ktx, androidx.core:core-ktx)
• Update gradle related stuffs (gradle, androidx.benchmark, com.android.library, com.android.application)
• Update dependency androidx.compose:compose-bom to v2024
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

gradle

gradle.properties

settings.gradle

build.gradle

app/build.gradle

benchmark/build.gradle

common/build.gradle

dispatcher/build.gradle

gradle/libs.versions.toml

• androidx.appcompat:appcompat 1.6.1
• androidx.core:core-ktx 1.10.1
• androidx.lifecycle:lifecycle-runtime-ktx 2.6.1
• com.google.android.material:material 1.9.0
• androidx.concurrent:concurrent-futures-ktx 1.1.0
• androidx.navigation:navigation-fragment-ktx 2.6.0
• androidx.navigation:navigation-ui-ktx 2.6.0
• androidx.camera:camera-core 1.3.0-beta01
• androidx.camera:camera-camera2 1.3.0-beta01
• androidx.camera:camera-view 1.3.0-beta01
• androidx.camera:camera-lifecycle 1.3.0-beta01
• junit:junit 4.13.2
• androidx.test.espresso:espresso-core 3.5.1
• androidx.test.ext:junit-ktx 1.1.5
• com.google.mediapipe:tasks-vision 0.10.2
• org.jetbrains.kotlinx:kotlinx-coroutines-android 1.7.3
• androidx.compose:compose-bom 2023.06.01
• com.google.accompanist:accompanist-themeadapter-material3 0.30.1
• androidx.constraintlayout:constraintlayout-compose 1.0.1
• androidx.activity:activity-compose 1.7.2
• androidx.lifecycle:lifecycle-viewmodel-compose 2.6.1
• androidx.compose.compiler:compiler 1.5.3
• io.insert-koin:koin-android 3.4.2
• io.insert-koin:koin-androidx-compose 3.4.5
• io.insert-koin:koin-test 3.4.1
• io.insert-koin:koin-test-junit4 3.4.1
• org.jetbrains.kotlinx:kotlinx-coroutines-test 1.7.3
• androidx.test:runner 1.5.2
• androidx.test:core 1.5.0
• androidx.benchmark:benchmark-junit4 1.1.1
• org.robolectric:robolectric 4.10.3
• androidx.test:rules 1.5.0
• androidx.fragment:fragment-ktx 1.6.0
• androidx.fragment:fragment-testing 1.6.0
• androidx.test.uiautomator:uiautomator 2.2.0
• androidx.test:orchestrator 1.4.2
• com.android.application 8.1.1
• com.android.library 8.1.1
• org.jetbrains.kotlin.android 1.9.10
• org.jetbrains.kotlin.jvm 1.9.10
• androidx.benchmark 1.1.1

producer/build.gradle

• androidx.emoji2:emoji2-views-helper 1.3.0

gradle-wrapper

gradle/wrapper/gradle-wrapper.properties

• gradle 8.3

---

• Check this box to trigger a request for Renovate to run again on this repository

Detect corner position of screen

• Config NDK and CMake
• Add and link OpenCV and ArUcoNano source files
• Implement JNI function to detect markers from 0 until 4
• Build a view to demonstrating markers

Provide content in full screen

• Disable decor fit system window
• Hide the system bars