natmlx/natdevice

High performance, cross-platform media device streaming for Unity Engine.

natdevice's Issues

Field of view is `NaN` on macOS

`CameraDevice.CapturePhoto` does not work on Windows

This used to work on NatDevice 1.1.

Crash on startup on OnePlus 9

Apparently doesn't affect other devices. Reported by IgorBoyko.

Crash in `CameraDevice::OnCaptureCompleted`

Reported by 3DBear:

backtrace:
  #00  pc 0x0000000000025c30  /data/app/~~2EHF_v7H1frufB-dr1jn8A==/xxx.xxx.xxx-MUMDMbXbPW_xumZACM-q9w==/lib/arm64/libNatDevice.so (void std::__ndk1::__tree_balance_after_insert<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*)+80)
  #01  pc 0x0000000000024ae8  /data/app/~~2EHF_v7H1frufB-dr1jn8A==/xxx.xxx.xxx-MUMDMbXbPW_xumZACM-q9w==/lib/arm64/libNatDevice.so (NatML::Devices::CameraDevice::OnCaptureCompleted(void*, ACameraCaptureSession*, ACaptureRequest*, ACameraMetadata const*)+576)
  #02  pc 0x00000000000281f4  /system/lib64/libcamera2ndk.so
  #03  pc 0x0000000000017884  /system/lib64/libstagefright_foundation.so (android::AHandler::deliverMessage(android::sp<android::AMessage> const&)+84)
  #04  pc 0x000000000001da24  /system/lib64/libstagefright_foundation.so (android::AMessage::deliver()+188)
  #05  pc 0x0000000000018bd4  /system/lib64/libstagefright_foundation.so (android::ALooper::loop()+572)
  #06  pc 0x0000000000015744  /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+460)
  #07  pc 0x00000000000a1e1c  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+140)
  #08  pc 0x0000000000014f14  /system/lib64/libutils.so
  #09  pc 0x00000000000ecc00  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
  #10  pc 0x000000000008c360  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

Crash in `CameraDeviceLegacy` constructor

Reported by 3DBear:

backtrace:
  #00  pc 0x000000000006b558  /system/lib64/libc.so (tgkill+8)
  #01  pc 0x00000000000689dc  /system/lib64/libc.so (pthread_kill+64)
  #02  pc 0x0000000000023f28  /system/lib64/libc.so (raise+24)
  #03  pc 0x000000000001c9ac  /system/lib64/libc.so (abort+52)
  #04  pc 0x0000000000432c24  /system/lib64/libart.so (art::Runtime::PreZygoteFork())
  #05  pc 0x00000000000e5f0c  /system/lib64/libart.so (art::LogMessage::LogLine(char const*, unsigned int, art::LogSeverity, char const*))
  #06  pc 0x00000000002f05f0  /system/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+2172)
  #07  pc 0x00000000002f09b4  /system/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+184)
  #08  pc 0x000000000032e3b8  /system/lib64/libart.so (art::JNI::CallObjectMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+768)
  #09  pc 0x0000000000020960  /data/app/xxx.xxx.xxx-1/lib/arm64/libNatDevice.so (_JNIEnv::CallObjectMethod(_jobject*, _jmethodID*, ...)+116)
  #10  pc 0x000000000002bff4  /data/app/xxx.xxx.xxx-1/lib/arm64/libNatDevice.so (int NatML::Android::JNIHelper::Call<AJCameraParameters*, AJCamera*>(AJCameraParameters**, AJCamera*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&) const+204)
  #11  pc 0x0000000000025e90  /data/app/xxx.xxx.xxx-1/lib/arm64/libNatDevice.so (NatML::Devices::CameraDeviceLegacy::CameraDeviceLegacy(unsigned int)+260)
  #12  pc 0x0000000000031f44  /data/app/xxx.xxx.xxx-1/lib/arm64/libNatDevice.so (NDGetCameraDevices+896)

Can I get sample codes of CameraDevice.CapturePhoto(Action<CameraImage>) ?

I want get Texture2D from CapturePhoto.
I just have got Texture2D from rawImage, But its image quality is low.

Proguard stripping crash

11-30 09:05:49.702 14990 15016 E CRASH   : Tombstone written to: /storage/emulated/0/Android/data/.../files/tombstone_00
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348] No pending exception expected: java.lang.ClassNotFoundException: ai.natml.natdevice.CameraCallback
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader, java.lang.Class) (Runtime.java:-2)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader) (Runtime.java:1115)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at void java.lang.Runtime.load0(java.lang.Class, java.lang.String) (Runtime.java:936)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at void java.lang.System.load(java.lang.String) (System.java:1631)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at boolean com.unity3d.player.UnityPlayer.nativeRender() ((null):-2)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at boolean com.unity3d.player.UnityPlayer.access$300(com.unity3d.player.UnityPlayer) ((null):-1)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at boolean com.unity3d.player.UnityPlayer$e$1.handleMessage(android.os.Message) ((null):-1)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at void android.os.Handler.dispatchMessage(android.os.Message) (Handler.java:103)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at void android.os.Looper.loop() (Looper.java:237)
11-30 09:05:49.703 14990 15016 F p.pianini.pian: thread.cc:2348]   at void com.unity3d.player.UnityPlayer$e.run() ((null):-1)

1.3.1 does not compile on Unity 2020.3.36f1

The package does not compile:

10: [2022-11-29T13:27:17Z - Unity] Library\PackageCache\[email protected]\Runtime\Outputs\TextureOutput.cs(112,29): error CS1061: 'Texture2D' does not contain a definition for 'Reinitialize' and no accessible extension method 'Reinitialize' accepting a first argument of type 'Texture2D' could be found (are you missing a using directive or an assembly reference?)
11: [2022-11-29T13:27:27Z - Unity] Library\PackageCache\[email protected]\Runtime\Outputs\TextureOutput.cs(112,29): error CS1061: 'Texture2D' does not contain a definition for 'Reinitialize' and no accessible extension method 'Reinitialize' accepting a first argument of type 'Texture2D' could be found (are you missing a using directive or an assembly reference?)

Reintialize needs to be replaced with Resize

Crash when access microphone (works on iOS and editor, crashes on Android)

experience a crash on Android (Samsung Galaxy Note 10) when using the NatDevice 1.3. It crashes when trying to get Microphone input.

Here's the ADB log.
I shortly checked it and from my understanding the following gets reported:

• It dosen't have permission to access microphone (which is not correct as you can see from the picture)
• It crash while executing code from your library (Line 2750 and below)
  log.txt
  

Crash in `CameraDevice::OnPreviewImage`

Reported by 3DBear:

backtrace:
  #00  pc 0x0000000000025a4c  /data/app/~~AFqvSifx7nT8AIWvGkjSGA==/xxx.xxx.xxx-_1gIoe6HpDlDEV2dMTztJw==/lib/arm64/libNatDevice.so (void std::__ndk1::__tree_remove<std::__ndk1::__tree_node_base<void*>*>(std::__ndk1::__tree_node_base<void*>*, std::__ndk1::__tree_node_base<void*>*)+576)
  #01  pc 0x0000000000024488  /data/app/~~AFqvSifx7nT8AIWvGkjSGA==/xxx.xxx.xxx-_1gIoe6HpDlDEV2dMTztJw==/lib/arm64/libNatDevice.so (NatML::Devices::CameraDevice::OnPreviewImage(void*, AImageReader*)+624)
  #02  pc 0x0000000000027a00  /system/lib64/libmediandk.so
  #03  pc 0x0000000000017884  /system/lib64/libstagefright_foundation.so (android::AHandler::deliverMessage(android::sp<android::AMessage> const&)+84)
  #04  pc 0x000000000001da24  /system/lib64/libstagefright_foundation.so (android::AMessage::deliver()+188)
  #05  pc 0x0000000000018bd4  /system/lib64/libstagefright_foundation.so (android::ALooper::loop()+572)
  #06  pc 0x0000000000015744  /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+460)
  #07  pc 0x00000000000a1e1c  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+140)
  #08  pc 0x0000000000014f14  /system/lib64/libutils.so
  #09  pc 0x00000000000ecc00  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
  #10  pc 0x000000000008c360  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

DllNotFoundException: NatDevice assembly

Hi,
I am facing with a strange error whenever I try to build the project inside unity:

The strangeness is that compiling for android device works well.
I tried to change the NatDevice version but nothing works. .

I am currently using those versions:

• NatDevice 1.3.2
• Unity editor 2022.1.0f1
• Linux Ubuntu

Probably, assembly dll is missing?

I think is a banal error but I have no idea how I can solve it.

Thank you,
AP

Crash in `CameraDeviceLegacy::OnPreviewImage`

Reported by 3DBear:

backtrace:
  #00  pc 0x000000000001ce7e  /system/lib/libc.so (abort+58)
  #01  pc 0x000000000034e807  /system/lib/libart.so (art::Runtime::Abort(char const*)+910)
  #02  pc 0x000000000000738f  /system/lib/libbase.so (android::base::LogMessage::~LogMessage()+494)
  #03  pc 0x0000000000232b53  /system/lib/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1214)
  #04  pc 0x000000000026ac29  /system/lib/libart.so (art::JNI::CallObjectMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+1020)
  #05  pc 0x0000000000019ca9  /data/app/xxx.xxx.xxx-3L5hP-FLiSu3AmB_OqNdlQ==/lib/arm/libNatDevice.so (_JNIEnv::CallObjectMethod(_jobject*, _jmethodID*, ...)+32)
  #06  pc 0x00000000000246df  /data/app/xxx.xxx.xxx-3L5hP-FLiSu3AmB_OqNdlQ==/lib/arm/libNatDevice.so (int NatML::Android::JNIHelper::Call<AJCameraSize*, AJCameraParameters*>(AJCameraSize**, AJCameraParameters*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&) const+122)
  #07  pc 0x0000000000020f3d  /data/app/xxx.xxx.xxx-3L5hP-FLiSu3AmB_OqNdlQ==/lib/arm/libNatDevice.so (NatML::Devices::CameraDeviceLegacy::OnPreviewFrame(void*, AJPixelBuffer*)+180)
  #08  pc 0x0000000000038173  /data/app/xxx.xxx.xxx-3L5hP-FLiSu3AmB_OqNdlQ==/oat/arm/base.odex

NatDevice session token is invalid

Everything works fine in Editor but I got this error on both Windows (x64) and Android builds right on the app startup.

InvalidOperationException: NatDevice session token is invalid. Check your NatML access key and plan.

• Unity 2021.3.12f
• Il2CPP .NetStandard 2.1
• 

AudioBuffer handler not firing on Galaxy Tab A running Android 7.1.1

Test script (below) is granted RECORD_AUDIO permission successfully and executes StartRunning() but AudioBuffer handler function is never fired. Log includes the follow snippet:

W ServiceManager: Permission failure: android.permission.CAPTURE_AUDIO_OUTPUT from uid=10189 pid=25916
E         : Request requires android.permission.CAPTURE_AUDIO_OUTPUT
E AudioPolicyIntefaceImpl: getInputForAttr() permission denied: capture not allowed
I AudioFlinger: AudioFlinger's thread 0xae483900 ready to run
D MultiRecordManager: closeInputStream() moduleHandle:10, modulename:primary, stream_in:0xb0ab74c0, source:0
I MultiRecordManager: closeInputStream() skipMultiRecord
D audio_hw_primary: adev_close_input_stream: enter:stream_handle(0xb0ab74c0)
D audio_hw_primary: in_standby: enter: stream (0xb0ab74c0) usecase(30: afe-proxy-record)
V audio_hw_primary: in_standby: exit:  status(0)
E audio_hw_primary: adev_close_input_stream, set jack_in to null
E AudioRecord: Could not get audio input for session 2521, record source 1, sample rate 44100, format 0x1, channel mask 0x10, flags 0
W AudioRecord: restoreRecord_l() failed status -22
E AudioRecord-JNI: Error -22 during AudioRecord native read
E NatML   : AudioDevice 6 failed to read samples with error: -2
D NatML   : AudioDevice 6 stopped recording

Full log: AudioBufferTestLog3.txt

Test script:

namespace NatML.Devices.Tests
{

    using System;
    using System.Collections;
    using UnityEngine;
    using Unity.Collections.LowLevel.Unsafe;

    sealed class AudioBufferTest : MonoBehaviour
    {

        enum SampleRate
        {
            _8000 = 8_000,
            _16000 = 16_000,
            _22050 = 22_050,
            _24000 = 24_000,
            _44100 = 44_100,
            _48000 = 48_000
        }

        enum ChannelCount
        {
            _1 = 1,
            _2 = 2
        }

        [SerializeField] SampleRate sampleRate = SampleRate._44100;
        [SerializeField] ChannelCount channelCount = ChannelCount._1;

        async void Start()
        {
            Application.SetStackTraceLogType(LogType.Log, StackTraceLogType.None);

            PermissionStatus status = MediaDeviceQuery.CheckPermissions<AudioDevice>();
            Debug.Log($"CheckPermissions: {status}");
            if (status != PermissionStatus.Authorized)
            {
                status = await MediaDeviceQuery.RequestPermissions<AudioDevice>();
                Debug.Log($"RequestPermissions: {status}");
            }

            // Get audio device
            var query = new MediaDeviceQuery(MediaDeviceCriteria.AudioDevice);
            var device = query.current as AudioDevice;
            // Start running
            device.sampleRate = (int)sampleRate;
            device.channelCount = (int)channelCount;
            device.StartRunning(OnAudioBuffer);
            Debug.Log($"StartRunning() mic:{device.name}");
            // Wait and stop
            //yield return new WaitForSeconds(5f);
            //Debug.Log("StopRunning()");
            //device.StopRunning();
        }

        unsafe void OnAudioBuffer(AudioBuffer buffer)
        {
            Debug.Log($"Sample buffer: {(IntPtr)buffer.sampleBuffer.GetUnsafePtr()}");
            Debug.Log($"Sample count: {buffer.sampleBuffer.Length}");
            Debug.Log($"Sample rate: {buffer.sampleRate}");
            Debug.Log($"Channel count: {buffer.channelCount}");
            Debug.Log($"Timestamp: {buffer.timestamp}");
        }
    }
}

[Unity] Native Crash - Unknown Function

We received a native crash. FYI we're only interested in the mic, the crash is caused by GetCameraDevices

Device
iPhone12,3

OS version
iOS 16.3

CPU
arm64e

GFX
Apple A13 GPU

Unity version
2021.3.16f1

Native StackTrace:

Thread 0 (crashed)
0   libsystem_kernel.dylib             0x00000001ec227674 <symbols missing for uuid: c76e6bed463530c68f19fb829bbe1ae1> 
1   libsystem_c.dylib                  0x00000001b5d88c8c <symbols missing for uuid: 3cd8c9e025fd3b29850774cdffe7f16c> 
2   libsystem_malloc.dylib             0x00000001bcc13e2c <symbols missing for uuid: 81ce80b167ce333c9a2da1af0a729172> 
3   libsystem_malloc.dylib             0x00000001bcc140d0 <symbols missing for uuid: 81ce80b167ce333c9a2da1af0a729172> 
4   libsystem_malloc.dylib             0x00000001bcc0e490 <symbols missing for uuid: 81ce80b167ce333c9a2da1af0a729172> 
5   libsystem_malloc.dylib             0x00000001bcc0c63c <symbols missing for uuid: 81ce80b167ce333c9a2da1af0a729172> 
6   libobjc.A.dylib                    0x00000001a789cee0 <symbols missing for uuid: c1140b6565c73a26bd127bc730320ea0> 
7   libobjc.A.dylib                    0x00000001a78b0390 <symbols missing for uuid: c1140b6565c73a26bd127bc730320ea0> 
8   libobjc.A.dylib                    0x00000001a78b013c <symbols missing for uuid: c1140b6565c73a26bd127bc730320ea0> 
9   libobjc.A.dylib                    0x00000001a7893cbc <symbols missing for uuid: c1140b6565c73a26bd127bc730320ea0> 
10  libobjc.A.dylib                    0x00000001a788e0c4 <symbols missing for uuid: c1140b6565c73a26bd127bc730320ea0> 
11  NatDevice                          0x0000000104761674 <symbols missing for uuid: c251ce5d3a2934669e12736b1a2f289b> 
12  UnityFramework                     0x000000010d51419c NatDevice_GetCameraDevices_mE5149060E23CBEEC1FAC84A3A8E5FD0F5C039AAA (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/NatML.Devices.cpp:13486)
13  UnityFramework                     0x000000010d5140b4 U3CDiscoverCameraDevicesU3Ed__21_MoveNext_m4534129E37544C1FEDA8E0D2323ACBA081D8630F (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/NatML.Devices.cpp:8548)
14  UnityFramework                     0x000000010d08a638 List_1_AddEnumerable_m76C85811BC9C905E7224C054AD480C3062FA7700_gshared (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/Generics50.cpp:0)
15  UnityFramework                     0x000000010d089aec List_1_InsertRange_m386ABA9B2C4950D31E787C87CDEF3D3E103F1B1A_gshared (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/Generics50.cpp:16954)
16  UnityFramework                     0x000000010d5135c8 MediaDeviceQuery_DiscoverDevices_m0520738B6315FF77EFB342F12AB3CF55102140C5 (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/NatML.Devices.cpp:8066)
17  UnityFramework                     0x000000010d5133b0 MediaDeviceQuery__ctor_mE2C86815B53C8771C6F5BD82319E56505C59FF82 (/opt/workspace/workspace//temp20230222-6547-1x6u3p0/Classes/Native/NatML.Devices.cpp:7877)

Crashes on `new MediaDeviceQuery` on Android

Chance of crashing following execution of new MediaDeviceQuery on Android. Tested using Google Play's Pre-Launch Report and Android Studio emulators.

Most common crash involves the message Abort message: 'Scudo ERROR: corrupted chunk header at address...'

Test Script:

namespace NatML.Devices.Tests
{
    using System;
    using System.Collections;
    using UnityEngine;
    using Unity.Collections.LowLevel.Unsafe;
    using System.Threading.Tasks;

    sealed class AudioBufferTest : MonoBehaviour
    {

        private async void Start()
        {
            Application.SetStackTraceLogType(LogType.Log, StackTraceLogType.None);

            await RequestPermissions();

            int reps = 1;

            while (reps < 100)
            {
                Debug.Log($"new MediaDeviceQuery(), attempt {reps}");
                var filter = MediaDeviceCriteria.AudioDevice;
                var query = new MediaDeviceQuery(filter);
                await Task.Delay(1000);
                reps++; // oops missed this line originally so the logs say "attempt 1" repeatedly
            }

        }
        private async Task RequestPermissions()
        {
            PermissionStatus status = MediaDeviceQuery.CheckPermissions<AudioDevice>();
            Debug.Log($"CheckPermissions: {status}");
            if (status != PermissionStatus.Authorized)
            {
                status = await MediaDeviceQuery.RequestPermissions<AudioDevice>();
                Debug.Log($"RequestPermissions: {status}");
            }

        }

    }
}

Expected behaviour: no crash.
Experienced behaviour: Crashes usually within about 10 repetitions with various messages on certain devices.
Full logs:

• Pixel 5 API 32 emulator MediaQueryCrash-Pixel5API32.txt
• Pixel 6 Pro API 33 emulator MediaQueryCrash-Pixel6ProAPI33.txt
• Galaxy Nexus API 28 emulator MediaQueryCrash-GalaxyNexusAPI28.txt

`CameraDevice.StartRunning` blocks for a long time

Long hang (30 seconds plus) when starting a device

It might be similar to this issue #3.

I am on latest version of NAT. Unity version 2021.3.10f1. The device in question is a Logi 270 HD

Example code:

_output = new TextureOutput();
CurrentDevice.StartRunning(_output);

The texture will render gray until the 30 seconds has passed before rendering the cam.

Update docs

There are lots of incomplete pages on the online docs.

iOS 16+ crash

We're facing the crash that starts to appear after upgrading NatDevice from old AssetStore version to 1.3.2 on iOS 16.1 and 16.2. App is passing NatML access token validation and camera access in the first scene, but crashes when right before loading the second. No crash on iOS 15.x.

That is from XCode log:

2023-01-11 21:06:46.976322+0300 i-Shooter[4880:325181] [as_client]     AVAudioSession_iOS.mm:2374  Failed to set category, error: '!pri'
i-Shooter(4880,0x225f63ac0) malloc: Heap corruption detected, free list is damaged at 0x282f81840
*** Incorrect guard value: 759412347317
i-Shooter(4880,0x225f63ac0) malloc: *** set a breakpoint in malloc_error_break to debug
i-Shooter(4880,0x225f63ac0) malloc: Heap corruption detected, free list is damaged at 0x282f81840
*** Incorrect guard value: 759412347317
(lldb) 

And here is exception breakpoint:

int main(int argc, char* argv[])
{
    @autoreleasepool
    {
        id ufw = UnityFrameworkLoad();
        [ufw runUIApplicationMainWithArgc: argc argv: argv];
        return 0;
    }
}

Compile time error after upgrading to NatDevice 1.2.3

After upgrading NatDevice to 1.2.3 from 1.2.2, I am seeing below compile time errors:

Library\PackageCache\[email protected]\Runtime\CameraImage.cs(128,24): error CS0173: Type of conditional expression cannot be determined because there is no implicit conversion between 'float' and ''
Library\PackageCache\[email protected]\Runtime\CameraImage.cs(139,24): error CS0173: Type of conditional expression cannot be determined because there is no implicit conversion between 'float' and ''

I am using Unity version 2021.1.28f1 with Android target platform.

Below are my player settings:

After building for iOS, it complains about not valid subscription

I have a valid subscription for NatML. When doing an iOS build, it works fine. When I start my game after the build within the editor, I get an exception about a not valid subscription (this resolves when I restart Unity, but I don't want to restart Unity after each build).

nvalidOperationException: NatDevice session token is invalid. Check your NatML access key and plan.
NatML.Devices.Internal.NatDevice.CheckStatus (NatML.Devices.Internal.NatDevice+Status status) (at Library/PackageCache/[email protected]/Runtime/Internal/NatDevice.cs:388)
NatML.Devices.MediaDeviceQuery+<GetAudioDevices>d__14.MoveNext () (at Library/PackageCache/[email protected]/Runtime/MediaDeviceQuery.cs:104)
System.Collections.Generic.List`1[T].AddEnumerable (System.Collections.Generic.IEnumerable`1[T] enumerable) (at <612a2c65aaf843d698f8d38b2ad7654a>:0)
System.Collections.Generic.List`1[T].InsertRange (System.Int32 index, System.Collections.Generic.IEnumerable`1[T] collection) (at <612a2c65aaf843d698f8d38b2ad7654a>:0)
System.Collections.Generic.List`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] collection) (at <612a2c65aaf843d698f8d38b2ad7654a>:0)
NatML.Devices.MediaDeviceQuery..ctor (System.Predicate`1[T] filter, System.Int32 capacity) (at Library/PackageCache/[email protected]/Runtime/MediaDeviceQuery.cs:60)
CameraInput.setText () (at Assets/Scripts/CameraInput.cs:68)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_0 (System.Object state) (at <612a2c65aaf843d698f8d38b2ad7654a>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:153)
UnityEngine.UnitySynchronizationContext.Exec () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:83)
UnityEngine.UnitySynchronizationContext.ExecuteTasks () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:107)

Setter of MediaDeviceQuery.previewResolution does not work correctly.

`
var query = new MediaDeviceQuery(MediaDeviceCriteria.FrontCamera);
cameraDevice = query.current as CameraDevice;
cameraDevice.previewResolution = (600, 270);

`
My code is as above.

But android logcat says:
2022-10-27 16:36:46.601 31143 31171 Debug NatML CameraDevice 1 preview resolution set to (640,360)

Why it says this difference?

CameraDevice Continuous ExposureMode isn't continous

On iOS setting the camera device exposure mode appears to be behave as AVCaptureExposureModeAutoExpose instead of as AVCaptureExposureModeContinuousAutoExposure. In that the exposure is set correct when you call this, but if the lighting changes afterwards the exposure does not adjust.

Crash on iOS while accessing camera and write to RenderTexture

iPhone 12 Pro, iOS 16.

As you suggested I use this code for accessing camera and save to RenderTexture in order to bind it to a material (I can setText method once):

 public async void setText()
    {
        if ((device == null || !device.running) && !instantiating)
        {
            instantiating = true;
            Debug.Log("Find Cam");
            // Find a hardware camera
            // Request camera permissions
            bool granted = await MediaDeviceQuery.RequestPermissions<CameraDevice>() == PermissionStatus.Authorized;
            if (granted)
            {
                var criterion = MediaDeviceCriteria.CameraDevice;
#if UNITY_EDITOR_OSX || UNITY_STANDALONE_OSX
                criterion = MediaDeviceCriteria.CameraDevice;
#endif
                MediaDeviceQuery query = new MediaDeviceQuery(criterion);
#if UNITY_EDITOR_OSX || UNITY_STANDALONE_OSX
                device = query.current as CameraDevice;
#else
                device = query.current as CameraDevice;
#endif
                StartCoroutine("setTexture");
            }
            else
            {
                ModalDialogues.instance.cameraInputFailed();
            }
            instantiating = false;
        }
    }

    public IEnumerator setTexture()
    {
        RenderTextureOutput output = null; 
        try
        {
            output = new RenderTextureOutput();
            device.StartRunning(output);

        }
        catch
        {
            ModalDialogues.instance.cameraInputFailed();
        }

        yield return new WaitUntil(() => output.texture);
        this.GetComponent<MeshRenderer>().material.mainTexture = output.texture;
    }

works on Editor (OS X) but crashes on iOS, given the stacktrace below:

Find Cam
Stacktrace is not supported on this platform.

2022-11-01 14:16:40.254555+0100 Vythm[5985:1482452] NatDevice: CameraDevice com.apple.avfoundation.avcapturedevice.built-in_video:0 started preview
2022-11-01 14:16:40.285714+0100 Vythm[5985:1482452] [SystemGestureGate] <0x10aa0a4a0> Gesture: System gesture gate timed out.
2022-11-01 14:16:40.325369+0100 Vythm[5985:1482743] Execution of the command buffer was aborted due to an error during execution. Caused GPU Address Fault Error (0000000b:kIOGPUCommandBufferCallbackErrorPageFault)
2022-11-01 14:16:40.358152+0100 Vythm[5985:1482744] Execution of the command buffer was aborted due to an error during execution. Caused GPU Address Fault Error (0000000b:kIOGPUCommandBufferCallbackErrorPageFault)
2022-11-01 14:16:40.361789+0100 Vythm[5985:1482743] Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (00000004:kIOGPUCommandBufferCallbackErrorSubmissionsIgnored)
2022-11-01 14:16:40.361850+0100 Vythm[5985:1482743] Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors)

Exposure and focus controls on Windows

Can not set / get exposure value on windows pc!

Consider ordering media devices

Suggestion from @tgrayston :

1. Any with Default = true (Android devices don't seem to ever have one as true though?)
2. Internal Devices
3. External Devices
4. Any others

Compile time error after upgrading to NatDevice 1.3.1

After upgrading NatDevice to 1.2.5 from 1.3.1, I am seeing below compile time errors:

Library\PackageCache\[email protected]\Runtime\Outputs\TextureOutput.cs(112,29): error CS1061: 'Texture2D' does not contain a definition for 'Reinitialize' and no accessible extension method 'Reinitialize' accepting a first argument of type 'Texture2D' could be found (are you missing a using directive or an assembly reference?)

I am using Unity version 2021.1.28f1 with Android target platform on a Windows PC.

CameraDevice failed to read preview image

2022-08-15 10:11:39.342 28908 29189 Error NatML CameraDevice 1 failed to read preview image
2022-08-15 10:11:39.373 28908 29189 Warn NdkImageReader Unable to acquire a lockedBuffer, very likely client tries to lock more than maxImages buffers

and Screen freezes after some minutes.

audio device return null on buffer

Hello i'm trying to send an audio stream recording mic to an Audio buffer but using StartRunning method on audio device return a null on buffer.

Here following the log of error we get:

ArgumentNullException: Value cannot be null.
Parameter name: array
Unity.Collections.NativeArray1[T]..ctor (System.Single[] array, Unity.Collections.Allocator allocator) (at <685c48cf8f0b48abb797275c046dda6a>:0)
NatML.Devices.AudioBuffer.Wrap (System.Single[] buffer) (at Library/PackageCache/[email protected]/Runtime/AudioBuffer.cs:124)
NatML.Devices.AudioBuffer..ctor (NatML.Devices.IMediaDevice1[TSampleBuffer] device, System.Single[] sampleBuffer, System.Int32 sampleRate, System.Int32 channelCount, System.Int64 timestamp) (at Library/PackageCache/[email protected]/Runtime/AudioBuffer.cs:58)
NatML.Devices.AudioBuffer.Clone () (at Library/PackageCache/[email protected]/Runtime/AudioBuffer.cs:104)
NatML.Devices.Outputs.AudioClipOutput.Update (NatML.Devices.AudioBuffer audioBuffer) (at Library/PackageCache/[email protected]/Runtime/Outputs/AudioClipOutput.cs:38)
NatML.Devices.AudioDevice+<>cDisplayClass28_0.b0 (System.IntPtr sampleBuffer) (at Library/PackageCache/[email protected]/Runtime/AudioDevice.cs:98)
NatML.Devices.AudioDevice.OnAudioBuffer (System.IntPtr context, System.IntPtr sampleBuffer) (at Library/PackageCache/[email protected]/Runtime/AudioDevice.cs:152)
UnityEngine.Debug:LogException(Exception)
NatML.Devices.AudioDevice:OnAudioBuffer(IntPtr, IntPtr) (at Library/PackageCache/[email protected]/Runtime/AudioDevice.cs:155)

Compile error Version 1.3.1 with Unity 2020.3.36

Texture2D.Reinitialize does not exist anymore. Should be replaced with Texture2D.Resize.
Cloud build cannot be used because code changes to the package cache are not possible that easily

10: [2022-11-29T13:27:17Z - Unity] Library\PackageCache\[email protected]\Runtime\Outputs\TextureOutput.cs(112,29): error CS1061: 'Texture2D' does not contain a definition for 'Reinitialize' and no accessible extension method 'Reinitialize' accepting a first argument of type 'Texture2D' could be found (are you missing a using directive or an assembly reference?)
11: [2022-11-29T13:27:27Z - Unity] Library\PackageCache\[email protected]\Runtime\Outputs\TextureOutput.cs(112,29): error CS1061: 'Texture2D' does not contain a definition for 'Reinitialize' and no accessible extension method 'Reinitialize' accepting a first argument of type 'Texture2D' could be found (are you missing a using directive or an assembly reference?)

I want to set all device preview resolution as 1 x 1.

How to set ratio of resolution of Preview.