alexribard/barracuda-u-2-nettest

Test project for implementing U-2-Net segmentation network in Barracuda

barracuda-u-2-nettest's Issues

Found unreferenced, but undisposed Tensor data which might lead to GPU resource leak

Running with Unity 2020.2.3f1 with Barracuda 1.0.4 or 1.3.0 preview in Editor, I get memory leaks and a load of yellow warnings for:

Found unreferenced, but undisposed Tensor data which might lead to GPU resource leak: (GPU:#-169063296 (1, 320, 320, 3) buffer: UnityEngine.ComputeBuffer created at: ) UnityEngine.Debug:LogWarning (object)

and also
''GarbageCollector disposing of ComputeBuffer allocated in D:\UnityProjects\BarracudaUNet2Test\Library\PackageCache\[email protected]\Barracuda\Runtime\Core\Backends\BarracudaReferenceCompute.cs at line 47. Please use ComputeBuffer.Release() or .Dispose() to manually release the buffer.''

Since this is in the update look, I tried adding m_worker.dispose(); at various points in the update loop and tried it in lateupdate but it didn't help

Barracuda inference error

Hi, I'm trying to run some ML models in Unity and found your informative repository, but I got this error.

ArgumentException: Off-axis dimensions must match
Unity.Barracuda.TensorExtensions.Concat (Unity.Barracuda.TensorShape[] shapes, System.Int32 axis) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/TensorExtensions.cs:229)
Unity.Barracuda.ModelAnalyzer.ListTemporaryTensorShapes (Unity.Barracuda.Model model, System.Collections.Generic.IDictionary`2[TKey,TValue] inputShapes, System.Collections.Generic.IDictionary`2[System.String,System.Nullable`1[Unity.Barracuda.TensorShape]]& shapesByName) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/ModelAnalyzer.cs:373)
Unity.Barracuda.ModelAnalyzer.ListTemporaryTensorShapes (Unity.Barracuda.Model model, System.Collections.Generic.IDictionary`2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/ModelAnalyzer.cs:66)
Unity.Barracuda.ModelAnalyzer.FindLargestNecessaryTensorShape (Unity.Barracuda.Model model, System.Collections.Generic.IDictionary`2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/ModelAnalyzer.cs:532)
Unity.Barracuda.GenericVarsWithPreallocation.PrepareStorage (Unity.Barracuda.Model model, Unity.Barracuda.IOps ops, System.Collections.Generic.IDictionary`2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:1125)
Unity.Barracuda.GenericWorker.PrepareForInput (System.Collections.Generic.IDictionary`2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:81)
Unity.Barracuda.GenericWorker.SetInput (System.String name, Unity.Barracuda.Tensor x) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:92)
Unity.Barracuda.GenericWorker.SetInput (Unity.Barracuda.Tensor x) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:99)
Unity.Barracuda.GenericWorker.Execute (Unity.Barracuda.Tensor input) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:111)
Inference.Update () (at Assets/Scripts/Inference.cs:56)

Could you tell me how can I fix this?

Thanks!

Heavily GPU bound on Android - I tried a few things

Using: Unity 2020.2.3f1, Barracuda 1.3.0 prev, Xperia XZ3 Adreno 630 with Vulkan

I made a fork here for testing on IOS - Iphone 10 with Metal and Android with the above spec
https://github.com/ROBYER1/Barracuda-U-2-NetTest

The application seems to be heavily GPU bound on Android but still just about runs and on IOS the output mask is always black - see issue here

I made sure to discardcontents of render textures and dispose/flush cache them as I noticed in profiling that loads of rendertextures were being left in memory as a leak.

Due to this, you need to set the size of the render textures in the assets folder manually now if you want to change them, until I find a tidier way to sort that without causing rendertexture leaks again.

However, on Android the model seems fairly heavy to run as you can see in the graph above, and the latency of the camera is very noticeable it is a good 2-5 seconds behind my movements

Just wondering if you can take a look into this if possible?

Fatal exception running on Android Vulkan/OpenGLES 3

Cloned this repo, left everything as default and built to Android with all settings left as is except for Graphics API changed from 'auto' to OpenGLES3 and Vulkan on separate test runs, the build has a fatal crash where I think this is the error (logs attached for each run)

CRASH signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 Error CRASH Cause: null pointer dereference

My assumption is that the app is getting stuck on a thread and the OS kills it a few moments later as the app opens, hangs with the scene rendering but no webcam texture then the app either crashes the phone or kills itself.
Tested with Unity 2020.2.3f1 on Xperia XZ3 with Adreno 630 GPU, Android 10, using Barracuda preview 1.3.0

Also attempted switching the GraphicsWorker from ComputePrecompiled to CsharpBurst to check if CPU would work however obviously even on my beasty desktop pc it is way below 1fps running on CPU

Crash logs for Vulkan and OpenGLES
Crash_U2Net_Vulkan_AndroidAdreno630.txt
Crash_U2Net_OpenGLES3_AndroidAdreno630.txt

Model doesn't run on IOS

Tested with Iphone 10 on IOS 14.4 with Unity 2020.2.3f1 and Barracuda 1.3.0 from my fork here https://github.com/ROBYER1/Barracuda-U-2-NetTest

When running, the camera image is sideways (I can fix that probably) and the model output mask is always black, there is some flickering black/white at the top and bottom of the app window where the notches are as you can see the slight different colour there in my screenshot above. The model works fine in editor on my macbook with the webcam.

Xcode doesn't flag any errors however in profiler attached, the model is obviously heavily GPU bound, if you are able to take a look at this please let me know?

It would be good to see this model running on IOS and Android if possible as I have seen other people using it in other frameworks on mobile from the U-2-net repo