Comments (15)
We are using Keras for development.
from barracuda-release.
Hi!
Conv1D is supported, only 3D convolutions are not.
As long as the model gets exported correctly to onnx it should be good :)
from barracuda-release.
Awesome, thanks! Perhaps this can be added to the "Approximate list of TensorFlow nodes supported by Barracuda script converter" section in the docs?
from barracuda-release.
Yes it will, we are currently working on improving the docs.
Thanks for the feedback!
from barracuda-release.
Some clarifications about my message.
If you are converting your keras model using the keras_to_barracuda.py
or tensorflow_to_barracuda.py
, it is indeed not supported.
However, exporting your model to a .onnx file (keras2onnx) should work.
https://github.com/onnx/keras-onnx
from barracuda-release.
Why not recommend everyone to use the keras2onnx instead of creating another seprate converter inside Barracuda? Or is there a specific need / advantage for that?
from barracuda-release.
Legacy reasons both internal and from the time we didn't support ONNX.
Going forward, we recommend converting models in .onnx (with tf2onnx or keras2onnx) and importing these in Unity.
But I recognize the confusion, docs will be updated.
from barracuda-release.
Ok, thanks for the clarification!
from barracuda-release.
Hi @AlexRibard, can you clarify availability/usage of Conv1d support? Using the release/1.0.0 branch, my ONNX Conv1D layers are being recognized as Conv2D (which then fail to import properly). I'm not seeing explicit Conv1D types in Model.cs or elsewhere.
from barracuda-release.
Hi @marns
Even if we don't have a specific code path for Conv1D, I assumed they would correctly import into Conv2D (with width = 1)
But I recently realized we have some import errors when parsing input Conv1D.
They will be addressed.
In the meantime, I would recommend to write out your Conv1D as a Conv2D (with width = 1)
Sorry for the confusion
from barracuda-release.
Thanks @AlexRibard - FYI to get this to work, I also had to swap the stride dimensions in the ONNX importer. It's listed as (width, height) but I think it should be (height, width) to match the other kernel dimensions. My model is exported from PyTorch.
from barracuda-release.
I pushed support for it on master. It should be available in the upcoming release.
@marns do you have specific code snippet to illustrate your point?
from barracuda-release.
@AlexRibard Here's the hack I used to swap stride dimensions, which works for my (4,1) and (2,1) kernels/strides.
I also tried swapping at runtime instead, which gave correct output dimensions but not correct results, so probably I missed something.
+++ b/Barracuda/Runtime/Core/ModelBuilder.cs
@@ -293,6 +293,9 @@ namespace Unity.Barracuda
/// </summary>
public Layer Conv2D(string name, object input, Int32[] stride, Int32[] pad, Tensor kernel, Tensor bias)
{
+ int temp = stride[0];
+ stride[0] = stride[1];
+ stride[1] = temp;
return Conv(name, Layer.Type.Conv2D, input, stride, pad, new int[0], kernel, bias);
}
@@ -368,6 +371,9 @@ namespace Unity.Barracuda
/// </summary>
public Layer MaxPool2D(string name, object input, Int32[] pool, Int32[] stride, Int32[] pad)
{
+ int temp = stride[0];
+ stride[0] = stride[1];
+ stride[1] = temp;
return Pool(Layer.Type.MaxPool2D, name, input, pool, stride, pad);
}
from barracuda-release.
Here is the code in trunk
ONNXModelImporter.cs
Add("Conv", (net, node) => {
int[] dilations = new[] { 1, 1 }; // @TODO trap on wrong values
int[] strides = node.Strides;
int[] pads = node.Pads;
node.IgnoredAttribute("kernel_shape", "Kernel shape is derived from K tensor weights instead");
var kernels = node.Input1Constant(onnxLayout: "KCHW", name: "W");
var kernelRank = node.Input1Rank;
if (kernelRank == 3) // Conv1D
{
dilations = node.DilatationsOptional(new[] { 1 }); // @TODO trap on wrong values
Debug.Assert(dilations.Length == 1);
dilations = new[] { dilations[0], 1 };
if (strides.Length == 1)
strides = new[] { strides[0], 1 };
if (pads.Length == 2)
pads = new[] { pads[0], 0, pads[1], 0 };
}
else if (kernelRank == 4) // Conv2D
{
dilations = node.DilatationsOptional(new[] { 1, 1 });
Debug.Assert(dilations.Length == 2);
}
else
{
Warn(net, node, $"Unsuported Conv kernel rank. Conv2D assumes rank 4, Conv1D assumes rank 3, but got {kernelRank}.");
}
Debug.Assert(dilations.Length == 2);
if (dilations[0] != 1 || dilations[1] != 1)
kernels = DilateKernel(kernels, dilations); // @TODO inefficient method. Support dilatation in kernel code properly
var biases = node.Input2ConstantOptional(Bias(kernels.shape), 0.0f, onnxLayout: "C", name: "B");
if (node.GroupOptional() > 1)
net.DepthwiseConv2D(node.Name, node.Input0, strides, pads, kernels, biases);
else
net.Conv2D(node.Name, node.Input0, strides, pads, kernels, biases);
Output(node, features: kernels.channels);
});
ONNXNodeWrapper.cs
public int Input1Rank { get { return m_ONNXModelTensors.variables[Input1].rank; } }
from barracuda-release.
Thanks for sharing, looking forward to the full Conv1d support.
Just to clarify, the issue I'm describing is with a Conv2d where e.g. Stride height=4, width=1 is treated as width=1, height=4 which produces wrong output sizes/results
from barracuda-release.
Related Issues (20)
- Inference ONNX int8 quantized models using Barracuda engine
- UpSampling Layers Do Not Use Correct Interpolation
- Challenge: Make Baracuda work with Lllama HOT 1
- Is Barracuda deprecated? HOT 3
- 0nnxlmpotException: Unknown type ScaterND
- Saved PNG image of Texture2D shows Blank Image. HOT 1
- [Feature Request] baracuda able to run GPT4ALL4bit and AlpacaLora4bit
- Out of memory after running multiple time the network, even disposing all the tensors
- Will Barracuda 4.0 will be come with Unity AI initiative. HOT 6
- OnnxImportException: Unexpected error while parsing layer /model.24/Split_output_0 of type Split. Is it Barracuda problem? HOT 3
- Barracuda with savedmodel file format HOT 2
- dynamic weights of Conv is not support
- ONNX model outputs precision issue
- PyTorch TransformerEncoder HOT 1
- Using GPU in android with Unity Barracuda HOT 2
- OnnxImportException: Unknown type LayerNormalization encountered while parsing layer /transformer/layers.0/norm1/LayerNormalization_output_0. HOT 2
- OnnxImportException: Unknown type Scan encountered while parsing layer UU000UU HOT 2
- Loading from Application.persistentDataPath
- Unknown type LayerNormalization (Depth Anything ONNX) HOT 1
- Problem with importing model HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from barracuda-release.