hollance / coreml-custom-layers Goto Github PK
View Code? Open in Web Editor NEWSource code for the blog post "Custom Layers in Core ML"
Source code for the blog post "Custom Layers in Core ML"
Hello, @hollance .
I implement a custom pytorch
layer on both CPU and GPU following your amazing blog. The cpu version works good, but when i implemented this op on GPU it cannot activate "encode" function. Always run on CPU. I have checked the coremltools.convert()
options with compute_units=coremltools.ComputeUnit.CPU_AND_GPU
, but it still doesn't work. Similar problems also mentioned in https://stackoverflow.com/questions/51019600/why-i-enabled-metal-api-but-my-coreml-custom-layer-still-run-on-cpu and https://developer.apple.com/forums/thread/695640. Any idea on help this would be grateful.
Update: I test your GPU code example, but it seems also execute in CPU ComputeUnit. Output are show below:
2022-01-12 17:04:02.593909+0800 CustomLayers[1515:575421] Metal GPU Frame Capture Enabled
2022-01-12 17:04:02.594216+0800 CustomLayers[1515:575421] Metal API Validation Enabled
init(parameters:) ["beta": 0.01, "engineName": Swish]
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0]]
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0]]
init(parameters:) ["beta": 0.01, "engineName": Swish]
init(parameters:) ["engineName": Swish, "beta": 0.01]
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0]]
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0]]
init(parameters:) ["beta": 0.01, "engineName": Swish]
setWeightData(_:) []
outputShapes(forInputShapes:) [[1, 1, 6, 256, 256]]
outputShapes(forInputShapes:) [[1, 1, 6, 256, 256]]
outputShapes(forInputShapes:) [[1, 1, 6, 256, 256]]
evaluate(inputs:outputs:) 1 1
Double 10 vector
[0.3277947902679443,0.03130210936069489,0.6019284725189209,0.001807634020224214,0.03013107925653458,0.001237826887518167,0.0006207154947333038,0.002744386438280344,0.001118888612836599,0.001314118970185518]
Hi,
Thank you so much for sharing this useful code with us.
I am trying to convert a keras code to core ML.
thia is the source code Im working on.
https://github.com/sbillburg/CRNN-with-STN
this is an OCR written in keras and with CRNN architecture.
in this code, loss is an custm layer written by LAMBDA layer.
I tried your code on this but it keeps raising error.
I was wondering do you have a workaround code with the loss function or any simpler approach.
Thanks for taking the time
TypeError: Keras layer of type <class 'tensorflow.python.keras.engine.functional.Functional'> is not supported.
Hello again:)
I truly appreciate your help and am very sorry taking your time.
your input toward this error will be very helpful to me.
the error raises here while it wanted to concatenate:
rnn2_merged = concatenate([rnn_2, rnn_2b])
I was wondering do you have any solution for this or any workaround or idea.
I was searching the whole day yesterday but really I could not find any useful link regarding this error how to figure it out.
this is the error:
Traceback (most recent call last):
File "/home/sgnbx/Downloads/projects/CRNN-with-STN-master/CRNN_with_STN.py", line 201, in <module>
custom_conversion_functions={"Lambda": convert_lambda},
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 760, in convert
custom_conversion_functions=custom_conversion_functions)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 556, in convertToSpec
custom_objects=custom_objects)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras2_converter.py", line 359, in _convert
converter_func(builder, layer, input_names, output_names, keras_layer)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_layers2.py", line 636, in convert_merge
mode = _get_elementwise_name_from_keras_layer(keras_layer)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_layers2.py", line 92, in _get_elementwise_name_from_keras_layer
raise ValueError('Only channel and sequence concatenation are supported.')
ValueError: Only channel and sequence concatenation are supported.
28 : concatenate_1, <keras.layers.merge.Concatenate object at 0x7f237c23b908>
Again thank you very much for taking the time Its much appreciated.
Hi. Thank you for the useful project. I encounter an issue when lambda layer receives multiple inputs.
# This is a custom activation function.
def swish(x,s1):
return K.sigmoid(x)*x
def merge(x):
return x[0]+x[1]
# Create a silly model that has our custom activation function as a new layer.
def create_model():
a = 3
inp = Input(shape=(256, 256, 3))
x = Conv2D(6, (3, 3), padding="same")(inp)
#x = Activation(swish)(x) # doesn't work! :-(
x = Lambda(swish, arguments={'s1':a})(x)
x = GlobalAveragePooling2D()(x)
x0 = Dense(10, activation="softmax")(x)
x1 = Dense(10, activation="softmax")(x0)
x = Lambda(merge)([x0,x1])
return Model(inp, x)
There will be an error.
Traceback (most recent call last):
File "test.py", line 98, in <module>
custom_conversion_functions={ "Lambda": convert_lambda })
File "/home/shamangary/anaconda3/envs/new/lib/python3.5/site-packages/coremltools/converters/keras/_keras_converter.py", line 745, in convert
custom_conversion_functions=custom_conversion_functions)
File "/home/shamangary/anaconda3/envs/new/lib/python3.5/site-packages/coremltools/converters/keras/_keras_converter.py", line 543, in convertToSpec
custom_objects=custom_objects)
File "/home/shamangary/anaconda3/envs/new/lib/python3.5/site-packages/coremltools/converters/keras/_keras2_converter.py", line 191, in _convert
graph.build()
File "/home/shamangary/anaconda3/envs/new/lib/python3.5/site-packages/coremltools/converters/keras/_topology2.py", line 643, in build
successors = self.edge_map[layer]
KeyError: 'lambda_2'
I was trying much more complex merging function other than just addition.
Do you know how to resolve this? Thank you!
like this [1, 1, 3, 64, 64]
how to write the shader when
texture2d_array<half, access::read> inTexture [[texture(0)]],
uses texture2d or is there a texture5d ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.