Could not parse example input in TFX_Pipeline_for_Bert_Preprocessing

[tensorflow]

Could not parse example input in TFX_Pipeline_for_Bert_Preprocessing

[hanneshapke]

Hi @mironnn,

The tf.Example data structure is not intuitive. The data structure needs to be serialized and then encoded as base64 strings. I have created this Github gist with the example code (no error handling).

Let me know if you have any questions regarding the example. It works with the output of the latest BERT pipeline version and required a recent version of TF Serving.

[hanneshapke]

In the coming days, I will publish a pipeline which doesn't require the tf.Example data structure.

[rcrowe-google]

@hanneshapke Thanks for the offer! I think the best place to document this would be in https://github.com/tensorflow/tfx/blob/master/docs/guide/keras.md

I would also be interested to include your Colab, probably under https://github.com/tensorflow/tfx/tree/master/docs/tutorials/serving

[mironnn]

@hanneshapke maybe you can help me?
I would really appreciate it

[mironnn]

@hanneshapke Thank you so much for your time and for the provided example.

Yeah, it would be great! It is very interesting to see how to make requests without preparation (serialization in the client) directly to the TF Serving with raw text.

[hanneshapke]

Hi @mironnn,
Here is the example of the model export without the tf.Example requirement. Please note that if your model has multiple inputs, the TensorSpecs in the concrete_function() need to be adjusted. Those inputs also need to be consumed in serve_tf_examples_fn. I have added the REST request example without the tf.Example requirement to the end of the notebook.

Colab version of the BERT Pipeline which exports a model for simple REST requests:
https://colab.research.google.com/gist/hanneshapke/f0980b7422d367808dae409536fe9b46/tfx_pipeline_for_bert_preprocessing_wo_tf-example.ipynb

@rcrowe-google I think the TFX docs should mention the export of Keras models without the tf.Example dependency. I am happy to update the existing TFX documentation. Do you mind pointing me in the right direction where additional comments would be most appropriate?

@mironnn Let me know if you have any questions. I think we can close this issue.

[rcrowe-google]

@joeliedtke for reference

[hanneshapke]

@rcrowe-google Thank you for your reply. I'll make those PRs tomorrow.
It seems to be timely. Someone else raised the question in tensorflow/tfx#1885

I will ping you and @joeliedtke when the PRs are ready for a review.

[ucdmkt]

+1 to @hanneshapke

May we revise the demonstration of serving_input_fn() for model export to not receive 1-D Bytes Tensors and do parse_examples() in the serving graph, but simply receive flat list of raw Tensors? It's fine to receive serialized tf.Examples as input in training input_fn, but this characteristics doesn't have to carry over to serving_input_fn(), and doing so with Keras model is causing non-intuitive behavior like this issue and tensorflow/tfx#1885.

[kylegallatin]

The tf.Example data structure is not intuitive. The data structure needs to be serialized and then encoded as base64 strings. I have created this Github gist with the example code (no error handling).

@hanneshapke is there an example for gRPC? I have b64 encoded data but can't find the right format to make the prediction over the protocol.