Optimize for CPU inference

[marl]

Optimize for CPU inference

[jongwook]

Yeah that's why I haven't put the table on README yet!

[jongwook]

Does Intel CPU have int8-optimized operations? I thought they are specific to CUDA.

Tensorflow already uses all CPU cores by default, and I'm not sure if there's practically any performance to gain unless we compromise the prediction frequency (using time step > 10 ms) or using a smaller model (like the one I used in the web demo)

[jongwook]

It's not precisely optimize but you might be interested in some of these numbers reported here

[justinsalamon]

We should add this table to the README file in the section where we explain about the different models available, ideally also including performance drop with respect to the full model (i.e. use a track from MDB for which we have a reference annotation and evaluate each model not only in terms of time, but also accuracy).

e.g. something similar to table 1 here: https://www.tensorflow.org/performance/quantization