huan/emoji-net

EmojiNet is an image to emoji recognizer based on MobileNet / Google Emoji Scavenger Hunt

EmojiNet

EmojiNet is an image to emoji recognizer based on MobileNet / Google Emoji Scavenger Hunt.

Image source: Emoji Scavenger Hunt

Introduction

This model is based on TensorFlow.js / MobileNet and it does not require you to know about machine learning. It can take as input any local file and returns an array of most likely predictions of emoji the image contains.

Usage

There are two main ways to get this model in your JavaScript project: via script tags or by installing it from NPM and using a build tool like Parcel, WebPack, or Rollup.

via Script Tag

Not supported yet.

Pull Requests are welcome!

via NPM

import { EmojiNet } from 'emoji-net'

const emojinet = new EmojiNet()

// Load the model and warm it up
await emojinet.load()

// recognize the image.
const emojiItemList = await emojinet.recognize('tests/fixtures/sofa.jpg')
console.log('emojiItemList:', emojiItemList)
// Output:
// emojiItemList: [
//   { id: 416, name: 'sofa', probobility: 0.9893624782562256 },
//   { id: 61, name: 'bed', probobility: 0.007496606558561325 },
//   { id: 241, name: 'wallet', probobility: 0.000651892158202827 }
// ]

API

new EmojiNet()

Create a EmojiNet instance.

const emojinet = new EmojiNet()

emojinet.load()

Load the modle and warm it up.

await emojinet.load()

emojinet.recognize()

const emojiItemList = await emojinet.recognize('tests/fixtures/sofa.jpg')
console.log('emojiItemList:', emojiItemList)
// Output:
// emojiItemList: [
//   { id: 416, name: 'sofa', probobility: 0.9893624782562256 },
//   { id: 61, name: 'bed', probobility: 0.007496606558561325 },
//   { id: 241, name: 'wallet', probobility: 0.000651892158202827 }
// ]

emojinet.dispose()

Free the resources.

emojiNet.dispose()

Inspired By: Emoji Scavenger Hunt

• A look at how we built the Emoji Scavenger Hunt using TensorFlow.js, Jacques Bruwer, JK Kafalas, and Shuhei Iitsuka, Google Brand Studio, Oct 11, 2018

About MobileNet

MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases. They can be built upon for classification, detection, embeddings and segmentation similar to how other popular large scale models, such as Inception, are used.

MobileNets trade off between latency, size and accuracy while comparing favorably with popular models from the literature.

For more information about MobileNet, check out this readme in tensorflow/models.

History

main

v0.2 (Aug 20, 2021)

First working version.

v0.0.1 (Aug 19, 2021)

Initial version.

Most of the code are copy/pasted from Google Emoji Scavenger Hunt:

an experiment that leverages the power of neural networks and your phone’s camera to identify the real world versions of the emojis we use every day

Author

Huan LI (李卓桓), Google Developers Expert in Machine Learning (ML GDE), [email protected]

Copyright & License

• Docs released under Creative Commons
• Code released under the Apache-2.0 License
• Code & Docs © 2021 Huan LI <[email protected]>

emoji-net's Issues

Convert a @tensorflow/[email protected] frozen model to @tensorflow/[email protected] version

The Emoji Scavenger Hunt open-source project has a great pre-trained mode but it's a deprecated frozen model format with old @tensorflow/[email protected] version.

I want to upgrade to @tensorflow/[email protected] so we need to upgrade it.

See:

• tensorflow/tfjs#1541 (comment)

Steps

1. Get a Python 2.7 environment

$ docker run --rm -ti -v $(pwd):/model ubuntu:18.04 bash
$ apt update && apt install -y python python-pip

2. Install

$ pip install tensorflowjs==0.8.5
$ tensorflowjs_converter --version
Using TensorFlow backend.

tensorflowjs 0.8.5

Dependency versions:
  keras 2.2.2
  tensorflow 1.13.1

3. Convert

# tensorflowjs_converter \
>     --input_format=tf_frozen_model \
>     --output_node_names final_result \
>     ./tensorflowjs_model.pb \
>     ./web
Using TensorFlow backend.
Traceback (most recent call last):
  File "/usr/local/bin/tensorflowjs_converter", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/tensorflowjs/converters/converter.py", line 352, in main
    strip_debug_ops=FLAGS.strip_debug_ops)
  File "/usr/local/lib/python2.7/dist-packages/tensorflowjs/converters/tf_saved_model_conversion_pb.py", line 329, in convert_tf_frozen_model
    graph = load_graph(frozen_model_path, output_node_names)
  File "/usr/local/lib/python2.7/dist-packages/tensorflowjs/converters/tf_saved_model_conversion_pb.py", line 67, in load_graph
    tf.import_graph_def(graph_def, name='')
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/importer.py", line 430, in import_graph_def
    raise ValueError(str(e))
ValueError: Cannot reshape a tensor with 1001 elements to shape [0,0] (0 elements) for 'MobilenetV1/Predictions/Reshape' (op: 'Reshape') with input shapes: [1,1001], [2] and with input tensors computed as partial shapes: input[1] = [0,0].