ibm/image-analysis-iot-alert

WARNING: This repository is no longer maintained :warning: This repository will not be updated. The repository will be kept available in read-only mode.

WARNING: This repository is no longer maintained ⚠️

This repository will not be updated. The repository will be kept available in read-only mode.

Skill Level: Any Skill Level
N.B: All services used in this repo are Lite plans.

Upload and process an image into IBM Cloud to receive alerts

Build an IoT project with IBM Cloud Functions (serverless), Node-RED, Node.js and along with IoT Platform.

Overview and Goal

The goal of this tutorial is to take images from any other source, process it and trigger alerts to inform a change, a danger, etc. This project can be a quick setup or can be attached to an existing project to do the analysis of images and send alerts.

This tutorial will use an app to insert images into Cloudant, process it and display an alert. It is divided into multiple parts followed by a set of steps in each part to help you build an application based on visual recognition. The folders in this repo are separate entities of applications that will need a separate setup.

The workflow is not limited to the diagram shown above only but it can be expanded. We will be showcasing how to do it and you can take on this project, expand it, change it or make a real use of it.

Let's take an overall look on these parts:

• Run any application to upload an image. In this example, I provided the viz-send-image-app folder (STEP 1)
• Create a Cloudant database service (STEP 2)
• Create IBM Cloud Functions (viz-openwhisk folder) (STEP 3)
• Include Watson Visual Recognition code in IBM Cloud Functions (STEP 4)
• Create IoT Platform service where IBM Cloud Functions will process images and send them to the IoT Platform (STEP 5)
• Create Node-RED nodes to send the any alert (STEP 6)

Prerequisites

You will need the following accounts and tools:

• IBM Cloud account
• Bluemix CLI
• Openwhisk CLI
• nodejs To verify you have nodejs installed run node -v at a command prompt/terminal.
• Optional: Git If you choose to use Git to download the code samples you must also have a GitHub.com account. You can also download the code as a compressed file without a GitHub.com account.

Deploy to IBM Cloud

In this tutorial, you will require to setup each folder as a separate applications on IBM Cloud.

There will be manual setups from the terminal, running command lines. Because credentials are needed in files.

Steps - IMPORTANT - Read Slow

As the diagram above in the picture presents six steps. Create all the bullet points mentioned and save their credentials on a spreadsheet. Make sure you start from here with the followings:

• Create a Watson IoT Platform service instance from the IBM Cloud Catalog

• Create a gateway and a device manually in the IoT platform. They will be auto-registered when data will flow later on from IBM Functions the first time.

• Create a Node-RED package (it already includes Cloudant database)

• Create Visual Recognition service instance

• Create IBM Cloud Functions from the Catalog

Save all the credentials from above for a later use

We will dive more into the details of each part in the next steps assuming that you have your IBM Cloud account set for use. We will not complicate building these applications, so we will be relying on the manual creation of these setups from IBM Cloud browser and minimizing the command lines. More command lines are used when setting up Openwhisk.

IMPORTANT: Make sure before you start with the steps, update the files to match your credentials and rename app name with yours.

Step 1 - App UI

We have a basic UI at viz-send-image-app to help us upload images into Cloudant database.

• Make sure you already created a Watson IoT Platform service and saved the credentials

• Make sure you already created a gateway and a device in IoT Platform. The gateway info will be used in Step 3 and the device info for credentials.json in this step.

• Make sure you already created a Node-RED package that comes with Cloudant database and had saved the credentials of Cloudant

*** Put needed credentials in credentials.json and rename app name in manifest.yml ***

To deploy this setup from a terminal, use the following commands (make sure CLIs are downloaded for these commands):

cd viz-send-image-app
bx api api.ng.bluemix.net
bx login -u YOUR_IBM_CLOUD_USERNAME -o org_name -s space_name
bx app push <APP_NAME>

To troubleshoot errors, use bx app logs YOUR_APP_NAME --recent command (i.e. bx app logs viz-image --recent).

• In a browser, access your app by typing your app's URL: https://YOUR_APP_NAME.mybluemix.net (YOUR_APP_NAME = whatever you named your app). For example, my app's url is as the following: https://viz-image.mybluemix.net/.

To it run locally:

cd viz-send-image-app
npm install
npm start

• In a browser, access your app by typing: localhost:3000

Step 2 - Cloudant database

• Make sure you already created a Node-RED package that comes with Cloudant database and had saved the credentials of Cloudant. Create a database (a table) in Cloudant and name it, this will store the incoming images.

Step 3 - IBM Cloud Functions (previously OpenWhisk)

Make sure you already created IBM Cloud Functions from the Catalog before you start Step 3.

• To get the OW_AUTH_KEY, run the following command in your terminal: bx wsk property get --auth

• Create an API key and token in Watson IoT Platform for the gateway

• For CURRENT_NAMESPACE and PACKAGE_NAME, plug your namespace to them (same OPEN_WHISK_NAMESPACE)

IMPORTANT :

• Add your credentials to credentials.env.example and rename it to credentials.env

• Make sure you rename your app + service names to your specific app + service names in the following files:

  • credentials.env.example
  • mac-ubuntu-linux.sh
  • windows.bat

Make sure you are able to run scripts locally like *.sh or *.bat

For Mac users (mac-ubuntu-linux.sh is for Linux based OS):

$ cd viz-openwhisk-functions
$ ./mac-ubuntu-linux.sh --install

For windows users:

$ cd viz-openwhisk-functions
$ windows.bat --install

If you decided to remove what you've installed, replace --install with --uninstall and run the command.

Step 4 - Visual Recognition

No action is required from the developer. To explain it, actually, IBM Cloud Functions will grab the image inserted to Cloudant DB and analyze it by Watson Visual Recognition that we have pushed in step 3. Then it posts an event to the Watson IoT Platform for further processing.

• First upload will be registered as a device to the Watson IoT Platform.
• Go the Watson IoT Platform and check for the processed image data as an event stored in the Cloudant DB when the setup of the project is done and you run the application.

Step 5 - Watson IoT Platform

• Make sure that by now you have already created IBM Cloud Functions instance
• Make sure also that you created a gateway and a device

Step 6 - Node-RED

• Make sure you already created a Node-RED package from the Catalog

• Copy and paste the json flow from viz-node-red/flow.json into Import -> Clipboard at your Node-RED https://YOUR_APP_NAME.mybluemix.net/red (YOUR_APP_NAME = whatever you named your app). Assuming that you already created Node-RED package from IBM Cloud's Catalog. If not, go ahead and create one. Ususally with Node-RED package, you'll get a Cloudant db. You can either use that or bind the one you created in Step 2. To avoid confusion, make sure you use one Cloudant service.

• Make sure that ibmiot in Node-RED have the correct information of your IoT Platform and make sure you create an API_KEY and API_TOKEN from the platform itself, click on Members -> Generate Key button.

Useful links

• IBM Cloud
• IBM Cloud Documentation
• IBM Cloud Developers Community
• IBM Watson Internet of Things
• IBM Watson IoT Platform
• IBM Watson IoT Platform Developers Community
• Simulate IoT Device
• Node-RED

Learn more

• Artificial Intelligence Code Patterns: Enjoyed this Code Pattern? Check out our other AI Code Patterns.
• Data Analytics Code Patterns: Enjoyed this Code Pattern? Check out our other Data Analytics Code Patterns
• AI and Data Code Pattern Playlist: Bookmark our playlist with all of our Code Pattern videos
• With Watson: Want to take your Watson app to the next level? Looking to utilize Watson Brand assets? Join the With Watson program to leverage exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.
• Watson Studios: Master the art of data science with IBM's Watson Studios
• PowerAI: Get started or get scaling, faster, with a software distribution for machine learning running on the Enterprise Platform for AI: IBM Power Systems
• Spark on IBM Cloud: Need a Spark cluster? Create up to 30 Spark executors on IBM Cloud with our Spark service
• Kubernetes on IBM Cloud: Deliver your apps with the combined the power of Kubernetes and Docker on IBM Cloud

License

This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.

Apache Software License (ASL) FAQ

image-analysis-iot-alert's Issues

Use new MAINTAINERS and CONTRIBUTING format

See https://github.com/IBM/Predictive-Market-Stress-Testing/blob/master/MAINTAINERS.md and https://github.com/IBM/Predictive-Market-Stress-Testing/blob/master/CONTRIBUTING.md for examples you can copy and paste

remove unmaintained tracker code

The metrics tracker project is no longer maintained.

Remove the following:

• Badge at top of README
• Privacy Notice from Readme
• Disabling Deployment Tracking from Readme
• Use of metric tracker in app, i.e. require('metrics-tracker-client').track();
• The metrics-tracker library from requirement.txt or package.json
• The repository.yml file at the app level

Update the deploy to ibm cloud button to use this image: https://bluemix.net/deploy/button.png

See https://github.com/IBM/watson-conversation-slots-intro/pull/71/files as an example

invoke-visual-recognition Action failing

After uploading an image (verified in Cloudant) the invoke-visual-recognition action is failing in the visual-recognition-iot-sequence. Summary of message back from Visual Recognition API:

"response":{"result":{"error":{"images_processed":0,"description":"No images were specified.","error_id":"input_error","code":400,"error":"No images were specified."}}

Output from cloudant/read Action matches the contents of the image in Cloudant.

The visrec API changed. This is most likely broken.

Did not repro, but as reported I think this code won't work with new vis rec instances that use have introduced IAM.

Image uploading app hard codes binding port => CF deploys fail healthchecks

viz-send-image-app/server.js always listens to port 3000 instead of obtaining port from the environment. This prevents the app from working in a CF Diego environment like IBM Cloud Foundry.