nheidloff/unity-watson-vr-sample

Virtual Reality Sample using IBM Watson, Unity and Google Cardboard

Virtual Reality Sample using IBM Watson, Unity and Google Cardboard

This project contains a simple Virtual Reality application which allows iPhone users with a Google Cardboard to move a player via voice commands and to ask a 3D character for weather information.

Check out the video for a short demo.

The picture shows a screenshot of the two cameras showing the 3D character. Users can move the player via voice commands and can have conversations with the character, for example:

• User: Start to walk.
• User: Stop.
• User: How is the weather?
• Character: In which location?
• User: Munich.
• Character: The temperature in Munich is currently 24 degrees.
• User: How is the weather in Berlin?
• Character: The temperature in Berlin is currently 28 degrees.

Technically the following services and tools are used:

• Unity
• IBM Watson SDK for Unity
• IBM Watson Speech To Text
• IBM Watson Assistant
• IBM Watson Text To Speech
• Weather Company Data
• Google Cardboard

Setup

Clone this repo and open the directory 'SimpleVR' in Unity.

Get an IBM Cloud account. The lite account is free (no credit card required, no time restriction) and supports the Watson services used in this sample.

Create a Watson Assistant instance and import workspace.json as a new workspace. Copy and paste the user name, user password and workspace id in SoldierConvo.js.

Create a Watson Speech To Text instance. Copy and paste the user name and user password in SoldierConvo.js.

Create a Watson Text To Speech instance. Copy and paste the user name and user password in SoldierConvo.js.

Create a Weather Company Data instance. Copy and paste the user name and user password in SoldierConvo.js.

Under build settings choose 'iOS' as platform.

After this you can run the application from within Unity. See the example script above.

Optionally you can replace the cylinder with a better 3D character, for example the cyber soldier.

Deployment to iPhone

Under iOS build/player settings some more properties need to be defined. See the screenshots folder for more details.

For 'Company Name' and 'Bundle Identifier' use a name that follows the iOS bundle identifier naming convention, for example 'net.heidloff.sample'.

Add 'Cardboard' as virtual reality SDK under 'XR Settings'.

Select 'Mute Other Audio Sources', 'Prepare iOS for Recording', 'Force iOS Speakers when Recording' and 'Allow downloads over HTTP' under the 'Configuration' section.

Add a textural description for the microphone usage.

Click 'Build' to generate an iOS build and open the *.xcworkspace file (not the *.xcodeproj file).

In order to fix the signing issues, unfortunately you need to change the following settings in Xcode every time after you've generated a new build. See the screenshots folder for more details.

Select 'Automatically manage signing' and your team under the 'General' tab.

With the 'Unity-iPhone' target selected, change all values to 'iOS Developer' for 'Code Signing Identity' on the 'Build Settings' tab.

With the 'Unity-iPhone Tests' target selected, change all values to 'iOS Developer' for 'Code Signing Identity' on the 'Build Settings' tab.

With the 'Unity-iPhone' project selected, change all values to 'iOS Developer' for 'Code Signing Identity' on the 'Build Settings' tab.

Connect your iPhone and press 'Build'.

After you've accepted the usage of the microphone, you can run the sample app.