This is an object detection aid for the visually impaired that consists of a white cane enhanced with a microcomputer and camera and an iOS application. As you use this white cane throughout your daily routine, you can use its object detection capabilities to receive helpful information about your environment! Specifically, the cane’s camera and microcomputer detect objects in front of you via machine learning, and then send that information via Bluetooth to your smartphone, where it can be relayed to you via audio feedback. See demo.
- Download the ODA app on your smartphone device.
- Make sure that your Bluetooth is turned on.
- Turn the Nano on by plugging the cord into the Nano’s battery, and accept its request to pair with your smartphone.
- Restart the Nano.
- Make sure your Bluetooth is turned on, open the app, and turn on the Nano by plugging the cord into the Nano’s battery. The smartphone and Nano will automatically pair.
- Once the Nano is fully set up (this should take about 1 minute), it will begin sending detected object information to the smartphone.
- To receive audio feedback on detected objects, swipe right anywhere on your screen. To turn off audio feedback, swipe left anywhere on your screen.
- To stop the Bluetooth process entirely, quit the application or turn off the Nano by unplugging the cord from the battery.
Remove the battery from the white cane and charge it with its micro-USB. When fully charged, reattach it to its velcro patch on the cane.
- NVIDIA Jetson Nano Developer Kit
- Raspberry Pi V2 Camera
- Portable Battery (This is the one we used)
- Bluetooth module for the Nano
-
Tensor Flow Object Detection API
The network we're using is ssd-inception-v2 which has 91 object classes
Once the Jetson Nano has been set up, open the terminal and clone this repo.
git clone https://github.com/rahul-mitra13/Object_Detection_Aid
Change directory to the gatt-server directory.
cd /Object_Detection_Aid/gatt-server
Run gatt-server.py.
python gatt-server.py
Clone this repo as above. Under the ObjectDetectionAid
directory, open ObjectDetectionAid.xcodeproj
with XCode. Turn smartphone's bluetooth on and build this project.
Once the jetson nano is set up, open .bashrc
.
vim .bashrc
Add the following line at the end of the .bashrc
.
python ~/Object_Detection_Aid/gatt-server/gatt-server.py
Add gnome-terminal
to Nano's startup applications.
Same as the case for manual compilation.