nicovanduijn/snap_cam

This package provides tools to work with the Snapdragon Flight cameras as well as perform optical flow for use with the PX4 flight stack.

Snapdragon: Camera driver

This package provides tools to work with the Snapdragon Flight cameras as well as perform optical flow for use with the PX4 flight stack. The package can be used with ROS by building with catkin or, alternatively, with pure cmake, where only the executables that do not depend on ROS will be built.

The package is to be compiled on the Snapdragon board. Two variants are provided: Building with ROS, where all features are available, and building with pure CMake, where only ROS-independent applications are compiled (including the optical flow node).

Setup

For the pure CMake install variant, clone the required repositories in a directory, e.g. ~/src:

cd ~/src
git clone https://github.com/PX4/snap_cam.git

Initialize the Mavlink submodule:

cd snap_cam
git submodule update --init --recursive

Install the dependencies

sudo apt-get install libeigen3-dev sip-dev libyaml-cpp-dev libboost-dev cmake

To install OpenCV, download and push the .deb package to the Snapdragon and install it using

adb push /path/to/file /home/linaro/ (run on host)
dpkg -i opencv3_20160222-1_armhf.deb (run on Snapdragon Flight)

or when using with ROS ros-indigo-opencv3 can be installed.

Building with pure CMake

mkdir -p build
cd build
cmake ..
make

Run the optical flow application with (note that you need to be root for this):

./optical_flow [arguments ...]

Building with ROS

Prerequisites

To run the ROS nodes on the Snapdragon Flight, ROS indigo (base) has to be installed. Follow this link to install it on your Snapdragon Flight. (preferably using the linaro user: $ su linaro)

If you're having permission issues while installing ros try

sudo chown -R linaro:linaro /home/linaro

Install the following dependencies:

ROS dependencies

sudo apt-get install ros-indigo-mavlink ros-indigo-orocos-toolchain ros-indigo-angles ros-indigo-tf2 ros-indigo-tf2-ros

create a catkin workspace

Next, create a catkin workspace (e.g. in /home/linaro)

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/src
catkin_init_workspace
cd ..
catkin_make

Then clone the following three catkin packages and build

cd src
git clone https://github.com/ros-perception/vision_opencv
git clone https://github.com/ros-perception/image_common
git clone https://github.com/PX4/snap_cam.git
cd ..
catkin_make

Image publisher node

Once your catkin workspace is built and sourced you can start the image publisher using

roslaunch snap_cam <CAM>.launch

where <CAM> is either optflow or highres to stream the optical flow or high resolution cameras, respectively. You can set the parameters (camera, resolution and fps) in the launch files (pathToYourCatkinWs/src/snap_cam/launch/<cam>.launch)

You can now subscribe to the images in your own ROS node.

Camera calibration

For optical flow computations, a calibration file needs to be used. This package contains default calibration files for VGA and QVGA resolution. Nevertheless, we recommend calibrating your camera (see below) for better performance. For this you must build this package with catkin as described above and launch the optical flow image publisher:

roslaunch snap_cam optflow.launch

Clone and build this package in a catkin workspace on your computer. Add any missing dependencies:

sudo apt-get install python-pyside

On your computer launch the calibration app:

export ROS_MASTER_URI=http://<snapdragon IP>:11311
roslaunch snap_cam cameraCalibrator.launch

NOTE: If your image topics are empty, make sure to set the environment variable ROS_IP to the respective IP on both devices.

Set the appropriate checkerboard parameters in the app. Start recording by clicking on the button and record your checkerboard from sufficiently varying angles. Once done, click stop recording. The camera calibration will be written to pathToYourCatkinWs/src/snap_cam/calib/cameraParameters.yaml. Push this file to your snapdragon.

adb push /pathToYourCatkinWs/src/snap_cam/calib/cameraParameters.yaml pathToSnapCam/calib/cameraParameters.yaml

Running the optical flow

Check the PX4 user guide for more info.

With pure CMake

Run the following in your build directory:

./optical_flow [arguments ...]

All arguments are optional.

• -r specifies the camera resolution. The default is VGA. Valid resolutions are VGA and QVGA.
• -f specifies the camera frame-rate. The default is 30. Valid values are 30, 60, 90.
• -n specifies the number of features with which to compute the optical flow. The default is 10.
• -c specifies the calibration file. The default is ../calib/<resolution>/cameraParameters.yaml.
• -e specifies the exposure. The default is 100. Valid values are between 0 and 511.
• -g specifies the gain. The default is 50. Valid values are between 0 and 255.
• -a enables auto exposure (with gain control on top if exceeding max exposure).

With ROS

After sourcing your workspace with source ~/catkin_ws/devel/setup.bash, run:

rosrun snap_cam optical_flow [arguments ...]

The arguments are the same as for the pure CMake build/=.