By Oleksiy Grechnyev, 31 Jan 2024
Computer Vision with OpenCV in C++ or Python is a good fun.
Camera (VideoCapture
) and display (imshow
) are easy to use.
Things get a bit trickier when deploying OpenCV algorithms
in C++ on Android. Here I give a complete working example.
Minimal knowledge of Android Development, C++, JNI, OpenCV is recommended to understand the code.
The code takes a video stream from the camera, processes it with OpenCV (C++), and displays the result on the screen. There are 5 example OpenCV algorithms you can switch on the fly:
- Photo negative of the central 1/9
- Canny Edges
- Thresholding and contour detection
- ORB keypoints visualized
- Lucas-Kanade sparse optical flow
A release (APK file) is available here on GitHub, fell free to try it on your phone.
Enjoy!
Warning: I am not a professional Android dev (I'm better with C++), and the code might be not of a production quality. But I tried my best, and it basically works.
I had to use Camera2 to access the camera. CameraX has "analyze", but it is not powerful enough. The camera resolution is set to 1280x960, which gives a reasonable FPS on a typical phone.
The code estimates both the fpsIn
of incoming frames (typically 30), and fpsOut
of
processed frames displayed on the screen, which is typically below 30, as the code loses frames
when algorithms are too slow.
The camera frames are converted from YuV Image to RGBA Bitmap. The Android Bitmap can be
directly passed into C++ and accessed via the jnigraphics
library. I convert it
to cv::Mat
.
Finally, the resulting cv::Mat
is converted back to a Bitmap and displayed on a
SurfaceView (scaled to the screen size, with a proper aspect ratio).
I use several Kotlin threads (with and without Handlers) to create an efficient pipeline.
Building this project is not trivial.
First, you'll need Android Studio with Android SDK, of course.
Second, you'll have to open the "SDK Manager" and install the latest "NDK (Side by Side)" and CMake, in order to get C++ for Android.
Third, go to the OpenCV website and download OpenCV for Android. I used version 4.9, file opencv-4.9.0-android-sdk.zip
.
Unzip it somewhere on your computer.
Finally, open the file CMakeLists.txt
and modify it (change OPENCV_DIR
) to hardcode the path to where you unzipped OpenCV on your computer.
Note: If you google for "OpenCV Android", you'll find lots of very confusing guides on the internet, most
of which simply do not work (seriously outdated or just broken). My method is not perfect but it
definitely works (as of Jan 2024). And I am linking OpenCV directly in CMakeLists.txt
, Android
Studio isn't aware of it, no need for confusing stuff like "Importing Modules"! If there is a better
way to use OpenCV for Android (using Gradle and with no hardcoded path), I am not aware of it.
Important: I am using OpenCV in C++, never the thing called "Java OpenCV wrapper", this is much more powerful, plus you can deploy your cross-platform OpenCV/C++ algorithms on different platforms. I am also using the official OpenCV release, no third-party versions (which are usually incomplete or outdated).
Important: You should use the static OpenCV C++ library on Android, thus the C++ STL must be also static (the default mode of the NDK build). The shared OpenCV on Android is used for the OpenCV Java wrapper and it is rather incomplete!