kalwalt/webarkit-jsfeat-cpp

c++ jsfeat version

webarkit-jsfeat-cpp

c++ jsfeat version to build a WASM version thanks to Emscripten. The project is a Work in Progress, not all the features are released to make a comparison with the original jsfeat code. At the moment only some classes are implemented and a bunch of functions are under testing. Anyway the main idea is to provide c++ code to build a WASM version, it's not intended to be used as a c++ library. Maybe this will become the future of webarkit

Classes

Matrix_t

This class is responsible to hold different types of data, to be used by all classes.

Matrix_smart

Experimental class very similar to Matrix_t but use shared_ptr in the core

Imgproc

It is one of important classes to manipulate image data for image processing. For now we have these functions:

• gaussain_blur
• grayscale
• grayscale_m
• pyrdown
• equalize_histogram
• resample
• warp_affine

Orb

Oriented FAST and rotated BRIEF (ORB) is one most important class for feature tracking. Ths feature is under testing.

Keypoint_t

A class to store key points for image tracking algorithms.

Keypoints

A class to store a vector of keypoint_t for image tracking algorithms.

Yape06

Feature detector from Computer Vision Lab, Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland.

Pyramid_t

A class to handle multiple pyramidal images.

This project is quite experimental so do not expect great results. More classes will be implemented in the future.

Useful functions

At the moment you can try load_jpeg_data function to load a jpeg file and get important jpeg data (width, height, dpi and raw data). See grayscale_example or grayscale_m_example to see the function in action. Try yape06_detect function as a compact version for the yape06 detect routine. See it in action in the yape06_detect_video_example.html

Libs

Libs are stored in build folder:

• jsfeatcpp.js the Release lib.
• jsfeatES6cpp.js the ES6 Release lib.
• jsfeatcpp_debug.js the Debug lib.
• jsfeatES6cpp_debug.js the ES6 Debug lib.

Usage

You can import the libs as a ES6 module in a script tag:

<script type="module">
    import jsfeatCpp from "./../build/jsfeatcpp.js"
</script>

There isn't a npm package yet, so this is the only way.

or the non ES6 version :

 <script src="../build/jsfeatcpp.js"></script>

but the Module object will be available only when it is full loaded, so wrap your code into the listener:

window.addEventListener('jsfeatCpp-loaded', function (e) {
    //your code here...
    })

Examples

Take a look at our examples in the examples folder:

• equalize_histogram_video_example.html
• gaussian_blur_video_example.html
• grayscale_example.html
• grayscale_m_example.html
• grayscale_rgba_example.html
• grayscale_video_example.html
• keypoint_t_test.html
• keypoints_test.html
• matrix_t_test.html
• orb_test.html
• pyrdown_video_example.html
• resample_video_example.html
• sample_orb_mixed.html
• yape06_image_example.html
• yape06_video_example.html
• yape06_detect_video_example.html

Both examples use the debug version of the lib but, of course you can use the non-debug version as well.

NPM

npm is used to build the libar.bc and libar_debug.bc libs inside the build folder. You don't need to run npm install but it is required to have node and npm installed globally.

Building the library

If you make changes and/or if you make some enhancement to the C++ code ypu need to build the library. Run the shell script in the main folder: For the first time run: ./build.sh emscripten-all this will build all the libs. After that if you need to compile and build only jsfeat libs: ./build.sh emscripten and the two libs will be compiled in the build directory.

If you run under Windows use the ./build.Unix.sh script instead.

We used emsdk 3.1.20.

webarkit-jsfeat-cpp's Issues

matrix_t data property out wrong values

consider the matrix_t_example

webarkit-jsfeat-cpp/examples/matrix_t_test.html

Lines 13 to 16 in 6104fdf

	var data_u8 = new Uint8Array ([23, 20, 12, 24, 212, 220, 120, 46, 78, 92, 35, 12, 120, 120, 120, 120]);
	var m_u8 = new Module.matrix_t(4, 4, U8_t | C1_t, data_u8);
	console.log(m_u8);
	console.log(m_u8.data);

data_u8 as data argument to be loaded, the result should be identical in m_u8.data, but instead we receive a wrong Uint8Array with length 0 and ArrayBuffer of size 268435456 (or similar).
This not happens instead in the grayscale_example

webarkit-jsfeat-cpp/examples/grayscale_example.html

Lines 33 to 34 in 6104fdf

	var inputData = m_1.data;
	console.log(inputData);

values are in the right range.

Using valarray instead of vector for Array type.

At the moment i declared all the members inside data_t class as vector's because they are Array's

webarkit-jsfeat-cpp/src/types/types.h

Line 13 in c02edb5

template <typename T> using Array = std::vector<T>;

But maybe we could try to use valarray instead of vector STL container, it has some advantages, for example it includes math functions (abs sin, cos..) and slicing yhe array that could be very useful. I will make some test, and will report here.

static matrix_t::get() method output wrong data values

test this

webarkit-jsfeat-cpp/examples/matrix_t_test.html

Line 17 in 6104fdf

console.log(Module.matrix_t.get(m_u8));

data entry is a null Uint8Array, i'm wondering if this is expected or we can do something to fix this.

matrix_t inheritance

I think matrix_t should be inherited from data_t class, in this way we can access data_t memeberes without create a pointer instance inside the matrix_t constructor.

Unit testing for C++ and Wasm code

I would provide unit testing for at least part of the code. I think i will use https://github.com/google/googletest for C++, i haven't decided for WASM javascript.

export imgproc as a class in bindings.cpp

At this moment grayscale and grayscale_m are exported with cwrap but i think it's possible to export the whole impgroc class. At least i can test if it is possible.

Make the lib as a Module with Modularize and add a proper name to the export.

I think it is preferable to build the lib as an ES6 Module, we need just to add this rules for the build:

-s EXPORT_ES6=1 -s USE_ES6_IMPORT_META=0 -s MODULARIZE=1

and eventually also add an unique name:

-s EXPORT_NAME='jsfeatcpp'

with the Modularize option we can avoid using the onRuntimeInitialized in the examples.

Avoid using pointers

It will be preferable to NOT use pointers as much as possible. For example in case of arguments as function(Matrix_t* m) use std::unique_ptr or std::shared_ptr

warning: non-void function does not return a value in all control paths [-Wreturn-type] }

The shown errror occur in the matrix_t class

webarkit-jsfeat-cpp/src/matrix_t/matrix_t.h

Lines 128 to 153 in 8ab7aab

	emscripten::val getData() const {
	if (type == Types::U8_t) {
	emscripten::val view{emscripten::typed_memory_view(u8.size(), u8.data())};
	auto result = emscripten::val::global("Uint8Array").new_(u8.size());
	result.call<void>("set", view);
	return view;
	} else if (type == Types::S32_t) {
	emscripten::val view{
	emscripten::typed_memory_view(i32.size(), i32.data())};
	auto result = emscripten::val::global("Int32Array").new_(i32.size());
	result.call<void>("set", view);
	return view;
	} else if (type == Types::F32_t) {
	emscripten::val view{
	emscripten::typed_memory_view(f32.size(), f32.data())};
	auto result = emscripten::val::global("Float32Array").new_(f32.size());
	result.call<void>("set", view);
	return view;
	} else if (type == Types::F64_t) {
	emscripten::val view{
	emscripten::typed_memory_view(f64.size(), f64.data())};
	auto result = emscripten::val::global("Float64Array").new_(f64.size());
	result.call<void>("set", view);
	return view;
	}
	}

because the return view is not at the end of the statement. This should be fixed.

convert data_t members to a type-safe union with std::variant;

The variant class let define a variable with multiple types. May we use internally the data_t class?

Avoiding typdef's in types

Inside types.h

webarkit-jsfeat-cpp/src/types/types.h

Lines 11 to 12 in c02edb5

	typedef unsigned char u_char;
	typedef unsigned int u_int;

but this could be written as:

using u_char = unsigned char;
using u_int = unsigned int;

Develop a new log system

Instead of using the standard outptut (C++ cout or C printf) we could use our internal Log system. The idea is to implement a simple class as WebARKitLog or ARLog

Issue with orb example

Actually the orb example doesn't fully works as the jsfeatNext sample_orb.html. Wen need to find where is the bug. I will collect here all the infos for solving it.