cocoa-xu/evision

Evision: An OpenCV-Erlang/Elixir binding

OS	Arch	ABI	Build Status	Has Precompiled Library
Linux	x86_64	gnu		Yes
Linux	x86_64	musl		Yes
Linux	aarch64	gnu		Yes
Linux	aarch64	musl		Yes
Linux	armv6 (armhf)	gnueabihf		Yes
Linux	armv7l (armhf)	gnueabihf		Yes
Linux	ppc64le	gnu		Yes
Linux	s390x	gnu		Yes
Linux	riscv64	gnu		Yes
Linux	riscv64	musl		Yes
macOS 12 Monterey	x86_64	darwin		Yes
macOS 14 Sonoma	aarch64	darwin		Yes
Windows 2022	x86_64	msvc		Yes
Windows 2022	aarch64	msvc		Yes

Docs

Online docs for the latest released version is available on Hex.pm, https://hexdocs.pm/evision/.

Useful links

• Installation
• Use Precompiled Library (Default)
• Compile evision from source
• Nerves Support
• Register Builtin Smart Cells
• Integration with Nx
• Access behaviour (Getting a sub-area of an image)

Examples

Some examples are available in the examples directory.

Description

evision will pull OpenCV source code from GitHub, then parse and automatically generate corresponding OpenCV-Elixir bindings.

This project uses and modifies gen2.py and hdr_parser.py from the python module in the OpenCV repo so that they output header files that can be used in Elixir bindings.

We hope this project can largely reduce the work of manually porting OpenCV functions/modules to Elixir.

Compatible OpenCV versions:

• 4.5.3
• 4.5.4
• 4.5.5
• 4.6.0
• 4.7.0
• 4.8.0
• 4.9.0

by compatible, it means these versions can compile successfully, and I tested a small range of functions. Tons of tests should be written, and then we can have a list for tested OpenCV versions.

Installation

In order to use evision, you will need Elixir installed. Then create an Elixir project via the mix build tool:

$ mix new my_app

Then you can add evision as dependency in your mix.exs.

def deps do
  [
    {:evision, "~> 0.1"}
  ]
end

Please note that although :evision is available on hex.pm now, it's still in its early versions. And it will remain on v0.1.x for a while, and all v0.1.x versions should be treated as in very active development. Please read the CHANGELOG.md for all breaking changes even it's a "minor" update.

Therefore, it's recommended to use a specific version (i.e., include the minor version number in deps, {:evision, "~> 0.1.8"}, instead of {:evision, "~> 0.1"}) at the moment.

Use Precompiled Library (Default)

The following environment variables can be set based on your needs.

(Note that precompiled binaries do not use FFmpeg. If you'd like to use FFmpeg, please compile from source (please see instructions in the next section) and set corresponding environment variables. We're considering this option at the moment.)

Important notes

It is recommended to use :evision from hex.pm.

def deps do
  [
    {:evision, "~> 0.1"}
  ]
end

TARGET_ABI

Required if and only if the target is using musl libc.

# (for nerves project, this environment variable is set by nerves)
export TARGET_ABI=musl
## (for armv7l which uses hard-float ABI (armhf))
export TARGET_ABI=musleabihf

This variable will only be checked when identifying the musl libc ABI so that the correct precompiled binaries can be downloaded. Therefore,

1. You don't need to keep it in the runtime environment.
2. If you want to change it later, the directory _build/${MIX_ENV}/lib/evision needs to be deleted first.

The default value for the TARGET_ABI env var is obtained using the following elixir code

target_abi = List.last(String.split(to_string(:erlang.system_info(:system_architecture)), "-"))
target_abi =
  case target_abi do
    "darwin" <> _ -> "darwin"
    "win32" ->
      {compiler_id, _} = :erlang.system_info(:c_compiler_used)
      case compiler_id do
        :msc -> "msvc"
        _ -> to_string(compiler_id)
      end
    _ -> target_abi
  end

EVISION_PREFER_PRECOMPILED

# optional. 
# set this to "false" if you prefer :evision to be compiled from source
# 
# default value is "true", and :evision will prefer to use precompiled binaries (if available)
export EVISION_PREFER_PRECOMPILED=false

This variable will only be checked whenever the mix compile task is invoked directly (mix compile) or indirectly (mix test). And in the Makefile we would skip everything if _build/${MIX_ENV}/lib/evision/priv/evision.so is presented. Therefore,

1. You don't need to keep it in the runtime environment.
2. If you want to change it later, the directory _build/${MIX_ENV}/lib/evision needs to be deleted first.

If you found the precompiled binaries do not suit your needs (e.g., perhaps you need OpenCV to be compiled with FFmpeg to handle more video formats.), it's possible to override the behaviour by setting the environment variable EVISION_PREFER_PRECOMPILED to false, and then please delete _build/${MIX_ENV}/lib/evision and recompile evision

Also, for Linux users only, the precompiled binary is not compiled with GTK support, therefore functions like Evision.HighGui.imshow/2 will not work. However, you can either use Evision.Wx.imshow/2 (if Erlang on your system is compiled with wxWidgets), or set the environment variable EVISION_PREFER_PRECOMPILED to false so that OpenCV can detect available HighGui backend when compiling from source.

export EVISION_PREFER_PRECOMPILED=false

For livebook users,

Mix.install([
  {:evision, "~> 0.1"}
], system_env: [
  {"EVISION_PREFER_PRECOMPILED", "false"}
])

EVISION_ENABLE_CONTRIB

Set environment variable EVISION_ENABLE_CONTRIB to true to enable modules from opencv_contrib.

# enable opencv_contrib modules (default)
export EVISION_ENABLE_CONTRIB=true

# disable opencv_contrib modules
export EVISION_ENABLE_CONTRIB=false

This variable will only be checked whenever the mix compile task is invoked directly (mix compile) or indirectly (mix test). And in the Makefile we would skip everything if _build/${MIX_ENV}/lib/evision/priv/evision.so is presented. Therefore,

1. You don't need to keep it in the runtime environment.
2. If you want to change it later from false to true, you can delete the file _build/${MIX_ENV}/lib/evision/priv/evision.so, set EVISION_ENABLE_CONTRIB to true, and then execute mix compile.

Defaults to true because for precompiled binaries, including these "extra" modules only increases less than 20 MBs (tested on aarch64-apple-darwin) in size.

However, 20 MBs for Nerves users can be a huge deal (still depending on your device, for example, +20 MBs is often much more acceptable for RPIs as they are usually equipped with >= 8 GB microSD cards while being absolutely a luxury thing for some other embedded devices).

EVISION_ENABLE_CUDA

Set environment variable EVISION_ENABLE_CONTRIB to true to enable CUDA support from opencv_contrib. Defaults to false.

Note that EVISION_ENABLE_CONTRIB will need to be true as well.

# enable CUDA support
export EVISION_ENABLE_CUDA=true
## set a CUDA version that matches your local CUDA driver
## (this environment variable is only required for users who'd like to use precompiled binaries)
## available ones are 
## 118, for CUDA 11.8+, built with CUDA 11.8.0
## 121, for CUDA 12.1+, built with CUDA 12.1.0
export EVISION_CUDA_VERSION=118
## opencv_contrib modules is enabled by default
export EVISION_ENABLE_CONTRIB=true

# disable CUDA support (default) 
export EVISION_ENABLE_CUDA=false

IMPORTANT NOTE FOR WINDOWS USERS

If EVISION_ENABLE_CUDA is true, please also set CUDA runtime dir otherwise Evision will fail to load.

cmd

set EVISION_CUDA_RUNTIME_DIR=C:/PATH/TO/YOUR/CUDA/RUNTIME/BIN

Also, please don't quote even if there are spaces in the path

set EVISION_CUDA_RUNTIME_DIR=C:/PATH WITH SPACE/TO/YOUR/CUDA/RUNTIME/BIN

Powershell

$Env:EVISION_CUDA_RUNTIME_DIR="C:/PATH/TO/YOUR/CUDA/RUNTIME/BIN"
$Env:EVISION_CUDA_RUNTIME_DIR="C:/PATH WITH SPACE/TO/YOUR/CUDA/RUNTIME/BIN"

EVISION_PRECOMPILED_CACHE_DIR

# optional.
## set the cache directory for the precompiled archive file
export EVISION_PRECOMPILED_CACHE_DIR="$(pwd)/.cache"

Acknowledgements

• gen2.py, hdr_parser.py and c_src/erlcompat.hpp were directly copied from the python module in the OpenCV repo. Changes applied.
• Makefile, CMakeLists.txt and c_src/nif_utils.hpp were also copied from the torchx module in the elixir-nx repo. Minor changes applied.

evision's Issues

ImageDataGenerator would belong here?

I am not sure if this should belong into Evision or a seperate project, its quite simple and used all the time for kaggle problems.

Following as close as possible to Keras ImageDataGenerator
https://github.com/keras-team/keras-preprocessing/blob/master/keras_preprocessing/image/image_data_generator.py

Missing

• zoom
• shear
• brightness
• rescale
• rotation
• others

Use it like

    train_stream = ImageDataGenerator.stream(%{
       path: "priv/chest_xray/train/",
       class_map: %{"NORMAL": 0, "PNEUMONIA": 1},
       cache: true,
       open_flags: OpenCV.cv_IMREAD_GRAYSCALE,
       width: 128,
       height: 128,
       rescale: 255,
       batch_size: 16,
       shape: {16, 1, 128, 128},
       width_shift_range: -100..100,
       height_shift_range: -100..100,
       horizontal_flip: true
   })

   test_stream = ImageDataGenerator.stream(%{
       path: "priv/chest_xray/test/",
       class_map: %{"NORMAL": 0, "PNEUMONIA": 1},
       cache: true,
       open_flags: OpenCV.cv_IMREAD_GRAYSCALE,
       width: 128,
       height: 128,
       rescale: 255,
       batch_size: 99999,
       shape: {99999, 1, 128, 128},
   })
   
 trained_model = model
   |> Axon.Loop.trainer(:categorical_cross_entropy, Axon.Optimizers.adam(0.001))
   |> Axon.Loop.metric(:accuracy)
   |> Axon.Loop.handle(:epoch_completed, fn_epoch)
   |> Axon.Loop.run(train_stream, epochs: epochs, compiler: EXLA)   

defmodule ImageDataGenerator do
    alias Evision, as: OpenCV

    def stream(args) do
        path = args.path
        class_map = args.class_map
        width = args.width
        height = args.height
        shape = args.shape
        cache = args[:cache]
        open_flags = args[:open_flags]
        rescale = args[:rescale]
        horizontal_flip = args[:horizontal_flip]
        vertical_flip = args[:vertical_flip]
        width_shift_range = args[:width_shift_range]
        height_shift_range = args[:height_shift_range]
        shear_range = args[:shear_range]
        zoom_range = args[:zoom_range]
        brightness_range = args[:brightness_range]
        batch_size = args[:batch_size] || 1

        cacheable_args = Map.take(args, [:path, :width, :height, :open_flags])
        train = :persistent_term.get({:cache_idg, cacheable_args}, nil)
        train = cond do
            !!train && !!cache -> train
            true ->
                train_set = Enum.reduce(class_map, [], fn({name, value}, acc)->
                    files = Path.wildcard("#{path}/#{name}/*")
                    files = Enum.map(files, fn(path)->
                        img = OpenCV.imread!(path, flags: open_flags)
                        img = OpenCV.resize!(img, [width, height])
                        bin = OpenCV.Mat.to_binary!(img)
                        %{path: path, onehot: value, bin: bin}
                    end)
                    acc ++ files
                end)
                cache && :persistent_term.put(:cache_train, train_set)
                train_set
        end
        
        batch_size = if batch_size > length(train) do
            length(train)
        else batch_size end

        0 = rem(length(train), batch_size)

        args = Map.merge(args, %{batch_size: batch_size})

        stream_1(train, args)
    end

    def stream_1(train_set, args) do
        Stream.resource(
          fn ->
            train_set = Enum.shuffle(train_set)
            idx = 0
            {train_set, idx}
          end,
          fn {data, idx} ->
            ts_start = :os.system_time(1000)
            offset = idx * args.batch_size
            cond do
              offset >= length(data) -> {:halt, {data, idx}}
              true ->
                slice = Enum.slice(data, offset, args.batch_size)
                {img_batch, label_batch} = Enum.reduce(slice, {"", ""}, 
                    fn(%{bin: bin, onehot: onehot}, {img_batch, label_batch}) ->
                        bin = transform(bin, args)
                        {img_batch <> bin, label_batch <> <<onehot::8>>}
                    end
                )
                {x_train, y_train} = prepare_nx(img_batch, label_batch, args)

                ts_end = :os.system_time(1000)
                took = ts_end - ts_start
                #IO.puts "batch took #{took}ms"

                {[{x_train, y_train}], {data, idx+1}}
            end
          end,
          fn _acc -> :ok end
        )        
    end

    def prepare_nx(img_batch, label_batch, args) do
        x_train = img_batch
        |> Nx.from_binary({:u, 8})
        |> Nx.reshape(args.shape)
        #TODO: case do in pipe dont compile with defn
        x_train = if args[:rescale] do
            Nx.divide(x_train, args.rescale)
        else x_train end

        y_train = label_batch
        |> Nx.from_binary({:u, 8})
        |> Nx.reshape({args.batch_size, 1})
        |> Nx.equal(Nx.tensor(Enum.to_list(0..1)))
        {x_train, y_train}
    end

    def transform(bin, args) do
        img = OpenCV.Mat.from_binary!(bin, {:u, 8}, args.width, args.height, 1)

        img = if !!args[:horizontal_flip] and :rand.uniform(2) == 1 do
            OpenCV.flip!(img, 1)
        else img end

        img = if !!args[:vertical_flip] and :rand.uniform(2) == 1 do
            OpenCV.flip!(img, 0)
        else img end

        img = if args[:width_shift_range] do
            factor = Enum.random(args.width_shift_range) / 1000
            opencv_shift_width(img, factor)
        else img end

        img = if args[:height_shift_range] do
            factor = Enum.random(args.height_shift_range) / 1000
            opencv_shift_height(img, factor)
        else img end

        #todo zoom
        #todo shear
        #todo brightness

        OpenCV.Mat.to_binary!(img)
    end

    def opencv_shift_width(img, factor) do
        {width, height} = OpenCV.Mat.shape!(img)
        to_shift = trunc(Float.round(width * factor))
        m = Evision.Nx.to_mat!(Nx.tensor([[1,0,to_shift],[0,1,0]], type: {:f, 32}))
        Evision.warpAffine!(img, m, [width, height])
    end

    def opencv_shift_height(img, factor) do
        {width, height} = OpenCV.Mat.shape!(img)
        to_shift = trunc(Float.round(width * factor))
        m = Evision.Nx.to_mat!(Nx.tensor([[1,0,0],[0,1,to_shift]], type: {:f, 32}))
        Evision.warpAffine!(img, m, [width, height])
    end

    def opencv_zoom(img, factor) do
        #img = OpenCV.Mat.from_binary!(hd(train).bin, {:u,8},128,128,1)

        factor = 1.0 + factor
        {width, height} = OpenCV.Mat.shape!(img)
        new_width = trunc(width * factor)
        new_height = trunc(height * factor)
        img2 = OpenCV.resize!(img, [new_width, new_height])

        # Original: 529x550
        # Zoomed: 794x825 


        #translation_matrix = np.float32([ [1,0,70], [0,1,110] ])   
        #img_translation = cv2.warpAffine(img, translation_matrix, (num_cols, num_rows))

        #height, width = img.shape[:2]
        #zoomed = zoom(img, 1.5)
    end
end

How to use warpPerspective.

Hey brother, I want to crop the image where only the single contour has been drawn by the image processing techniques. I have seen on google and youtube that people uses warpPerspective and i was trying to do the same by the same way as they are doing. But unfortunately it did not work.

Could you please tell the usage of that function as well as those fucntions whichever required in order to use warpPerspective.
If using warpPerspective, I can not achieve the desired output may you suggest any way to achieve that.

Add cache in CI script

Wrap Evision.Mat as a struct?

@cocoa-xu, thanks for this great library.

I'm just now working on zero-copy integration between Image and eVision (with all the hard work on my side done by @akash-akya).

I'm wondering if in the future you intend to wrap the NIF reference to an Evision.Mat into an Elixir struct? With the current type being a naked reference its not possible to pattern match on the type and I don't think there is a way to safely identify if the reference is an Evision.Mat?

How to crop/cut?

In python its

import cv2
img = cv2.imread("lenna.png")
crop_img = img[y:y+h, x:x+w]
cv2.imshow("cropped", crop_img)
cv2.waitKey(0)

Evision.Mat.transpose/{1, 2} calls shape/1 not shape!/1; returns invalid shape

When calling Evision.Mat.transpose/{1,2} I think there are three issues:

1. The call to shape/1 on line 229 should be shape!/1
2. The call to shape/1 on line 269 should be shape!(mat) |> Tuple.to_list()
3. The return result from transpose/{1, 2} returns an invalid shape even after these two issues are corrected. This results in downstream operations like Evision.imwrite/2 failing.

Example

# Create a tensor
iex(1)> tensor = Nx.tensor  [                                           
...(1)>     [
...(1)>       [55, 165, 0],
...(1)>       [55, 165, 0],
...(1)>       [55, 165, 0]
...(1)>     ],
...(1)>     [
...(1)>       [75, 165, 0],
...(1)>       [75, 165, 0],
...(1)>       [75, 165, 0]
...(1)>     ],
...(1)>     [
...(1)>       [155, 165, 0],
...(1)>       [155, 165, 0],
...(1)>       [155, 165, 0]
...(1)>     ],
...(1)>     [
...(1)>       [255, 165, 0],
...(1)>       [255, 165, 0],
...(1)>       [255, 165, 0]
...(1)>     ]
...(1)>   ], type: {:u, 8}
#Nx.Tensor<
  u8[4][3][3]
  ...
>

# Convert to an Evision mat
iex> {:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, #Reference<0.2283891390.1959395348.49054>}

# Check the shape is the same as the original tensor
iex> Evision.Mat.shape(mat)                 
{:ok, {4, 3, 3}}

# Transpose the axis. This will be the case when converting
# tensors that come from `Image` because we need to transpose
# from {width, height, bands} to {height, width, bands}
iex> {:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, #Reference<0.2283891390.1959395348.49056>}

# Now the shape has too many dimensions!
# The first three tuple elements appear to be the correct
# ones. Perhaps the NIF function has an "off by one" issue?
iex> Evision.Mat.shape(transposed)                     
{:ok, {3, 4, 3, 3}}

`{:ok, ...}` is too verbose`!`

Fail to download the precompiled binary of 0.1.0

I ran the following Elixir script:

System.put_env("EVISION_PREFER_PRECOMPILED", "true")
System.put_env("EVISION_PRECOMPILED_VERSION", "0.1.0")
System.put_env("EVISION_PRECOMPILED_CACHE_DIR", "#{System.user_home!()}/.cache")
Mix.install([
  {:evision, "~> 0.1.0", github: "cocoa-xu/evision", tag: "v0.1.0"}
])

Then, I got the following error:

* Getting evision (https://github.com/cocoa-xu/evision.git - v0.1.0)
remote: Enumerating objects: 4412, done.        
remote: Counting objects: 100% (834/834), done.        
remote: Compressing objects: 100% (260/260), done.        
remote: Total 4412 (delta 599), reused 741 (delta 526), pack-reused 3578        
Resolving Hex dependencies...
Dependency resolution completed:
New:
  dll_loader_helper 0.1.7
  elixir_make 0.6.3
* Getting elixir_make (Hex package)
* Getting dll_loader_helper (Hex package)
==> dll_loader_helper
Compiling 1 file (.erl)
Compiling 1 file (.ex)
Generated dll_loader_helper app
==> elixir_make
Compiling 1 file (.ex)
Generated elixir_make app
==> evision
CMake Warning:
  Ignoring empty string ("") provided on the command line.


-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using PRIV_DIR: /Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/priv
-- EVISION_PREFER_PRECOMPILED is true: try to download and use the precompiled library
-- precompiled tarball URL: https://github.com/cocoa-xu/evision/releases/download/0.1.0/evision-darwin-arm64-0.1.0.tar.gz
-- precompiled tarball cache file: /Users/zacky/.cache/evision-darwin-arm64-0.1.0.tar.gz
CMake Error at CMakeLists.txt:78 (message):
  Cannot download precompiled package from
  https://github.com/cocoa-xu/evision/releases/download/0.1.0/evision-darwin-arm64-0.1.0.tar.gz:
  "HTTP response code said error"

  You may download it manually and put it in
  /Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/priv/evision-darwin-arm64-0.1.0


-- Configuring incomplete, errors occurred!
See also "/Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/cmake_evision/CMakeFiles/CMakeOutput.log".
See also "/Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/cmake_evision/CMakeFiles/CMakeError.log".
incomplete build of OpenCV found in '/Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/cmake_opencv_4.5.5', please delete that directory and retry
make: *** [/Users/zacky/Library/Caches/mix/installs/elixir-1.14.0-rc.1-erts-13.0.3/8262caacbab21b886e01a468491819cb/_build/dev/lib/evision/priv/evision.so] Error 1
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
** (Mix.Error) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    (mix 1.14.0-rc.1) lib/mix.ex:513: Mix.raise/2
    (elixir_make 0.6.3) lib/mix/tasks/compile.make.ex:154: Mix.Tasks.Compile.ElixirMake.run/1
    (mix 1.14.0-rc.1) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.0-rc.1) lib/mix/tasks/compile.all.ex:92: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.14.0-rc.1) lib/mix/tasks/compile.all.ex:72: Mix.Tasks.Compile.All.compile/4
    (mix 1.14.0-rc.1) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.14.0-rc.1) lib/mix/tasks/compile.all.ex:33: Mix.Tasks.Compile.All.run/1
    (mix 1.14.0-rc.1) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4

It seems to try to download https://github.com/cocoa-xu/evision/releases/download/0.1.0/evision-darwin-arm64-0.1.0.tar.gz.

I guess the correct URL is https://github.com/cocoa-xu/evision/releases/download/v0.1.0/evision-darwin-arm64-0.1.0.tar.gz.

Can't round trip QRcode encode/decode

QRcode detection is working well (as far as I can test) but when I generate a QRcode and attempt to detect it I am not having any luck. For example:

iex> qrcode = Evision.QRCodeEncoder.encode Evision.QRCodeEncoder.create(), "This is a string"           
%Evision.Mat{
  channels: 1,
  dims: 2,
  type: {:u, 8},
  raw_type: 0,
  shape: {25, 25},
  ref: #Reference<0.1002129465.3225026580.34308>
}
iex> Evision.QRCodeDetector.detectAndDecode Evision.QRCodeDetector.qrCodeDetector(), qrcode             
{"",
 %Evision.Mat{
   channels: 2,
   dims: 2,
   type: {:f, 32},
   raw_type: 13,
   shape: {1, 4, 2},
   ref: #Reference<0.1002129465.3225026580.34310>
 }, {:error, "empty matrix"}}

1. The encoded image is very small, {25, 25} and I don't actually know how - even if its possible - to define parameters for the encoder to make the image larger.
2. I can see there is a module Evision.QRCodeEncoder.Params but I'm not having any success try to define a param structure. For example:

iex> Evision.QRCodeEncoder.Params.get_mode Evision.QRCodeEncoder.Params.qrcodeencoder_params()
zsh: segmentation fault  iex -S mix

Thanks for your patience @cocoa-xu while I work out have to navigate the wonderful world of OpenCV/eVision - help and guidance much appreciated.

knnMatch with trainDescriptors is missing

Hi,

I'd like to port the following Python code:

import cv2

def get_keypoints(img):
    gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    sift = cv2.AKAZE_create()
    return sift.compute(gray, sift.detect(gray))

base = cv2.imread("base_img.jpg")
target = cv2.imread("target_img.jpg")

kp_base, des_base = get_keypoints(base)
kp_target, des_target = get_keypoints(target)

bf = cv2.BFMatcher()
matches = bf.knnMatch(des_base, des_target, k = 2)

I wrote it in Elixir as follows, but I got an error:

defmodule DiffBench do
  def get_keypoints!(img) do
    gray = Evision.cvtColor!(img, Evision.cv_COLOR_RGB2GRAY)
    sift = Evision.AKAZE.create!()
    kp = Evision.AKAZE.detect!(sift, gray)
   Evision.AKAZE.compute!(sift, gray, kp)
  end
end

base = Evision.imread!("base_img.jpg")
target = Evision.imread!("target_img.jpg")
{_, des_base} = DiffBench.get_keypoint!(base)
{_, des_target} = DiffBench.get_keypoint!(target)

bf = Evision.BFMatcher.bfMatcher!()
matches = Evision.DescriptorMatcher.knnMatch!(bf, des_base, des_target, 2) # I got an error here

I guess the following:

1. knnMatch in Python is overloaded by a function with / without trainDescriptors, but Evision doesn't seem to have such a function with trainDescriptors.
2. The module Evision.BFMatcher should have knnMatch functions.

OpenCV.gettextsize seems to return invalid size.

{:ok, {{label_height, label_weight}, baseline}} = OpenCV.gettextsize("test text", OpenCV.cv_font_hershey_simplex, 0.5, 1)

expecting label_height and label_weight to be greater than 1.0.

doubt on image croping in Evision.Nx and Nx.slice

i haven doubt in Nx.slice we are working on it we converted image to 2d array by using your instructions could you please help us @navrkr @cocoa-xu

how can slice function work on it

we are trying to give all but its not work please help us sir

Originally posted by @Jagan3534 in #102 (comment)

imshow function is broken in pre-compiled library.

I recently updated from elixir 1.0.0-dev to 1.0.9 version of Evision and everything is great till now, just tried getting the image as output and this error popped (Note: All the issue is with cvwait and imshow doesn't work without cvwait).

[Bug] Autocomplete doesn't work.

I checked with other packages like nx and complex.

I am using neovim with elixir ls. Also tested with vscode elixirls extension.

I guess that is because of how evision works and compiles from cpp code.

In neovim:
Doesn't Works

Works

Functions in `snake_case`

Is there any technical limitation that obligates having functions in camelCase? If not, I would like to suggest renaming the functions to use the snake_case convention.

Build with OpenCV 5.x

Prepare for OpenCV 5.x.
Branch test-opencv-5.0-pre.

Share image data between Vix and eVision (and Nx)

I am the author of the Image library that is only possible because of @akash-akya's Vix which is a binding to libvips. Given Vix and eVision have similar problem domains, and very similar matrix implementations, I would like to be able to share image data between the two libraries. This would allow Image (and of course any other library) to leverage the capabilities of both Vix and eVision (and very desirably Nx) in a very memory efficient and performant manner.

Objectives

1. Share image data without copying the matrix (I think a shared interpretation is possible) - but need to make sure we know who has memory ownership and who is managing the memory (and I'm not good at this part).
2. Having a way to agree and share data formats (ie a common lexicon to refer to image data type and matrix shape - perhaps using the Nx terminology?)
3. Act as a model of how to do the same integration with Nx. I know eVision has Nx integration but it appears to be based upon converting to and from Elixir binaries - which I'd prefer not to do since lots of NIF copying would contradict the memory efficiency and performance objectives.

I'm looking for guidance, suggestions and ideas - and hopefully someone motivated to tackle this because its a bit beyond my competence. :-)

Given the intent to also provide integration with Nx, any comments from @josevalim would also be very welcome.

Can't compile evision 0.1.8 on MacOS (Intel)

@cocoa-xu this is looking really good but I think there may be an issue with 0.1.8. I have no issues with 0.1.7 on either MacOS Intel or MacOS ARM but with 0.1.8 I see two issues:

1. Issues with TLS peer verification (I have some code that might help if you're using :httpc)
2. What looks like an asset missing from the package?

kip@Kips-iMac-Pro image % iex -S mix                                                           
Erlang/OTP 25 [erts-13.1] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]


11:33:28.673 [info] EVISION_PREFER_PRECOMPILED: true; try to download and use the precompiled library.
 
11:33:28.676 [info] Requested version `0.1.8` has precompiled binaries.

11:33:28.676 [info] Current target `x86_64-apple-darwin` has precompiled binaries.
 
11:33:28.705 [warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

 
11:33:29.115 [warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

 
11:33:33.109 [info] Precompiled binary tarball downloaded and saved to /Users/kip/Library/Caches/evision-x86_64-apple-darwin-0.1.8.tar.gz, sha256=2af7710079074f607342717a0a7a6e6d6d90aca2b7038420636602954f6afa65
==> evision
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
** (MatchError) no match of right hand side value: {:error, "the precompiled NIF file does not exist in the checksum file. Please consider run: `EVISION_FETCH_PRECOMPILED=true mix evision.fetch --all` to generate the checksum file."}
    /Users/kip/Development/image/deps/evision/mix.exs:298: Mix.Tasks.Compile.EvisionPrecompiled.prepare/3
    (mix 1.14.0) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:92: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:72: Mix.Tasks.Compile.All.compile/4
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:33: Mix.Tasks.Compile.All.run/1
    (mix 1.14.0) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.0) lib/mix/tasks/compile.ex:134: Mix.Tasks.Compile.run/1

Evision.DNN.Classification can't read ONNX file directly

image = Evision.imread("dog.jpg")
weights = "efficientnet-b7.onnx"
classes = "imagenet.names" |> File.read! |> String.split("\n")

model = 
  Ev.DNN.ClassificationModel.classificationModel(weigth)
  |> Ev.DNN.ClassificationModel.setInputParams(
    scale: 1.0 / 255.0,
    size: {600, 600},
    mean: {124.675, 116.28, 103.53},
    swapRB: true,
    crop: true
  )
  
{class_id, confidence} = Ev.DNN.ClassificationModel.classify(model, image)

return

opencv/opencv-4.6.0/modules/dnn/src/net.cpp:104: error: (-215:Assertion failed) !empty() in function 'forward'

It worked fine via DNN.readNetFromONNX.

model = 
  Ev.DNN.readNetFromONNX(weights)
  |> Ev.DNN.ClassificationModel.classificationModel()
  |> Ev.DNN.ClassificationModel.setInputParams(
    scale: 1.0 / 255.0,
    size: {600, 600},
    mean: {124.675, 116.28, 103.53},
    swapRB: true,
    crop: true
  )
  
{class_id, confidence} = Ev.DNN.ClassificationModel.classify(model, image)
Enum.at(classes, class_id)

"malamute, malemute, Alaskan malamute"

Error in Evision.findContours function

Respected ma'am,
please go through with following error occur in the mentioned function

iex(29)> Evision.findContours(img, 1, 1)
{:error,
"OpenCV(4.6.0) /home/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/imgproc/src/contours.cpp:195: error:
(-210:Unsupported format or combination of formats) [Start]FindContours supports only CV_8UC1 images when mode !=  
CV_RETR_FLOODFILL otherwise supports CV_32SC1 images only in function 'cvStartFindContours_Impl'\n"}
iex(30)> 

This function was working fine, two hours ago, but now returning above error.

[Feature-GUI] Implementing `Evision.Wx`

As mentioned in #103:

Even compiling OpenCV with highgui module, sometimes the Evision.HighGui.imshow function does not really work as expected (in evision): the window(s) it opened will not respond to any GUI events until calling Evision.HighGui.waitKey again.

To make it as good as one would expect it to be, like how it is in C++/Python, perhaps it would require one to invest a few weeks of time for it. But I'm focusing on other aspects of this library right now, also I'm hesitating to do it because I don't know if this would worth that amount time.

Also, as :wx has official support from Erlang, and it's well tested, and one can even build a relatively complex GUI app with :wx if they want.

This module will try to make itself easy to use. Meanwhile, the Evision.HighGui module will stay in the source code even after Evision.Wx is completed.

Here is a non-exhaustive list of desired/prospective functions in Evision.Wx. All of them, including the type and the number of their input arguments, are subject to changes in the future, or will not be implemented if the desired functionality is partially or wholly impossible to achieve in Erlang/Elixir with :wx.

• imshow/2. Equivalent of cv::imshow.
• destroyWindow/1. cv::destroyWindow.
• destroyAllWindows/0. cv::destroyAllWindows.
• namedWindow/2. cv::namedWindow.

Functions available in OpenCV's highgui module can be found here.

Any contributions or suggestions (either about Evision.Wx or Evision.HighGui) are more than welcomed - this also applies to the whole project. :)

How should I use Evision.Feature2D.detect!/2?

Hi,

The following python code works well:

import cv2

img = cv2.imread("test_img.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
sift = cv2.AKAZE_create()
kp = sift.detect(gray)

However, the following elixir code doesn't work well:

Mix.install([{:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}])

img = Evision.imread!("test_img.jpg")
gray = Evision.cvtColor(img, Evision.cv_COLOR_RGB2GRAY)
sift = Evision.AKAZE.create!()
kp = Evision.Feature2D.detect!(sift, gray)

I got the following error:

** (ArgumentError) argument error
    (evision 0.1.0-dev) :erl_cv_nif.evision_cv_Feature2D_detect(#Reference<0.2183360161.3024486401.45738>, [image: #Reference<0.2183360161.3024486402.45065>])
    (evision 0.1.0-dev) lib/generated/opencv_feature2d.ex:141: Evision.Feature2D.detect!/2

I don't understand the Evision.Feature2D.detect!/2 function because it is not documented enough, so please let me to know how I should use the function.

Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

Hi,

I did the following steps:

mix new evision_test
cd evision_test

Then, I added evision to mix.exs as follows:

  defp deps do
    [
      {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
    ]
  end

Then, I did the following steps:

mix deps.get
mix compile

Then, I got the following error:

==> evision
[/Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip or
        /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.zip, and cannot find /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.ZIP, period.
make: *** [/Users/zacky/evision_test/deps/evision/3rd_party/opencv/opencv-4.5.5/modules/core/include/opencv2/core/utils/configuration.private.hpp] Error 9
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

Then, I did the following steps:

mix deps.compile evision

Then, I got the following fatal error:

==> evision
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
(snip)
-- General configuration for OpenCV 4.5.5 =====================================
--   Version control:               ca737c9
-- 
--   Platform:
--     Timestamp:                   2022-01-26T09:38:53Z
--     Host:                        Darwin 21.2.0 arm64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/make
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 13.0.0.13000029)
--     C++ flags (Release):         -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python2 stitching ts video videoio
--     Disabled:                    gapi world
--     Disabled by dependency:      -
--     Unavailable:                 java python3
--     Applications:                perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        zlib (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         libpng (ver 1.6.37)
--     TIFF:                        libtiff (ver 42 / 4.2.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     OpenEXR::OpenEXR (ver 3.1.3)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       YES (ver 3.4.0)
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                NO
--     Link libraries:              -framework OpenCL
-- 
--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
--     Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.18)
--     numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
--     install path:                lib/python2.7/site-packages/cv2/python-2.7
-- 
--   Python (for build):            /usr/bin/python2.7
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /Users/zacky/evision_test/_build/dev/lib/evision/priv
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5
[  0%] Building C object 3rdparty/ittnotify/CMakeFiles/ittnotify.dir/src/ittnotify/ittnotify_static.c.o
[  0%] Generate opencv4.pc
[  0%] Building C object 3rdparty/zlib/CMakeFiles/zlib.dir/adler32.c.o
[  0%] Building C object 3rdparty/openjpeg/openjp2/CMakeFiles/libopenjp2.dir/thread.c.o
(snip)
[ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/vkcom/vulkan/vk_loader.cpp.o
[ 72%] Linking CXX shared library ../../lib/libopencv_dnn.dylib
[ 72%] Built target opencv_dnn
make[1]: *** [all] Error 2
make: *** [/Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 2
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

My environment is as follows:

$ uname -a 
Darwin zackym1air.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101 arm64
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

Thank you.

Import Nx tensor that is in {width, height, channels} shape with RGB data format

Cocoa, I have tried everything I can think of to take an Nx tensor that is of the shape {width, height, channels} that contains RGB format data, and convert it in eVision to {height, width, channels} in BGR format.

Any chance I might ask for your advice and recommendations?

What I've tried

Here is what I have tried which is, I think, the required process but the saved image is definitely not what is expected!

tensor = File.read!("path_to/color_checker.etf") |> :erlang.binary_to_term()
{:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, bgr} = Evision.cvtColor(transposed, Evision.cv_COLOR_RGB2BGR())
Evision.imwrite "some_path/color_checker.jpg", bgr

I have followed the converse process in Image and the results line up with expectations. Which is not too surprising since Image expects data in {width, height, channels} and RGB format. I mention this just to note that I have verified that the tensor does represent the underlying example image.

Artifacts

The image converted to a tensor and stored with :erlang.term_to_binary/1 and zipped:
color_checker.etf.zip

The original image:

Guard Preventing Passing "None" For calcHist()

In Python, this is allowed:

cv2.calcHist([image], [0, 1, 2], None, [8, 8, 8],[0, 256, 0, 256, 0, 256])

When making the equivalent call in evision (I substituted nil for None):

Cv.calcHist([image], [0, 1, 2], nil, [8, 8, 8],[0, 256, 0, 256, 0, 256])

the result is ** (FunctionClauseError) no function clause matching in Evision.calcHist/5 **

with the bolded text for mask being the issue.

def calcHist(images, channels, mask, histSize, ranges) when is_list(images) and is_list(channels) and **is_reference(mask)** and is_list(histSize) and is_list(ranges)

Looking at the OpenCV code, the spec is:

Python: cv2.calcHist(images, channels, mask, histSize, ranges[, hist[, accumulate]])

where

mask – Optional mask. If the matrix is not empty, it must be an 8-bit array of the same size as images[i] .

imgproc.hpp has

CV_EXPORTS void calcHist( const Mat* images, int nimages, const int* channels, InputArray mask, OutputArray hist, int dims, const int* histSize, const float** ranges, bool uniform = true, bool accumulate = false );

In hdr_parser.py the mapping seems to be:

if arg_type == "InputArray": arg_type = mat
so maybe Python is converting "None" to something automatically? Maybe a blank matrix?

I'm not sure if the solution is to convert nil to blank matrix, or pass in a blank matrix (and maybe make a method to auto-generate it from the image).

I'd be happy to help with this, if I can get some guidance on the best way to go since my knowledge of evision is limited.

Provide precompiled binaries and auto-generated Elixir source code

Using GitHub CI should be okay to achieve this goal. The following operating systems and CPU architectures will be supported.

• macOS, x86_64
• Linux, x86_64
• Linux, arm64

macOS + arm64 could be done (by cross-compiling on an x86_64 macOS), but definitely will need to make changes to the CMakeLists.txt. We can compile this natively after arm64 macOS is available to use on GitHub CI.

Of course, there are several things to note:

• We will compile all supported modules in the precompiled version, which means the footprint will be larger (but it should be just ~5-20 megabytes of increment compared to the minimal setup)
• FFmpeg libraries will be needed. Or we can do another version that is not compiled with FFmpeg support.

Error running gen2.py

Hi,
I am trying to compile this module on Nvidia Jetson Nano.
The compilation cannot correctly finish since the gen2.py script is complaining with the error:

<class '_ast.Num'> not implemented yet

Can you help me in understanding what I can do to fix it? If needed, I can provide the full output of mix deps.compile.

Thank you.

Evision.boxPoints/1 is not working

It is asking for a map as input but we are getting a tuple as output for minAreaRect

*** (FunctionClauseError) no function clause matching in Evision.boxPoints/1

The following arguments were given to Evision.boxPoints/1:

    # 1
    {{224.0, 262.5}, {343.0, 344.0}, 90.0}

Attempted function clauses (showing 1 out of 1):

    def boxPoints(box) when is_reference(box) or is_map(box) and is_map_key(box, :__struct__) and is_atom(:erlang.map_get(:__struct__, box))

(evision 0.1.7-dev) lib/generated/evision.ex:6346: Evision.boxPoints/1

[Feature | Livebook] Make the default preview for Evision.Mat "Image" instead of "Raw"

It maybe better to keep the default output to "Image" instead of "Raw".
An alternative could be to save the last type of preview selected.
Even better some way to change the preview type. Like a setting.

It is annoying to switch to image each time I run the cell

Some previous discussion
livebook-dev/livebook#1506

sorry for what is did . we have a small problem in sort sir

I apologise if i sound rude, i did not know english that well may be this is why i sounds rude.

here same we got the error in the elixir in python we wrote like this
in elixir we tried to give but in sort 2 nd parameter is doubted sir in Evision hexdoxs we tried to see the example but in new version they dont show
contours = sorted(contours, key=cv2.contourArea, reverse=True)

could you please help us

Namespace functions should be transformed to `OpenCV.{NameSpace}.{functionName}`

small doubt regarding contours in evision

i get the values of findContours after getting is their any method to seperate contours using in built function like in python imutil.grabcontours

the code is here

`def grab_contours(cnts):
    # if the length the contours tuple returned by cv2.findContours
    # is 2 then we are using either OpenCV v2.4, v4-beta, or
    # v4-official
    if len(cnts) == 2:
        cnts = cnts[0]

    # if the length of the contours tuple is 3 then we are using
    # either OpenCV v3, v4-pre, or v4-alpha
    elif len(cnts) == 3:
        cnts = cnts[1]

    # otherwise OpenCV has changed their cv2.findContours return
    # signature yet again and I have no idea WTH is going on
    else:
        raise Exception(("Contours tuple must have length 2 or 3, "
            "otherwise OpenCV changed their cv2.findContours return "
            "signature yet again. Refer to OpenCV's documentation "
            "in that case"))

    # return the actual contours array
    return cnts`

could please help us sir @cocoa-xu @josevalim

Help getting the livebook examples to run?

This library looks very promising!

I was able to get the code to compile locally on my Macbook Air M1 (took a bit of doing, e.g. adding homebrew paths, etc.), but when I try to run the livebooks locally, I am getting this error.

The error is obviously not that make or gcc is missing:

I was getting that same error just running a mix compile of evision, until I fixed my CPATH & LD_LIBRARY_PATH to include the homebrew paths, e.g. add to ~/.zshrc:

 export LD_LIBRARY_PATH="/opt/homebrew/lib:/opt/homebrew/opt/"
 export CPATH=/opt/homebrew/include

I can see that the livebook has the proper paths set, e.g. System.get_env from livebook shows:

  ...
  "LD_LIBRARY_PATH" => "/opt/homebrew/lib:/opt/homebrew/opt/",
  "CPATH" => "/opt/homebrew/include",
  ...

Anyway, just wondering if you are able to run the livebooks & maybe I am missing something obvious!

Thanks again for working on this!!!

How to shift?

> OpenCV.warpAffine! img, [[1,0,70],[0,1,100]], [128,128]
Segmentation fault (core dumped)

To shift the image by 70 pixels on the x and 100 on the y. Image size is 128x128 1 channel

Proposal: bridging with Nx

I'm developing Excv, a bridge between Nx and OpenCV: https://github.com/zeam-vm/excv

I hope that evision will be integrated with such bridging. I'd like to contribute to evision if you wish so. Would you please inform me where the #Reference structure includingMat?

segmentation fault

Respected Ma'am \ Sir,
After updating evision, segmentation fault happened in the function Evision.findContours(Evision.mat, method, mode ).

code snippet:-

  iex(1)> path = "images/grid_1.png"
  "images/grid_1.png"
  iex(2)> {_, img_mat} = Evision.imread path
  {:ok,
       %Evision.Mat{
           channels: 3,
           dims: 2,
           raw_type: 16,
           ref: #Reference<0.2914473186.3120168976.115461>,
           shape: {1080, 1080, 3},
           type: {:u, 8}
  }}
  iex(3)> Evision.findContours(img_mat, 1, 1)
  Segmentation fault (core dumped)

cropping of image using ROI not working

Please refer to the below images to understand about the issue.
This is a function used to get image:

This is the error window:

Its working in python for reference:

Link to the documentation seems broken

I might have fixed it. If it looks good let me know, I will make a PR.

main...tusqasi:evision:main

OpenCV stitching seems to not work

Love this effort, just gave it a whirl on stream and had lots of fun.

I tried to create a stitcher and both OpenCV.Sticher.create() and OpenCV.stitcher_create() would just return the below error in an error tuple:

 'OpenCV(4.5.4) /home/lawik/projects/eyes/deps/evision/3rd_party/opencv/opencv-4.5.4/modules/stitching/src/stitcher.cpp:94: error: (-5:Bad argument) Invalid stitching mode. Must be one of Stitcher::Mode in function \'create\'\n'}

I couldn't find a way of passing a mode and the docs seemed to indicate it mostly wants a boolean. I don't know, I may be missing something but in case this identifies a useful issue with evision I figured I'd mention it.

Issue with of Evision.drawcontour function

(need to have private chat regarding this project)

Please refer to the following syntax error and fix it asap please.

Proposal: Function grouping by ex_doc because too many functions in the OpenCV module to read and search

When opening docs of evision, I found too many functions in the OpenCV module to read and search.

So, I propose grouping functions by the following ex_doc features:

http://f.cybrox.eu/exdocdemo/Mix.Tasks.Docs.html#module-groups

Checksum mismatch updating to eVision 0.1.10

@cocoa-xu congrats on the big update. I'm updating from 0.1.8 but despite cleaning everything I'm seeing a checksum mismatch:

kip@Kips-MacBook-Pro image % mix deps.compile evision                            

16:10:27.715 [info] EVISION_PREFER_PRECOMPILED: true; try to download and use the precompiled library.

16:10:27.718 [info] Requested version `0.1.10` has precompiled binaries.

16:10:27.718 [info] Current target `aarch64-apple-darwin` has precompiled binaries.

16:10:27.718 [info] Current NIF version `2.16` has precompiled binaries.

16:10:27.728 [info] Precompiled binary tarball cached at /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz, sha256=6af3996073ff8b6e67e854f22aa35844a7bdfed195352e61b3d686eabe3998a4

16:10:27.728 [error] Checksum mismatched: /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz[sha256=6af3996073ff8b6e67e854f22aa35844a7bdfed195352e61b3d686eabe3998a4], expected:[sha256=6209110fea311e0f5c14d459772b6dd549b96af69c3be400980a368f7b198f0b]

16:10:27.728 [warning] Will delete cached tarball /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz and re-download

16:10:28.890 [info] Precompiled binary tarball downloaded and saved to /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz, sha256=6af3996073ff8b6e67e854f22aa35844a7bdfed195352e61b3d686eabe3998a4

16:10:28.892 [error] Checksum mismatched: downloaded file: /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz[sha256=6af3996073ff8b6e67e854f22aa35844a7bdfed195352e61b3d686eabe3998a4], expected:[sha256=6209110fea311e0f5c14d459772b6dd549b96af69c3be400980a368f7b198f0b]
==> evision
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
** (RuntimeError) Checksum mismatched: downloaded file: /Users/kip/Library/Caches/evision-nif_2.16-aarch64-apple-darwin-0.1.10.tar.gz[sha256=6af3996073ff8b6e67e854f22aa35844a7bdfed195352e61b3d686eabe3998a4], expected:[sha256=6209110fea311e0f5c14d459772b6dd549b96af69c3be400980a368f7b198f0b]
    /Users/kip/Development/image/deps/evision/mix.exs:484: Mix.Tasks.Compile.EvisionPrecompiled.prepare/4
    (mix 1.14.0) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:92: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:72: Mix.Tasks.Compile.All.compile/4
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.14.0) lib/mix/tasks/compile.all.ex:33: Mix.Tasks.Compile.All.run/1
    (mix 1.14.0) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.0) lib/mix/tasks/compile.ex:134: Mix.Tasks.Compile.run/1

I've removed the cached tarball, mix deps.clean evision and repeated but still the same result. Any suggestions?

OpenCV gapi module

Based on the contents of pyopencv_gapi.hpp, it seems that the binding code will need to call an Elixir function and fetch the result.

This perhaps could be done if we send related ERL_NIF_TERMs to a separate Erlang thread and send the result back. But it's not easy to properly design and write the whole thing.

Maybe it's not worth the effort... I probably won't port this module anytime soon.

This module is highly likely to be excluded from evision 0.1.0.

An error occur when compiling on Jetson (Linux, aarch64)

Hi,

I compiled evision on NVIDIA Jetson AGX Xavier (Ubuntu 18.04 Linux Kernel 4.9.201-tegra, aarch64), but the following error occurred:

==> evision
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   117    0   117    0     0    393      0 --:--:-- --:--:-- --:--:--   393
100 89.8M    0 89.8M    0     0  9401k      0 --:--:--  0:00:09 --:--:-- 14.4M
CMake Error: The source directory "/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/BUILD_OPENEXR=ON" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:96: recipe for target '/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt' failed
make: *** [/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

Are Linux and aarch64 not supported?

Evision.DNN.DetectionModel can't setInputParams at YOLOv4

I tried to use YOLOv4 at Evision.DNN.DetectionModel

dog = "yolov4/dog.jpg"
weights = "yolov4/yolov4.weights"
config = "yolov4/yolov4.cfg"
classes = "yolov4/coco.names"
mat = Evision.imread(dog)

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.DetectionModel.detect(mat)

return error

{:error,
 "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/dnn/src/model.cpp:98: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'\n"}

i guess need such as python code

model.setInputParams(1.0, (416, 416), (0.0, 0.0, 0.0), true, false)

but

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams(scale: 1.0, size: {416, 416}, mean: {0, 0, 0}, swapRB: true, crop: false)

** (UndefinedFunctionError) function Evision.DNN.Model.setInputParams/2 is undefined or private. Did you mean:

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams({416, 416})

{:error, "cannot get `cv::dnn::Model` from `self`: mismatched type or invalid resource?"}

How about the following solution?
1 Convert the loaded model to Evision.DNN.Model
2 Enable setInputParams in DetectionModel as well

Evision.DNN.readFromDarknet load and detect are success
but, I wan't to impl post process :-(

Enviroments
evision 0.1.16
elixir 1.14.1
elrang 25.0.1
os macOS 12.6

doubt on image croping in elixir

i haven doubt in Nx.slice we are working on it we converted image to 2d array by using your instructions could you please help us @navrkr @cocoa-xu

how can slice function work on it

we are trying to give all but its not work please help us sir

Originally posted by @Jagan3534 in #102 (comment)

Evision.Nx.to_nx and Evision.Nx.to_mat are corrupting their inputs leading to a crash

Hi,

I am trying to slice a tensor with Nx. I tried:

img = imread(path)
{:ok, new} = Evision.Nx.to_nx(img) |> Evision.Nx.to_mat()
imshow(new)

This should return the original matrix as is, but this crashes with:

terminate called after throwing an instance of 'cv::Exception'
        what():  OpenCV(4.6.0) /home/erts/eYRC-23/Tasks/SolutionFolders/task1b_evision_resource/deps/evision/3rd_party/opencv/opencv-4.6.0/modules/core/src/array.cpp:2494: error: (-206:Bad flag (parameter or structure field)) Unrecognized or unsupported array type in function 'cvGetMat'
zsh: IOT instruction (core dumped)  iex -S mix

What could be the problem here?
Thank you

Evision Show

gray = Evision.imread(image, flags: Evision.cv_IMREAD_GRAYSCALE())
When i tried to convert the image to gray scale it return only reference scale
but not return shape,dimension ?

Make function names more readable/friendly

Function names are not quite readable/friendly as I use lowercases for every function to get it worked ASAP. Now it's time to improve it.

The reason why I use all lowercase is that some C++ function names start with capital letters, some have abbreviations in the names. Some examples,

Approach 1

cv::BFMatcher::BFMatcher()
// will be illegal in Elixir if we make it OpenCV.BFMatcher.BFMatcher/0
// also, it starts with an abbreviation, BF => brute-force
// so we should use 
// OpenCV.BFMatcher.bfMatcher/0

Therefore, we can have the following rules

• For consecutive capital letters, the last one should be kept as is, and the rest of them should be transformed to lowercase, e.g., cv::BOWImgDescriptorExtractor::BOWImgDescriptorExtractor() should be OpenCV.BOWImgDescriptorExtractor.bowImgDescriptorExtractor/0
• For any single capital letter, leave it as is unless it is the first one.

good cases:
 BFMatcher => bfMatcher
 R => r

acceptable:
  getkNNSamples => getknnSamples

bad cases:
 BOWKMeansTrainer => bowkMeansTrainer
 getFPS => getfpS
 TrackerGOTURN_Params => trackergoturN_Params

Approach 2

What about using underscore?

Rules:

1. For consecutive capital letters, add an _ right before the last one and then make them lowercase.
2. For a single upper case letter, prefix it with _ (unless there exists an _ before the transformation) and make it lowercase.
3. If the function name consists of only one upper case letter, just make it lowercase.

good cases:
 BFMatcher => bf_matcher
 BOWImgDescriptorExtractor => bow_img_descriptor_extractor
 setPreFilterType => set_pre_filter_type
 R => r
 dnn_Model => dnn_model

acceptable:
 getkNNSamples => getknn_samples

bad cases:
 setROI1 => setro_i1
 getFPS => getfp_s

Approach 3

Just make sure the first letter is lowercase.

good cases:
 R => r
 getkNNSamples => getkNNSamples
 getFPS => getFPS
 TrackerGOTURN_Params => trackerGOTURN_Params

bad cases:
 BFMatcher => bFMatcher
 BOWKMeansTrainer => bOWKMeansTrainer

Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

cv::Mat (and its variants) is the OpenCV's implementation of multi-dimensional array, so technically, it can be used as a backend for the Nx (numerical-elixir).