carvac/librtprocess

A project to make RawTherapee's processing algorithms more readily available.

librtprocess's Issues

how to get the best result from the API? white balance correction?

Hi CarVac,

As you pointed early in another post, the best demosaicing result comes from white balanced data? How to achieve that using the API?

I found there is already an implementation of white balance application. However, the color (RGB) scales/coefficents needs to be known beforehand. What is the correct way to get the coeffcients?

Best
JVision

Create a milestone

Hello

I would like to suggest that you create a v1.0 milestone and assign issues to it to make it public and clear what needs to be done for the first stable version to get released.

I want to open tickets in the issue trackers of the distros I use to prepare them to create librtprocess packages, but I don't know how far librtprocess is from a release yet.

add -ftree-vectorize

When making a RelWithDebinfo build, some functions get quite slow because -ftree-vectorize is not included in -O2

@cryptomilk What's the cleanest way to add this flag?

IGV crash (non-SSE2)

In line 506 of igv.cc I always get a crash when trying to run the function. This line is code of the non-SS2 IGV version. I compiled on ARM which has no SS2. When changing line 44 to make use of this version on Intel CPUs, the same crash happens.

Line 503-507:
for (int row = 0; row < height - 0; row++)
for (int col = 0, indx = row * width + col; col < width - 0; col++, indx++) {
int c = fc(cfarray, row, col);
rgb[c][indx] = CLIP(rawData[row][col]); //rawData = RT datas
}

On crash, the debugger tells for my test:
c = 2
indx = 484417
row = 0
col = 1
width = 1856
height = 1044

For me it looks like indx is wrong - but why?! I would expect indx = 1 at this time. Any idea?

RCD is non deterministic

Hello, we have noticed a few years ago in Siril that RCD was not always giving the same result from the same input image, with sometimes one or two pixels changing values. Now I was trying to make some photometry work with a debayered image and the unit test don't always give the same results, which becomes a real problem.

It doesn't seem to happen with small images, but my test images can be around 2000 pixels by 1000, 16 bits unsigned, converted to float as usual. Siril disabled OpenMP for librtprocess. I tried replacing mallocs by callocs and initializing arrays on the stack, the result is the same. Valgrind detects an uninitialized value in the result buffer (Conditional jump or move depends on uninitialised value(s)).

The pixels that change value are always 10 rows from the bottom and red, and it was also seen once with a blue pixel 10 columns from the right. Example coordinates of pixels with changing values in a set of 6 images (seen by subtracting two): (1251, 9), (1487, 9), (149, 9). Would you be able to look at this?

Add Capture Sharpening from RT

Just created this issue to not forget about it...

Add MSVC support?

Hi CarVac,
This is a super useful project for people dealing with the raw sensor data. Like a sunshine into the dark world.
would appreciate if this could be easily built under MSVC.

Install to /usr/local/

I cloned librtprocess, compiled and installed it per the instructions, and the cmake-generated Makefile installed the libraries to /usr. Really, they should go to /usr/local, and distro packages should redirect if they so choose to build a dist package.

Build the shared library optionally

There is an option to build the static library, but the shared library is always built now. Would it be possible to deactivate the build of the shared library with an option? I don't need it and it takes a long time to compile. Thank you

Missing 0.12.0 git tag?

Looks like in fedora is librtprocess 0.12.0 with source code state from bc2f53b.
I'm not sure what is going on. Does it mean that some giit tag has been removed?
If not .. do you have any plans to make 0.12.0 release soon?

Add symbol versioning

Hi,

the once you have more or less agreed to an API for the first version, you need symbol versioning.

Use abimap!

You can find cmake files to manage abimap in the libssh project. Let me know if you need help.

Demoisaiced values are out of range

I'm passing an array of values [0, 65535] to vng4_demosaic and I'm getting values between -1632.537354 and 86103.359375 as a result. What's happening, what should I do?

With rcd_demosaic, I'm getting proper values.

Optimize lmmse demosaic speed and memory footprint

Lmmse is one of the main demosaicers for high ISO (noisy) images.

Currently it's also still one of the slow demosaicers and has a quite large memory footprint.

I will try a tiled processing to reduce processing time and memory footprint

Demosaicing crashes

Hi, I've tried to integrate librtprocess in siril, and I encounter segfaults when trying to debayer. I have probably missed something... Here's the temporary code. You cannot easily test it because the makefile modification is not on the repository, because it's hard to automate, but I did it manually.

I tried with rcd and vng4, it segfaults with both. What am I doing wrong?

0xf5295c50 in rcd_demosaic () at /home/vinvin/siril/deps/librtprocess/src/demosaic/rcd.cc:283
283                             blue[row][col] = CLIP(rgb[2][idx] * 65535.f);
(gdb) p idx
$3 = 15631
(gdb) p row
$4 = 269
(gdb) p col
$5 = 9

Thread 23 "siril" received signal SIGSEGV, Segmentation fault
librtprocess::interpolate_row_redblue (rawData=0x14d7a78, cfarray=0xfffee2b4, ar=0xe18c95e8, ab=0xe6653c88, pg=0xe3f8ac30, cg=0xe3f8e938, ng=0xe3f92640, i=828, width=3906) at /home/vinvin/siril/deps/librtprocess/src/include/bayerhelper.h:76
76                  ab[j] = cg[j] + rb * 0.25f;
(gdb) p cg[j]
$1 = 291.828125
(gdb) p j
$2 = 222
(gdb) p rb
$3 = -34.6629333
(gdb) bt
#0  0xf5284644 in librtprocess::interpolate_row_redblue(float const* const*, unsigned int const (*) [2], float*, float*, float const*, float const*, float const*, int, int) (rawData=0x14d7a78, cfarray=0xfffee2b4, ar=0xe18c95e8, ab=0xe6653c88, pg=0xe3f8ac30, cg=0xe3f8e938, ng=0xe3f92640, i=828, width=3906)
    at /home/vinvin/siril/deps/librtprocess/src/include/bayerhelper.h:76
#1  0xf529b8c8 in _Z13vng4_demosaiciiPKPKfPPfS4_S4_PA2_KjRKSt8functionIFbdEE._omp_fn.1(void) () at /home/vinvin/siril/deps/librtprocess/src/demosaic/vng4.cc:334

Thanks!

Port new ca correction code from rt as soon as it is in dev branch of rt

Todo for 1.0

librtprocess build is very slow on Windows

Following the build instructions, the librtprocess build is very slow on Windows

Example for Amaze demosaic of a 100 MP file: 11.8 seconds

With the cmake command from attached patch the same file is amaze demosaiced in 1.5 seconds

diff --git a/README.md b/README.md
index 044dd1a..e7e93da 100644
--- a/README.md
+++ b/README.md
@@ -47,7 +47,7 @@ This is version 0.10.0, which furnishes the following routines:
 Build instructions for Windows msys2 environment:
 
 1. Make a subdirectory named `build`, and `cd` to that directory.
-2. Run `cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" ..`
+2. Run `cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" -DCMAKE_BUILD_TYPE="Release" ..`
 3. Run `make`
 4. Run `make install`.
 

Can someone test if it behaves same on Linux or Mac?

Port RT badxixels code to librtprocess

I would like to port some of the RT badpixels code to librtprocess to handle bad pixels processing in HdrMerge. Any objections?

Should RCD scale const be 65535.0?

librtprocess/src/demosaic/rcd.cc

Line 74 in 427e598

constexpr float scale = 65536.f;

Configure number of thread used by the lib

Hello it's me again. I'd like to have a way to configure how many threads librtprocess will use for demosaicing. For the context, I'm integrating it in siril that already parallelizes the processing of several files, so we can't give all threads to demosaicing, otherwise we'd have to only process one at time and wait for I/O.
Thanks

Make C API

Don't crash on failed memors allocation in ca_correct

My last commit had a bug in avoid colour shift. I reverted it and will start again now.

Document the API

Well, we need to document what everything does.

Create automated tests

Smart pointer mistake can be fixed

Please, refer LuminanceHDR/LuminanceHDR#199

Build error with gcc 11

Hello, it seems that the new compiler on Arch Linux has problems with size_t in librtprocess headers:

/home/j/.cache/yay/siril/src/siril-0.99.8.1/subprojects/librtprocess/src/include/librtprocess.h:30:313: error: ‘size_t’ has not been declared
   30 | rpError amaze_demosaic(int raw_width, int raw_height, int winx, int winy, int winw, int winh, const float * const *rawData, float **red, float **green, float **blue, const unsigned cfarray[2][2], const std::function<bool(double)> &setProgCancel, double initGain, int border, float inputScale, float outputScale, size_t chunkSize = 2, bool measure = false);
      |                                                                                                                                                                                                                                                                                                                         ^~~~~~

I don't have access to this, it was reported on siril as a build issue here: https://aur.archlinux.org/packages/siril/

Maybe there is an include required for size_t now, apparently it's in stddef.h.

clang error with hpdh.cc

I'm getting an argument # mismatch in apple clang compilation with the calls to hphd_vertical and hphd_horizontal.

I can compile with these changes:

diff --git a/src/demosaic/hphd.cc b/src/demosaic/hphd.cc
index 68890db..92c9e8f 100644
--- a/src/demosaic/hphd.cc
+++ b/src/demosaic/hphd.cc
@@ -349,7 +349,7 @@ rpError hphd_demosaic(int width, int height, const float * const *rawData, float
         }
     }
 #else
-    rc = hphd_vertical(hpmap, 0, W, H);
+    rc = hphd_vertical(rawData, hpmap, 0, W, H);
 #endif
 
     if (!rc) {
@@ -376,7 +376,7 @@ rpError hphd_demosaic(int width, int height, const float * const *rawData, float
             }
         }
 #else
-        rc = hphd_horizontal(hpmap, 0, H);
+        rc = hphd_horizontal(rawData, hpmap, 0, 0, H);
 #endif
         if (!rc) {
             setProgCancel(0.43);

how to use the demosaic algorithms in the library?

Hi CarVac,

I was trying to write a small test program to see how this raw processor works. However, after fumbling around for a few hours, I could not get the correct result. I must have done something wrong somewhere. Is there any guidance in this usage except the readme?

Here's what I did:
step 1: filling the cfa pattern array:
` unsigned cfa[2][2];

if (bayer_pattern == RGGB) {
cfa[0][0] = 0, cfa[0][1] = 1, cfa[1][0] = 1, cfa[1][1] = 2;
}else if (bayer_pattern == BGGR){
cfa[0][0] = 2, cfa[0][1] = 1, cfa[1][0] = 1, cfa[1][1] = 0;
}else if (bayer_pattern == GRBG) {
cfa[0][0] = 1, cfa[0][1] = 0, cfa[1][0] = 2, cfa[1][1] = 1;
};`

step 2: filling raw data
` int pixelsize = width * height;
/* AmAZeMEmE wants an image as floating points and 2d array as well */
const float ** restrict imagefloat2d = (const float **)malloc(height * sizeof(float *));
float * red1d = (float *)malloc(pixelsize * sizeof(float));
float ** restrict red2d = (float **)malloc(height * sizeof(float *));
float * green1d = (float *)malloc(pixelsize * sizeof(float));
float ** restrict green2d = (float **)malloc(height * sizeof(float *));
float * blue1d = (float *)malloc(pixelsize * sizeof(float));
float ** restrict blue2d = (float **)malloc(height * sizeof(float *));

/* AmAZe also wants to return floats, so heres memeory 4 it */
#ifdef _OPENMP
    #pragma omp parallel for
#endif
for (int y = 0; y < height; ++y) { 
    imagefloat2d[y] = (const float *)(src + (y*width)); 
    red2d[y] = (float *)(red1d + (y*width));
    green2d[y] = (float *)(green1d + (y*width));
    blue2d[y] = (float *)(blue1d + (y*width));
};
/* to do:
    use multithreading to solve 
*/
{
    /* run the Amaze */
    amaze_demosaic(width, height, 0, 0, width, height, imagefloat2d, red2d, green2d, blue2d, cfa, callback, 1.0, 0, 1.0f, 1.0f);
    //rcd_demosaic(width, height,imagefloat2d, red2d, green2d, blue2d, cfa, callback);
    
};
/* Giv back as RGB, not separate channels */
for (int i = 0; i < pixelsize; i++)
{
    int j = i * 3;
    dst[j]      = clip<uint32_t, T>((uint32_t)red1d[i]);
    dst[j + 1]  = clip<uint32_t, T>((uint32_t)green1d[i]);
    dst[j + 2]  = clip<uint32_t, T>((uint32_t)blue1d[i]);
}

`

Or even better, if there any document somewhere?

Regards
JVision

don't duplicate

You might want to have a look at that project and not duplicate it: https://github.com/halide/Halide

The main problem I see with your project (although it's nice to mutualize shared code) is the lack of GPU support.

mac clang conversion error

Compiling on mac with clang++-mp-3.9.1

/Users/tune/librtprocess/src/include/boxblur.h:40:12: error: no viable conversion from 'JaggedArray<float>' to 'float *'
    float* temp = buffer;
           ^      ~~~~~~

I tried:
float* temp = *buffer;
which does compile.

Port color propagation from RT

@heckflosse

State of Libre Graphics 2020

The Libre Graphics Meeting 2020 is happening at the end of May in Rennes!

As usual, the first conference will be the State of Libre Graphics, where we will give an overview of what has happened during the last year in the world of Libre Graphics.

You're very welcome to submit two slides for your project. They will be presented by the moderator of the State of Libre Graphics.
Instructions for submitting the slides can be found in the Github repository that will contain all slides:

https://github.com/libregraphicsmeeting/state-of-lg-2020

If you have further questions, don't hesitate to open a ticket in the same repository.

And, of course, you're very welcome to join us in Rennes!