this implements an integrated cnn denoiser based on Attila Afra's open image denoise architecture. in particular this is compatile with the weights produced by this fork here.. the fork introduces minor differences to the u-net, mainly leaky relu parameter, channel counts, and pixel shuffles on both ends to account for bayer patters. note that this does not work with fuji xtrans or full colour images.
the weights are read from ~/.config/vkdt/data/jddcnn-weights-heavy.dat or,
for the slightly more light-weight variant with only 16 base channels
~/.config/vkdt/data/jddcnn-weights-light.dat.
download links will follow once the training converged.
in the pipeline it replaces denoising and demosaicing.
the input is rggb bayer data in four planes of half resolution, scaled to the
[0,1] range. that is, the input should be after the denoise module (but be
sure to leave the denoising strength at 0). this way, the black borders will be
cropped and the data will be rescaled to the correct range.
the original glsl cooperative matrix convolution code was written by Adrien Vannson.
a word of warning: this module is extremely slow. it can take like a full
second on a 42MP image (RTX 3080 Ti). you might want to directly route out the
result via o-exr (select custom primaries in the export widget to route the
camera rgb along with the image matrix out to exr, or maybe you want to export
after input device transform in the colour module) and then do the rest of
the grading on the exr for faster interaction.
the light version of the net tends to be less sharp, but is much faster and
also avoids some psychedelic halucination problems of the heavy one.
as a sidenote, working with neural networks is pretty exciting.
model use the heavy model (8.6M parameters) or the lighter one with 2M parametersinput a raw rggb bayer pattern image, after the denoise module scaled it to [0,1]output the denoised and demosaiced rgb image