The image De-bayered from RAW-RGB format has noise on object edge.

Discussion related to "under the hood" OpenMV topics.
iiifr
Posts: 3
Joined: Tue Dec 17, 2019 4:39 am

The image De-bayered from RAW-RGB format has noise on object edge.

Postby iiifr » Tue Dec 17, 2019 7:20 am

Hello everyone,

I used a F767 Nucleo board and a OV7725 module connecting with jump wires to do my research, but the quality of image was awful, too much interference, so I switch to OpenMV3 M7 board now.
Instead of re-writing code in micropython, I prefer to continue writing my undone C code, so I reference the OpenMV schematic and use CubeMX to re-create my project, and compiling the baremetal firmware myself.

My research need the pure image data without doing color calibration and image compression, so I let OV7725 send RAW RGB data back, and I de-bayer them myself, to see if the captured image is OK.
The image quality is better than before, but the image has noise between different color, like the picture below.
Image
Here is the same scene taken by my phone.
Image
I loaded stock firmware back to OpenMV and used OpenMV IDE to check captured image, found out the image was fine.
In order to solving this problem, I checked the source code of OpenMV firwmware, and tried to make OV7725 configuration of my own project be close to OV7725 configuration of OpenMV. No luck happened.

I start to think maybe it's my de-bayer function going wrong, so I load OpenMV stock firmware back, and run the following python code on OpenMV:

Code: Select all

import sensor, image, time, pyb

sensor.reset() 
sensor.set_pixformat(sensor.BAYER)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)
clock = time.clock() 

img = sensor.snapshot()
print('')
for i in range(img.height()):
    a = []
    for j in range(img.width()):
        a.append(img.get_pixel(j, i))
    print(a)
print("**END**")
while(True):
    pass
Then I manually copy the printed RAW RGB data as input of my de-bayer function.
The function I already compare it with the macro COLOR_BAYER_TO_RGB565 in the file omv/img/imlib.h of source code, and did some modification, they are almost the same.

Code: Select all

    void bayer2rgb(char *imgbuf, char *rawbuf, int nr, int nc)
    {
        int i, j, R, G, B, index;

        for (i = 0; i<nr; i++)
        {
            for (j = 0; j<nc; j++)
            {
#define IDX(r,c) ((r)*nc+(c))
                if (i % 2 == 0 && j % 2 == 0)
                {
                    R = (rawbuf[IDX(i - 1, j - 1)] + rawbuf[IDX(i - 1, j + 1)] + rawbuf[IDX(i + 1, j - 1)] + rawbuf[IDX(i + 1, j + 1)]) / 4;
                    G = (rawbuf[IDX(i, j - 1)] + rawbuf[IDX(i, j + 1)] + rawbuf[IDX(i - 1, j)] + rawbuf[IDX(i + 1, j)]) / 4;
                    B = rawbuf[IDX(i, j)];
                }
                else if (i % 2 == 1 && j % 2 == 1)
                {
                    R = rawbuf[IDX(i, j)];
                    G = (rawbuf[IDX(i, j - 1)] + rawbuf[IDX(i, j + 1)] + rawbuf[IDX(i - 1, j)] + rawbuf[IDX(i + 1, j)]) / 4;
                    B = (rawbuf[IDX(i - 1, j - 1)] + rawbuf[IDX(i - 1, j + 1)] + rawbuf[IDX(i + 1, j - 1)] + rawbuf[IDX(i + 1, j + 1)]) / 4;
                    
                }
                else if (i % 2 == 1 && j % 2 == 0)
                {
                    R = (rawbuf[IDX(i, j - 1)] + rawbuf[IDX(i, j + 1)]) / 2;
                    G = rawbuf[IDX(i, j)];
                    B = (rawbuf[IDX(i - 1, j)] + rawbuf[IDX(i + 1, j)]) / 2;
                }
                else if (i % 2 == 0 && j % 2 == 1)
                {
                    R = (rawbuf[IDX(i - 1, j)] + rawbuf[IDX(i + 1, j)]) / 2;
                    B = (rawbuf[IDX(i, j - 1)] + rawbuf[IDX(i, j + 1)]) / 2;
                    G = rawbuf[IDX(i, j)];
                }
#undef IDX

                index = (i * nc + j) * 3;
                imgbuf[index] = R;
                imgbuf[index + 1] = G;
                imgbuf[index + 2] = B;

            }
        }
    }
The image generated by this de-bayer function also has the same problem.
I think the possible problem maybe is in my de-bayer function, or the OpenMV firmware just like my own firmware, cannot setup properly for RAW RGB format neither.
I can't find out possible solution furthermore, so I hope I can get some help here, thanks.
User avatar
kwagyeman
Posts: 3837
Joined: Sun May 24, 2015 2:10 pm

Re: The image De-bayered from RAW-RGB format has noise on object edge.

Postby kwagyeman » Tue Dec 17, 2019 12:36 pm

I understand you need help on this... but, you're essentially asking us to debug code unrelated to our product... we don't provide help support for that...

Given our debayer method is open-source... um, why not just start with our code and modify it slightly repeatedly and check that you changes don't break the code? This is typically how you modify a system to fit your needs. I don't quite understand what you are trying to accomplish.
Nyamekye,
iiifr
Posts: 3
Joined: Tue Dec 17, 2019 4:39 am

Re: The image De-bayered from RAW-RGB format has noise on object edge.

Postby iiifr » Tue Dec 17, 2019 10:18 pm

Thanks for your reply, I understand.
I'll keep digging the OpenMV source code and find solution. If it still doesn't work, I'll modify OpenMV source code directly.

However, I have 2 questions need to be confirmed:
1. According to this thread viewtopic.php?f=6&t=1060&hilit=RAW , I guess OpenMV get processed bayer data from OV7725, then de-bayer data, and the image doesn't appear the weird visual effect I met in my situation, am I right?
2. In OpenMV IDE there is a frame window which will show the image captured by OV7725, can you tell me where is this function being implemented in OpenMV source code? I mean, where is the code that get image data from OV7725 and send back to OpenMV IDE ?

Hope you can answer me, thank you very much.
User avatar
kwagyeman
Posts: 3837
Joined: Sun May 24, 2015 2:10 pm

Re: The image De-bayered from RAW-RGB format has noise on object edge.

Postby kwagyeman » Thu Dec 19, 2019 11:31 am

1. Yes, we debayer the image. It's slow, but, it works.
2. There's no one function that sends data to the IDE. See the usbdbg.c file and the framebuffer.c code. It's a complex system to transfer data to the IDE.
Nyamekye,
iiifr
Posts: 3
Joined: Tue Dec 17, 2019 4:39 am

Re: The image De-bayered from RAW-RGB format has noise on object edge.

Postby iiifr » Fri Dec 20, 2019 2:34 am

I finally found bug. The debayer function should not be void bayer2rgb(char *imgbuf, char *rawbuf, int nr, int nc) , it should be void bayer2rgb(unsigned char *imgbuf, unsigned char *rawbuf, int nr, int nc)
Declare (signed) char array, let the function mistake value like 255 as -1.

It was me being stupid, the problem is irrelevant with OpenMV.
Still thanks for your patience reply.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 8 guests