OpenMV H7 Plus with MT9V034 -- sensor.set_auto_exposure

redcrow · March 10, 2021, 5:17am

We are moving to the H7 Plus in effort to classify using a neural network and now we are experiencing an issue on 2 units now with the H7 Plus that was not present with the H7. On both cams we have used the same sensor (MT9V034).

We configure the sensor as below and then we are in a loop waiting for an input signal, take a snapshot, analyze the image and repeat.
Every first image is over exposed and then subsequent images are still a little grainy (not consistent).
If there is an extended delay from a trigger (say 5 seconds) then the next image will be over exposed and then subsequent images are not (yet still grainy).

Exact same code on the H7 was outputting crisp images each time.

In an effort to get to the root cause with the H7 Plus I added a call to the main while loop to call the sensor.set_auto_exposure(False, exposure_us=1750)) and the problem of the 1st image over exposed went away and the images were crisp but having other timing issues in terms of calling the set_auto_exposure repeatedly.

It seems like there is an issue with the set_auto_exposure with the H7 plus and the MT9V034 (did not try with other sensors) and would like to try get this resolved.

Any ideas or suggestions would be greatly appreciated.

Not sure if it related to issue #1156

00090 00091

 configure camera...
    sensor.reset()
    sensor.set_pixformat(sensor.GRAYSCALE)
    sensor.set_framesize(sensor.QQVGA) 
    sensor.set_windowing((winx, winy, winw, winh))
    sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, True)
    sensor.set_auto_gain(False)
    sensor.set_auto_exposure(False, exposure_us = 1750)

    # -------------------------------------------------------------------------------------------------
    # Setup the GPIO (P7/P8/P9) Pins
    # -------------------------------------------------------------------------------------------------
    # Setup the Input Pin that receives signal from sensor
    IP_IN = pyb.Pin(pyb.Pin.board.P8, pyb.Pin.IN, pyb.Pin.PULL_UP)
    # Setup the Output Pin that signals the Main Board the "CAM DATA READY/NOT_READY" status
    IP_OUT = pyb.Pin(pyb.Pin.board.P7, pyb.Pin.OUT_OD)

main while loop...
    if triggered 
    snapshot
    analyze

kwagyeman · March 10, 2021, 5:58pm

Hi, I’ll checkout the MT driver on the H7 and H7 Plus since I have a little bit of free time and see if I can solve the issue.

redcrow · March 11, 2021, 7:02am

Appreciate the effort.

kwagyeman · March 14, 2021, 4:30am

Hi, I just looked into this. So, you may wish to turn black level calibration off:

sensor.__write_reg(0x47, 1)

This is an auto method that is still running. We need to add a global control for this for all cameras.

Otherwise, the software is the same for both cameras. The only difference I can think of is that the power rails are different and that the H7P draws more power. It may be the case the rails are wiggling more than normal. I’m not sure why a steady stream of images is easier to handle than a single snapshot. But, it may be the switching reg onboard not being able to handle the impulse response.

If you can scope the power rails going to your MT9V034 chip you’ll probably see the issue. I just looked through the datasheet and I didn’t find anything else that explains this behavior. Just the BLC auto method still running.

Checkout the datasheet otherwise: https://cdn.shopify.com/s/files/1/0803/9211/files/APTINA-MT9V034.PDF?8193313460631929412

I don’t see anything in there that explains this behavior that we haven’t already turned off or that wasn’t already off.

What’s truly weird is that this changes between triggered and non-triggered mode. Given that there’s nothing in the datasheet that says the camera’s behavior changes based on this I can only imagine it’s some type of impulse response issue related to the power rails.

redcrow · March 16, 2021, 6:49am

It is still performing the same way with the entry to the registry ( 0x47 (71) Black Level Calibration Control ). We will work on scoping the rails to see if there is something to notice there. I am little concerned because we just placed a substantial order for the H7 plus models along with the MT9V034. Thanks for your support and we hope to get this worked out. Are the changes made with issue #1156 something we can get and put onto to the board?

kwagyeman · March 16, 2021, 12:25pm

Hi, try out the latest H7 Plus firmware. I re-wrote the MT9V034 driver. There were a lot of changes. To note, the clock on the H7 board you had was very out of spec. It’s pulled in now to 26.66MHz.

firmware.zip (1.1 MB)

The exposure code now too is different. It works correctly now and returns the current exposure in all modes.

The driver was actually quite wrong from AGC and exposure control. I didn’t write it using the Aptina datasheet last time but used the OnSemi one which has a lot less info.

redcrow · March 16, 2021, 8:03pm

Hi, I have not had the chance to look put this on the scope yet. I tried out the latest H7 Plus firmware. I do see the changes with the exposure at least in regards to the initial exposure at 67680 versus ~16000 on previous firmware. The issue with the first image overexposed and subsequent images grainy is still occurring.

IOCTL_TRIGGER_MODE-True_00010 IOCTL_TRIGGER_MODE-True_00011 IOCTL_TRIGGER_MODE-True_00023 IOCTL_TRIGGER_MODE-True_00024

Out of curiosity I changed
sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, True)
to
sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, False)
the result was that the overexposure on the first image and grainy subsequent images went away.

IOCTL_TRIGGER_MODE-False_00010 IOCTL_TRIGGER_MODE-False_00011 IOCTL_TRIGGER_MODE-False_00023 IOCTL_TRIGGER_MODE-False_00024

On another system H7 Plus with MT9V034 with firmware (FW 3.6.9) we made the same change by setting the trigger mode to false and the result was that the overexposure on the first image and grainy subsequent images went away.

We have about 6 more systems with the H7 and the MT9V034 with trigger mode set to true but the images are coming out crisp and no issues with the overexposed first image.

Thanks for the support.

kwagyeman · March 16, 2021, 9:52pm

Maybe it’s a problem with the trigger pin on the H7 plus.

I’ll check this.

Do you have a tear script I can try to easily generate the issue in a non lab environment on the H7 versus the H7 plus? I’ll get this debugged for you ASAP.

redcrow · March 16, 2021, 11:27pm

I modified the trigger_mode example below and was able to repeat the problem. We have a daughter board connected to the H7 Plus and have an external trigger that we wait for (putting this below in regards to pins).

# ------------------------------
# Setup the GPIO (P7/P8/P9) Pins
# ------------------------------
# Setup the Input Pin that receives signal from sensor
IP_IN  = pyb.Pin(pyb.Pin.board.P8, pyb.Pin.IN, pyb.Pin.PULL_UP)

# Setup the Output Pin that signals the Main Board the "CAM DATA READY/NOT_READY" status
IP_OUT = pyb.Pin(pyb.Pin.board.P7, pyb.Pin.OUT_OD)

# Setup the Output Pin that signals an ERROR condition (presently unused)
IP_ERR = pyb.Pin(pyb.Pin.board.P9, pyb.Pin.OUT_OD)

Example script to simulate without external input:

# Global Shutter Triggered Mode Example
import sensor, image, time

sensor.reset()                      # Reset and initialize the sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to GRAYSCALE
sensor.set_framesize(sensor.QQVGA)    # Set frame size to VGA (160x120)

print("Initial exposure == %d" % sensor.get_exposure_us())
sensor.set_auto_gain(False)                         # must be turned off for color tracking
sensor.set_auto_exposure(False, exposure_us = 10000)
print("New exposure == %d" % sensor.get_exposure_us())

#sensor.__write_reg(0x47, 1)

sensor.skip_frames(time = 2000)     # Wait for settings take effect.
clock = time.clock()                # Create a clock object to track the FPS.

sensor.ioctl(sensor.IOCTL_SET_TRIGGERED_MODE, True)

counter = 0
img_counter = 0
MAX = 11
SIMULATE_DELAY_BETWEEN_NEXT_TRIGGER_TIME = 5

# -------------------------------------------------------------------------------------------------
# After this sleep it will over expose when the next image is taken if trigger set to TRUE
# -------------------------------------------------------------------------------------------------
time.sleep(5)

while(True):
    clock.tick()                    # Update the FPS clock.
    img = sensor.snapshot()         # Take a picture and return the image.
    print(clock.fps())              # Note: OpenMV Cam runs about half as fast when connected
                                    # to the IDE. The FPS should increase once disconnected.

    # -------------------------------------------------------------------------------------------------
    # Simulate image analysis processing time
    # -------------------------------------------------------------------------------------------------
    time.sleep(.4)

    img.save("/data_20210316/%05d.jpg" % img_counter)

    print(counter)
    counter += 1
    img_counter +=1
    # -------------------------------------------------------------------------------------------------
    # After this sleep it will over expose when the first image is taken if trigger set to TRUE
    # -------------------------------------------------------------------------------------------------
    if (counter == MAX):
        counter = 0
        print("sleep ", SIMULATE_DELAY_BETWEEN_NEXT_TRIGGER_TIME)
        time.sleep(SIMULATE_DELAY_BETWEEN_NEXT_TRIGGER_TIME)

kwagyeman · March 18, 2021, 3:52am

h7_h7p.zip (25.7 KB)

Hi, see these image buckets. Is this the problem you said happens? It happens on both my H7 and my H7P.

Have you tried changing to triggered mode earlier? This is changing a mode of the camera. So, a corrupted image output is kinda par for the course. Not trying to be dismissive here… but, the camera modules generally output corrupted images when you change major register settings. We usually delay in our firmware for some number of milliseconds to hide these bad images from the user. However, they still come out of the camera.

Note, this is with the latest firmware on the H7 and H7P. I remember there being a PR a while back about fixing the exposure pin on the H7… I wonder if there’s something going on with that?

redcrow · March 22, 2021, 4:54pm

H7_20210322.zip (59.9 KB)

Hi attached are images taken from an H7 with firmware 3.5.0 and an H7 with firmware 3.9.3. Both sets were taken using the test script with trigger mode set to true. In effort to understand why the difference it looks to be a change made in the firmware? We have tried setting trigger mode earlier with the H7 Plus and the same effect is experienced. At this point it appears that just setting trigger mode to false is going to work for us given the consistent crisp images.

kwagyeman · March 22, 2021, 5:12pm

The later firmware has a better camera driver and captures more images. So, it may just be the older firmware dropped the image where the change happens. When we get double buffering working we will capture all images outputted from the camera.