External Camera with SPI bus

Discussion related to "under the hood" OpenMV topics.
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

External Camera with SPI bus

Postby lak4cyut » Sun Feb 17, 2019 10:33 pm

Hi,
This is my first contact with OpenMV, it is so powerful.
But for my purpose that I want using an external camera (maybe with SPI bus) to replace embedded camera.
It means I hope I can control my external camera with OpenMV IDE just like embedded camera (using "sensor" class to control it, and show the frames in framebuffer window).

So far, all I known is I need to modify the firmware, add a new driver to implement all functions in sensor_t, and init my driver in sensor_init (sensor.c).
Is it enough to fit my purpose? Have I missed anything?

Thanks for everyone reply me, and welcome for any suggestions. ^.^
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Sun Feb 17, 2019 10:44 pm

Hi, we've made this easier in our next firmware release. You will now be able to create images and then draw on them in python and then send these to the IDE. So, you need to edit our C code unless you need the speed. SPI can be done in python too.

If you want to do things in C it's pretty easy to. Sensor.c shows off how to deal with multiple cameras. In fact, on the H7 we have a SPI bus to the camera connector for you to use for this (and an I2C bus there too along with the camera bus).
Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Sun Feb 17, 2019 11:14 pm

Hi, we've made this easier in our next firmware release. You will now be able to create images and then draw on them in python and then send these to the IDE. So, you need to edit our C code unless you need the speed. SPI can be done in python too.
Are you mean I can read image (frame) for my camera (with SPI) and send the image to framebuffer or another speical way? Actually my camera FPS is only.. maybe 10 ~ 30, not so fast.
Would you please get me more information about this? Thanks so much.
If you want to do things in C it's pretty easy to. Sensor.c shows off how to deal with multiple cameras.
Yes, I saw you pick the camera in sensor_init, and the driver implement functions that defined in sensor_t, I was plan to follow that to implement my driver..
In fact, on the H7 we have a SPI bus to the camera connector for you to use for this (and an I2C bus there too along with the camera bus).
I am sorry I not really understand this part, do you mean there is already have the API to access SPI? Do you have any example to show that?
(BTW, my device is OpenMV3 R2 OV7725-M7)

Thanks for your quick reply, it is helpful to me.
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Tue Feb 19, 2019 2:00 am

Hi,
Are you mean I can read image (frame) for my camera (with SPI) and send the image to framebuffer or another speical way? Actually my camera FPS is only.. maybe 10 ~ 30, not so fast.
Would you please get me more information about this? Thanks so much.
So, like:

Code: Select all

img = Image(30, 40, sensor.GRAYSCALE, copy_to_fb=True)
data = spi.read(30 * 40)
for y in range(40):
    for x in range(30):
        img.set_pixel(data[(y*30)+x])
If you put code like that in a loop with our latest firmware (not yet released however) then you can manipulate the image in python. That said, python is kinda slow. So, a large image will not work fast. You have to go to C then.
Yes, I saw you pick the camera in sensor_init, and the driver implement functions that defined in sensor_t, I was plan to follow that to implement my driver..
That's the proper way in C.

Code: Select all

I am sorry I not really understand this part, do you mean there is already have the API to access SPI? Do you have any example to show that? 
(BTW, my device is OpenMV3 R2 OV7725-M7)
No, the hardware is just there. But, you have to write C code for whatever you want to do.

Note that the OpenMV Cam is quite a nice to use ARM platform. It has SWD support and can be programmed via USB using OpenMV IDE. With my dev environment re-flashing is simple.
Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Thu Feb 21, 2019 2:31 am

So, like:

Code: Select all

img = Image(30, 40, sensor.GRAYSCALE, copy_to_fb=True)
data = spi.read(30 * 40)
for y in range(40):
    for x in range(30):
        img.set_pixel(data[(y*30)+x])
Wow, it looks great!!
I saw the spi API is already prepared, I guess the unfinished part is to flush Image's data to framebuffer, is it?

Code: Select all

img = Image(30, 40, sensor.GRAYSCALE, copy_to_fb=True)
==========================================================================
No, the hardware is just there. But, you have to write C code for whatever you want to do.
I think I can reference sdcard_spi.c to study how to use SPI API, do you have any better reference document/source code/suggestion?

Code: Select all

static BYTE spi_send(BYTE out)
...
static bool spi_send_buff(const BYTE *buff, uint32_t size)
...
static bool spi_recv_buff(BYTE *buff, uint32_t size)
...
Thank you!!
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Thu Feb 21, 2019 9:50 pm

Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Thu Feb 21, 2019 10:51 pm

kwagyeman wrote: Um, just use the PYB SPI module: http://docs.micropython.org/en/v1.9.3/p ... b.SPI.html
Ya, I have checked it before, when your new firmware is release, I will use it first.

But I should still write a C driver because my camera maybe support to FHD.
So I just like to make sure what my understand is correct, the main SPI APIs in C are as follows,

Code: Select all

HAL_SPI_Init()
HAL_SPI_Receive()
HAL_SPI_TransmitReceive()
HAL_SPI_Transmit()
Where can I find the document/references for those APIs?
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Fri Feb 22, 2019 12:29 am

You have to read this:

https://github.com/openmv/micropython/b ... tm32/spi.c

As for the STM32 HAL. They put the documentation in the c file. Not the h file. So, read the HAL C file.

https://github.com/micropython/stm32lib ... _hal_spi.c
Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Fri Feb 22, 2019 3:42 am

kwagyeman wrote: You have to read this:

https://github.com/openmv/micropython/b ... tm32/spi.c

As for the STM32 HAL. They put the documentation in the c file. Not the h file. So, read the HAL C file.

https://github.com/micropython/stm32lib ... _hal_spi.c
Got it, thank you.
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Sun Mar 10, 2019 10:30 pm

UPDATE-
I found out MICROPY_BEGIN_ATOMIC_SECTION() maybe the key point of this behavior.
If I remove MICROPY_BEGIN_ATOMIC_SECTION() & MICROPY_END_ATOMIC_SECTION, my code is not block anymore.
But HAL_SPI_Transmit() return HAL_TIMEOUT.....

But I don't know why it is happen. I just follow the procedure of spi.c, why does it blocking here?
Would you please point out what I am wrong or miss? Thank you.

===========================================================
I make 2 test and use the oscilloscope to observe its behavior.
But one of them is failed, when I send data with SPI API, it seems block.
Do you have any suggestion about how to debug in this situation or any idea about it?

Below is the detail. Thanks.

Test 1, control SPI and CS pin (with GPIO) in Python.
It works great.

Test 2, control SPI and CS pin in C.
I use HAL_GPIO_Init & HAL_GPIO_WritePin to control CS pin, it works.
And I init SPI with HAL_SPI_Init, it return HAL_OK, but when I send the data with HAL_SPI_Transmit, it seems block (Not crash, just like block in that API, OpenMV IDE show the device is busy.).

Here is my code, and I put the code in ov7725.c get_gain_db() to let me can easy call it with current API sensor.get_gain_db() .. (you known, just a test...)
One more strange thing, because my log "ooxx" is not show in terminal, that is why I thought it is block in the API...

Code: Select all

    SPIHandle.Instance               = SPI2;
    SPIHandle.Init.Mode              = SPI_MODE_MASTER;
    SPIHandle.Init.Direction         = SPI_DIRECTION_2LINES;
    SPIHandle.Init.DataSize          = SPI_DATASIZE_8BIT;
    SPIHandle.Init.CLKPolarity       = SPI_POLARITY_HIGH;
    SPIHandle.Init.CLKPhase          = SPI_CR1_CPHA;
    SPIHandle.Init.NSS               = SPI_NSS_SOFT;
    SPIHandle.Init.BaudRatePrescaler = SPI_BAUDRATEPRESCALER_16;
    SPIHandle.Init.FirstBit          = SPI_FIRSTBIT_MSB;
    SPIHandle.Init.TIMode            = SPI_TIMODE_DISABLED;
    SPIHandle.Init.CRCCalculation    = SPI_CRCCALCULATION_DISABLED;
    SPIHandle.Init.CRCPolynomial     = 0;

    if (HAL_SPI_Init(&SPIHandle) != HAL_OK) 
    {
        /* Initialization Error */
        printf("HAL_SPI_Init() failed.");
        BREAK();
    }
    
    GPIO_InitStructure.Pin = CS_PIN;
    GPIO_InitStructure.Mode = GPIO_MODE_OUTPUT_PP;
    GPIO_InitStructure.Pull = GPIO_PULLUP;
    GPIO_InitStructure.Speed = GPIO_SPEED_FREQ_VERY_HIGH;
    HAL_GPIO_Init(CS_PORT, &GPIO_InitStructure);

    W_CS_HIGH();

Code: Select all

    [b]mp_uint_t atomic_state = MICROPY_BEGIN_ATOMIC_SECTION();[/b]
    W_CS_LOW();
    printf("ooxx\n");
    bool res = (HAL_SPI_Transmit(
        &SPIHandle, data, 1, SPI_TIMEOUT) == HAL_OK);
    W_CS_HIGH();
    [b]MICROPY_END_ATOMIC_SECTION(atomic_state);[/b]
    return res;
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Wed Mar 13, 2019 9:51 pm

Well, I found the root cause...
I thought that SPI GPIO will be configured after call HAL_SPI_Init(), but when I checked its detail, I found out that HAL_SPI_MspInit is not implemented.
So I need to manually configure the GPIO.

Below is my initial code, after that, my SPI2 controls are work fine.
(Post here because I think that may help some people..)

Code: Select all

    // Init GPIO for SPI2.
    GPIO_InitTypeDef GPIO_InitStructure;

    __HAL_RCC_SPI2_CLK_ENABLE();

    GPIO_InitStructure.Pin = MISO_PIN | MOSI_PIN | SCLK_PIN;
    GPIO_InitStructure.Pull  = GPIO_PULLUP;
    GPIO_InitStructure.Speed = GPIO_SPEED_HIGH;
    GPIO_InitStructure.Mode  = GPIO_MODE_AF_PP;
    GPIO_InitStructure.Alternate = GPIO_AF5_SPI2;
    HAL_GPIO_Init(SPI_PORT, &GPIO_InitStructure);

    // Init SPI
    memset(&SPIHandle, 0, sizeof(SPIHandle));
    SPIHandle.Instance               = SPI2;
    SPIHandle.Init.Mode              = SPI_MODE_MASTER;
    SPIHandle.Init.Direction         = SPI_DIRECTION_2LINES;
    SPIHandle.Init.DataSize          = SPI_DATASIZE_8BIT;
    SPIHandle.Init.CLKPolarity       = SPI_POLARITY_HIGH;
    SPIHandle.Init.CLKPhase          = SPI_CR1_CPHA;
    SPIHandle.Init.NSS               = SPI_NSS_SOFT;
    SPIHandle.Init.BaudRatePrescaler = SPI_BAUDRATEPRESCALER_2;
    SPIHandle.Init.FirstBit          = SPI_FIRSTBIT_MSB;
    SPIHandle.Init.TIMode            = SPI_TIMODE_DISABLED;
    SPIHandle.Init.CRCCalculation    = SPI_CRCCALCULATION_DISABLED;
    SPIHandle.Init.CRCPolynomial     = 0;
    if (HAL_SPI_Init(&SPIHandle) != HAL_OK) 
    {
        /* Initialization Error */
        printf("HAL_SPI_Init() failed.\n");
        BREAK();
    }

    // Init GPIO for SPI SS(CS) pin, software control.
    GPIO_InitStructure.Pin = CS_PIN;
    GPIO_InitStructure.Mode = GPIO_MODE_OUTPUT_PP;
    GPIO_InitStructure.Pull = GPIO_PULLUP;
    GPIO_InitStructure.Speed = GPIO_SPEED_FREQ_VERY_HIGH;
    HAL_GPIO_Init(CS_PORT, &GPIO_InitStructure);
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Thu Mar 14, 2019 11:54 am

Hi, thanks for posting!
Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Fri Apr 12, 2019 2:49 am

Hi kwagyeman and everyone,
I got a good news, it is my SPI camera is workable.... but bad news is, its performance is terrible.... (sad)

So, I plan to speed my SPI r/w with SPI-DMA.
My question is can I implement it directly without any HW change? (via SPI2)

If the answer is yes, where can I find the example or reference to study how to do it?
I have checked the lepton.c, but it seems base on stm32H7 not F7...(there are couple APIs different..)

If the answer is no, how can I do to let DMA R/W SPI2?

Thanks for the any help...
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Fri Apr 12, 2019 12:13 pm

Hi, the default MP spi read method uses DMA to suck the data in. It's quite fast. What's the SPI camera protocol that it requires a lot of interaction? Generally, the data transfer should move quite quickly. The FLIR Lepton was very complicated because the VOSPI protocol is complex. Otherwise, it should move fast.
Nyamekye,
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Mon Apr 15, 2019 2:17 am

What's the SPI camera protocol that it requires a lot of interaction?
I am not R/W SPI via MicroPython.
I implement a new driver to control my camera with SPI, replace ovxxxx.c, and in my current implementation, I does not use DMA, so I just want to improve it.


And thank you, I reference spi.c to study how to R/W SPI with DMA. But there is a little part that I don't really understand what it means.
In spi.c, I found a condition '!DMA_BUFFER(src/dst)', is it means I need allocate the buffer with a special API? Otherwise how can I make sure my buffer is end by 000011b?

Code: Select all

    #define DMA_BUFFER(p)       ((uint32_t)p & 3)
     ...
     
        if (len == 1 || query_irq() == IRQ_STATE_DISABLED || !DMA_BUFFER(src)) {
	    // Read data directly
        } else {
            // Read data with DMA
User avatar
iabdalkader
Posts: 1141
Joined: Sun May 24, 2015 3:53 pm

Re: External Camera with SPI bus

Postby iabdalkader » Mon Apr 15, 2019 12:08 pm

lak4cyut wrote:
Mon Apr 15, 2019 2:17 am
In spi.c, I found a condition '!DMA_BUFFER(src/dst)', is it means I need allocate the buffer with a special API? Otherwise how can I make sure my buffer is end by 000011b?

Code: Select all

    #define DMA_BUFFER(p)       ((uint32_t)p & 3)
     ...
     
        if (len == 1 || query_irq() == IRQ_STATE_DISABLED || !DMA_BUFFER(src)) {
	    // Read data directly
        } else {
            // Read data with DMA
You need to reserve memory in the linker script that's accessible by the DMA. This is different from chip to chip (see the datasheets).

EDIT: Using DMA is faster, but this is probably not the only issue, maybe check the SPI clock.

If you don't mind, which SPI cam you're using ? Link ?
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Mon Apr 22, 2019 1:34 am

Hi, sorry for late reply.
You need to reserve memory in the linker script that's accessible by the DMA.
Would you like to teach me how to set up it (my device is OpenMV3 R2)? I want R/W my SPI sensor with DMA.

And, a little question, according kwagyeman's reply, OpenMV MicroPython SPI module is uses DMA, I think it means there is already reserve a memory for that.
Why can I not just use the that memory? Is it because they will conflict?
If you don't mind, which SPI cam you're using ? Link ?
The sensor is for internal research, not public.
Thanks for your help.
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Mon Apr 22, 2019 1:58 am

After checked, can I use the following DMA configuration to implement my sensor driver?

Code: Select all

const dma_descr_t dma_SPI_2_RX = { DMA1_Stream3, DMA_CHANNEL_0, DMA_PERIPH_TO_MEMORY, dma_id_3,   &dma_init_struct_spi_i2c };
const dma_descr_t dma_SPI_2_TX = { DMA1_Stream4, DMA_CHANNEL_0, DMA_MEMORY_TO_PERIPH, dma_id_4,   &dma_init_struct_spi_i2c };
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Mon Apr 22, 2019 2:25 am

Excuse me, one more question....
I still don't understand 'DMA_BUFFER(src/dst)'. How is it be defined? Why 0011b?

Code: Select all

#define DMA_BUFFER(p)       ((uint32_t)p & 3)
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Tue Apr 30, 2019 2:50 am

Hi iabdalkader & kwagyeman,
Is there exists a bug about "DMA_BUFFER" macro? I think it missing a ! (not).

After a little research, I guess this macro is used for check the align (4 bytes.)
But it will return 0 (false) when target address is aligned on 4 bytes.

And your check condition is below, then means, DMA will only available when address NOT aligned on 4 bytes.
I think that is a little strange. FYI.

Code: Select all

        if (len == 1 || query_irq() == IRQ_STATE_DISABLED || !DMA_BUFFER(src)) {
	    // Read data directly
        } else {
            // Read data with DMA
        }
BTW, I finish my implementation, succeed in read SPI via DMA.
User avatar
iabdalkader
Posts: 1141
Joined: Sun May 24, 2015 3:53 pm

Re: External Camera with SPI bus

Postby iabdalkader » Tue Apr 30, 2019 12:28 pm

There a "!" in the if statement.
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Wed May 01, 2019 10:09 pm

NO!! Please take look!!
Your !DMA_BUFFER(src) is going to run non-DMA SPI APIs!!
This if statement is wrong logic!!

Code: Select all

        if (len == 1 || query_irq() == IRQ_STATE_DISABLED || !DMA_BUFFER(src)) {
	    // !!!!!!!!!!!!!!!!  Read data directly !!!!!!!!!!!!!!!!!
        } else {
            // Read data with DMA  <-- But now the address is not aligned on 4 bytes.
        }
https://github.com/openmv/micropython/b ... spi.c#L455
User avatar
kwagyeman
Posts: 3805
Joined: Sun May 24, 2015 2:10 pm

Re: External Camera with SPI bus

Postby kwagyeman » Thu May 02, 2019 12:54 am

Ibrahim, can you check this carefully.

Thanks,
Nyamekye,
User avatar
iabdalkader
Posts: 1141
Joined: Sun May 24, 2015 3:53 pm

Re: External Camera with SPI bus

Postby iabdalkader » Thu May 02, 2019 10:00 am

I don't see the problem..

This is the DMA_BUFFER macro for the F7:

#define DMA_BUFFER(p) ((uint32_t)p & 3)


This is the if statement:

if (len == 1 || query_irq() == IRQ_STATE_DISABLED || !DMA_BUFFER(dest)) {


If the length of the buffer is 1 byte OR the IRQs are disabled OR it's NOT a DMA buffer (not aligned) then read the data without using the DMA. Otherwise it's more than 1 byte and IRQs are enabled and it's aligned.

What is the problem ?
User avatar
iabdalkader
Posts: 1141
Joined: Sun May 24, 2015 3:53 pm

Re: External Camera with SPI bus

Postby iabdalkader » Thu May 02, 2019 10:12 am

Oh I see the issue now, I actually fixed this bug before for the SDCARD but I forgot to fix it in the SPI code. This is the fixed macros:

Code: Select all

#define ALIGNED_BUFFER(p)     (((uint32_t)p & 3) == 0)

#if defined(STM32H7)
// NOTE: H7 SD DMA can only access AXI SRAM.
#define DMA_BUFFER(p)       ((uint32_t) p >= 0x24000000 && (uint32_t) p < 0x24080000)
#elif defined(STM32F4)
// NOTE: F4 CCM is not accessible by GP-DMA.
#define DMA_BUFFER(p)       ((uint32_t) p > 0x10010000)
#else
// Assume it's DMA'able
#define DMA_BUFFER(p)       (true)
#endif
lak4cyut
Posts: 15
Joined: Fri Feb 15, 2019 12:22 am

Re: External Camera with SPI bus

Postby lak4cyut » Wed May 08, 2019 10:28 pm

iabdalkader wrote: Oh I see the issue now, I actually fixed this bug before for the SDCARD but I forgot to fix it in the SPI code. This is the fixed macros:
^.^ I am happy to know I'm not crazy.

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 3 guests