Live streaming?

Discussion related to "under the hood" OpenMV topics.
jubey24
Posts: 1
Joined: Fri Sep 04, 2015 4:17 pm

Live streaming?

Postby jubey24 » Fri Sep 04, 2015 4:21 pm

Among the other capabilities of detecting objects live streaming would be a very interesting thing. Is this featured? What acessories would be necessary for that?
Thanks!
User avatar
iabdalkader
Posts: 1214
Joined: Sun May 24, 2015 3:53 pm

Re: Live streaming?

Postby iabdalkader » Fri Sep 04, 2015 4:43 pm

Just the WiFi shield. Here's a demo using the old camera and CC3000:

https://www.youtube.com/watch?v=sae9jQ2RidI

Note: we're prototyping a better WiFi module, hopefully it will be ready soon.
uolli08
Posts: 1
Joined: Sun Nov 22, 2015 11:49 am

Re: Live streaming?

Postby uolli08 » Sun Nov 22, 2015 11:51 am

It is wonderful! BTW, what is the App of HTC phone used for that demo?

Thanks
User avatar
iabdalkader
Posts: 1214
Joined: Sun May 24, 2015 3:53 pm

Re: Live streaming?

Postby iabdalkader » Mon Nov 23, 2015 3:28 pm

uolli08 wrote:It is wonderful! BTW, what is the App of HTC phone used for that demo?

Thanks
I can't remember :) search for "mjpeg" in the app store.
Beyonlo
Posts: 2
Joined: Sat Jan 16, 2016 7:07 pm

Re: Live streaming?

Postby Beyonlo » Sat Jan 16, 2016 9:44 pm

It support just mjpeg? If not, what others encode supported? Mpeg4, h264 and so on.

Thanks
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Live streaming?

Postby kwagyeman » Mon Jan 18, 2016 9:48 am

Live streaming is just JPEG compressed images. We don't have any fancy hardware encoder. h264 is usually implemented via a custom co-processor. You really only get that feature on super high end stuff.
Nyamekye,
rokko
Posts: 7
Joined: Mon Nov 09, 2015 6:17 pm

Re: Live streaming?

Postby rokko » Tue Feb 09, 2016 6:31 am

How long does it take to compress 320x240 color image in JPEG? Im using encoder that encodes for 85ms YCbCr->JPEG and 105 ms RGB565->YCbCr->JPEG, its only 10fps.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Live streaming?

Postby kwagyeman » Tue Feb 09, 2016 1:08 pm

Ibrahim was able to get 30 FPS last night using a line processing algorithm JPEG codec. However, I'm not sure how stable these results are. We'll let you know once we've tried everything. The OV2640 will have higher frame rates than the OV7725 camera. But, the OV7725 has better quality right now. We have to manually compress images on the OV7725 camera.

Reality - I understand people are not going to be happy with the low res, that said, for security purposes you don't nee more than 10 FPS. If you're really interested in high quality video we expect you'll have a second camera triggered by the OpenMV Cam. But, for recording purposes 320x240 at +10 FPS is pretty acceptable. You can see what's going on more or less.

Note that I'm talking about saving images to the SD card. I have not clue how things will turn out with an internet connection.
Nyamekye,
rokko
Posts: 7
Joined: Mon Nov 09, 2015 6:17 pm

Re: Live streaming?

Postby rokko » Tue Feb 09, 2016 3:24 pm

Im using OV7720 (VGA/QVGA) connected to STM32F429I-DISCO via DCMI. Jpeg encoder was taken from here https://github.com/mitchd/vt-fox-1/tree ... de/encoder and it works, but slowly. Thinking about libjpeg or encoder from openmv, but libjpeg throws errors in linking and openmv saves strange jpegs as attached.
Attachments
IMAGE320.JPG
320x240
IMAGE320.JPG (27.9 KiB) Viewed 9909 times
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Live streaming?

Postby kwagyeman » Tue Feb 09, 2016 5:55 pm

Yeah, we can't really support not our hardware. But... the weird image is probably because we have to store the RG565 data byte reversed in RAM. Who knows... All that code has been really highly optimized by Ibrahim.
Nyamekye,
User avatar
iabdalkader
Posts: 1214
Joined: Sun May 24, 2015 3:53 pm

Re: Live streaming?

Postby iabdalkader » Tue Feb 09, 2016 6:37 pm

The current JPEG encoder can compress QVGA in ~70ms.
If you're having problems using it, it's probably parameters passed to the jpeg_compress (the destination buffer size should be set in dst->bpp it needs to be that way in our code), also there's an RGB565 to YUV lookup it's inverted like Kwabena mentioned, the RGB565 are assumed to be inverted (they're read LSB first), that shouldn't be a problem for you though.
rokko
Posts: 7
Joined: Mon Nov 09, 2015 6:17 pm

Re: Live streaming?

Postby rokko » Thu Feb 11, 2016 7:04 am

Im writing directly to SDRAM, so I changed jpeg_put functions:

Code: Select all

static void jpeg_put_char(jpeg_buf_t *jpeg_buf, char c)
{
    jpeg_buf->buf[jpeg_buf->idx++]=c;
}

static void jpeg_put_bytes(jpeg_buf_t *jpeg_buf, const void *data, int size)
{
    memcpy(jpeg_buf->buf+jpeg_buf->idx, data, size);
    jpeg_buf->idx += size;
}
And calling jpeg_compress from main:

Code: Select all

#define SDRAM_BANK_ADDR	((uint32_t)(0xD0000000))

image_t src = {.w=IMG_WIDTH, .h=IMG_HEIGHT, .bpp=2, .data=(uint8_t*)SDRAM_BANK_ADDR};
image_t dst = {.w=IMG_WIDTH, .h=IMG_HEIGHT, .bpp=0, .data=((uint8_t*)SDRAM_BANK_ADDR + (IMG_WIDTH * IMG_HEIGHT * 2))};

jpeg_compress(&src, &dst, 90);
jpeg_size = dst.bpp;
But it hangs in a while cycle inside jpeg_calcBits:

Code: Select all

static void jpeg_calcBits(int val, uint16_t bits[2]) {
    int t1 = val < 0 ? -val : val;
    val = val < 0 ? val-1 : val;
    bits[1] = 1;
    while(t1 >>= 1) {
        ++bits[1];
    }
    bits[0] = val & ((1<<bits[1])-1);
}
When I change (t1 >>= 1) to ((t1 >>= 1) > 0) it makes corrupted jpeg.
User avatar
kwagyeman
Posts: 4430
Joined: Sun May 24, 2015 2:10 pm

Re: Live streaming?

Postby kwagyeman » Thu Feb 11, 2016 3:27 pm

Looks like the int value should be an uint. Think about what would happen if t1 was the max negative value... right shift would copy the upper bits forever. The max negative 32 bit value can't be ABSed like in the code. Anyway, that problem seems unlikely.

Sorry, I can't help you more than that. We've got our own serious bugs over here.
Nyamekye,
rokko
Posts: 7
Joined: Mon Nov 09, 2015 6:17 pm

Re: Live streaming?

Postby rokko » Fri Feb 12, 2016 4:25 am

Anyway thanks for answers, I'll try to make it work.
User avatar
iabdalkader
Posts: 1214
Joined: Sun May 24, 2015 3:53 pm

Re: Live streaming?

Postby iabdalkader » Wed Apr 13, 2016 10:57 am

jubey24 wrote:Among the other capabilities of detecting objects live streaming would be a very interesting thing. Is this featured? What acessories would be necessary for that?
Thanks!
Hi, initial support for WiFi streaming has been implemented in the latest FW/IDE release. We're using the new Atmel WINC1500 WiFi module, it's a really impressive module. Here's a demo of WiFi streaming FIR images:
Image

The WiFi module is available for pre-order:
https://openmv.myshopify.com/collection ... ifi-shield

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 5 guests