why after FW3.6.4 I can't call nn.load()

General discussion about topics related to OpenMV.
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

why after FW3.6.4 I can't call nn.load()

Postby xky » Tue Jul 28, 2020 12:50 am

I'm use this to recognize Numbers of 0-9,is there any function or way to do this?
After I Upgrade FW, I can't use.
Attachments
Err.png
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Tue Jul 28, 2020 1:01 am

Hi, all the old CNN stuff has been removed. It doesn't fit anymore on the flash side-by-side with TensorFlow.

Regarding the numbers network... Just use Edge Impulse (https://edgeimpulse.com/) to train your own custom network using picture of numbers. If you need a dataset of numbers see the chars74k dataset. http://www.ee.surrey.ac.uk/CVSSP/demos/chars74k/.

OpenMV IDE features integration in the latest release to upload assets to edge impulse.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Wed Jul 29, 2020 11:23 am

Do you mean I can upload pictures and train model in OpenMV IDE ? How to use it ?
I have MV4 H7, not Plus. Is that enough to run the number TF ?
In my place to upload Pictures that is too slow. I don't know why.
Can you help me to upload and train 0-9 Numbers?
Attached is Pictures.
Last edited by xky on Wed Jul 29, 2020 12:20 pm, edited 1 time in total.
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Wed Jul 29, 2020 11:40 am

Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Wed Jul 29, 2020 12:25 pm

I can't access the address, maybe because I'm in China.
Can you help me train this?
I Attached datasets in Attachments.
Attachments
Numbers.zip
(10.85 MiB) Downloaded 2 times
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Wed Jul 29, 2020 12:44 pm

Yes, let me do that. However, we need to get that unblocked...
Nyamekye,
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Wed Jul 29, 2020 1:16 pm

Hi, you need to convert all the images to .jpg and name in the correct format for our converter. You gave me a bunch of jpg images.

Edge impulse folks say the tool works in China. Not sure why you can't access it.
Nyamekye,
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Wed Jul 29, 2020 5:36 pm

Hey, I uploaded your dataset to train and... it's like impossible. Can't get above 36% accuracy.

So, the point of the Edge Impulse stuff is for you to train on actual data.

Please try to login to Edge Impulse and capture real data from the camera and not synthetic data.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Wed Jul 29, 2020 10:08 pm

I tried, Yes it can access, but only slow.
And I use OpenMV IDE to Export datasets to Edge Impulse, I got SSL error.
(I confirmed Username and Password are correct.)
use API Key is the same error.
Attachments
1.jpg
1.jpg (11.09 KiB) Viewed 96 times
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Wed Jul 29, 2020 10:33 pm

Hi, please try to use the edge impulse uploader scripts. Then, it's documented on their API.

Regarding the number dataset you gave me. I cannot seem to train a net that works for that. Can you instead just capture images of what you plan to have the OpenMV Cam look at and zipup that dataset? Training a model for that should be super simple.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Thu Jul 30, 2020 12:37 am

I upload all JPEG for each number 100 sample.
after training by several ways, the result is not good.
Do you know some way to recognize numbers on LCD screen?
Now I can split all ROI of them, but don't have a good NN.
Attachments
1.jpg
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Thu Jul 30, 2020 10:19 am

Hi, yes, the dataset you gave me I can't seem to train a decent CNN on it as mentioned before.

Regarding the SSL error, you are on windows correct? I'll make a new IDE that retries the upload in http to get past that.

Hi, do you have this jpeg dataset?
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Thu Jul 30, 2020 9:28 pm

Yes Windows
And I have. Attached below. Each number 100 picture, I think is enough.
Wish you can train a good NN in some other way. If you can, Please give me a copy.
Attachments
Num.zip
(5.44 MiB) Downloaded 2 times
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Fri Jul 31, 2020 2:06 pm

Hi, I'll see what I can do...

This data is really hard to train on however. I think it has to do with the way it's presented. Not sure why. Pretty much any CNN I tried didn't work.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Fri Jul 31, 2020 11:18 pm

I trained one, Tested it look good.
but when I put it in OpenMV, it work really bad, why?
Attachments
2.jpg
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Sat Aug 01, 2020 12:00 am

Hi, the network is over fit.

If you look at the data it expects a white image with a black character. So, you need to give it exactly that.

You should binarize each character that you need it.

Also, the classify method will scale to for the character into the CNN window. So, you need to produce squarish looking pictures to feed the classify method.

Basically, given the data set you trained the network on... Feed it only images that look at the same.

It's important the realize that the network is not smart. It does not understand anything about what it's trained on. Don't make the assumption it knows that it's looking at. It will only work as long as you control variables.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Sat Aug 01, 2020 1:29 am

I pose a Picture, it is now how it's running.
Each Big charter have a Rectangle. Above the big Rectangle, the left small picture (SubImg) is what I put into NN, the Right is NN output.
I think it at lest will have some good result, but you see, it is very bad.

out = tf.classify(mobilenet, SubImg, min_scale=1.0, scale_mul=0.8, x_overlap= -1, y_overlap= -1)
OutList = out[0].output()
max_idx = OutList.index(max(OutList))

is here any mistake? I Think the NN will not bad like this.
Attachments
3.jpg
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Sat Aug 01, 2020 9:55 am

Hi, you should use the crop method to make new image from each of those characters. Then, binarize the cropped image, then draw that cropped image in the middle of a white area with plenty of width/height boundaries.

Think about it. What you are feeding the network doesn't look like what it's trained on.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Sat Aug 01, 2020 12:46 pm

I get it down. Still use NN, it works.
the key is lower the filters of 2D/Loop. Numbers don't need too much.
and before put into NN, need to pre-fix the SubImg.
Do you have some suggestion to get speed up?
Attachments
3.jpg
User avatar
kwagyeman
Posts: 4230
Joined: Sun May 24, 2015 2:10 pm

Re: why after FW3.6.4 I can't call nn.load()

Postby kwagyeman » Sat Aug 01, 2020 1:03 pm

Awesome! Mind posting some details on exactly what you did for others?

As for a speed up... We are working on CMSIS-NN support. When it is integrated we should get a 5x speed boost. However, it's challenging because when we use it right now it causes memory corruption... And because the TensorFlow library is so big we can't compile in debug mode and find the error. We've reached out to ARM for help and are working with edge impulse to get his operational.
Nyamekye,
xky
Posts: 10
Joined: Tue Jul 28, 2020 12:39 am

Re: why after FW3.6.4 I can't call nn.load()

Postby xky » Sun Aug 02, 2020 6:14 am

It is as you said, use the crop method to make new image from each of the characters, barbarize the cropped image, then draw that cropped image in the middle of a white area with Same of width/height pixs.

What I feeding the network is exact what it's trained on. Such as trained it use 32x32 sample, then feed it with 32x32 pictures. So before feed, I must pre-fix Each Roi's Picture. That's all.

Wish CMSIS-NN can be used in future. Now it works with 20 characters only 1.3 FPS. Still can't use in real application. Is it true if save it to the CPU as main.py, it can run fast ?

Some bug report: when I min the window and then max the window of IDE, all the sub window get back to the original size, not the custom size. and sometime the menu (such as File), when click it don't display the down sub menu.

I think CMSIS-NN is big, If the IDE can deploy different FW based on users intent, maybe get the memory free.

Return to “General Topics”

Who is online

Users browsing this forum: EJosue15 and 1 guest