Camera disconnects from IDE when running script (sometimes)

Discussion related to "under the hood" OpenMV topics.
trevor
Posts: 5
Joined: Wed Nov 29, 2017 1:39 pm

Camera disconnects from IDE when running script (sometimes)

Postby trevor » Wed Nov 29, 2017 1:56 pm

Hello,

So I am having an issue with the camera (M7) disconnecting when I try to run a script. I will click the green play button and it will hang for a few seconds, and then disconnect without running the script.

I saw this problem mentioned here: viewtopic.php?t=315
I followed the three steps listed there:
1. green button stays green indefinitely
2. Frame buffer disabled, script runs indefinitely
3. Frame buffer enabled, script runs indefinitely

This is true for all of the example scripts I have tried.

However, when I try to run my own script ( based mostly on example scripts ), is when the problem happens. I have noticed that if the file has been edited in a different text editor (in my case Vim and Sublime) the problem will happen. Furthermore, if I copy/paste code from another one of my scripts the problem happens. Lastly, sharing work with my group mates and implementing it in my own code will also cause this problem. If I remove the copy and pasted code, it WILL run the script, but if I then try to further edit that file within the IDE the script will not run and the camera disconnects. This is causing huge issues since it is making it very difficult to collaborate with my classmates without having this problem occur.

I am running OSX 10.12.3

Thanks in advance for any help!
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Wed Nov 29, 2017 2:31 pm

Hi, there's a work around for this in the mean time that might work for you.

1. Disconnect from the camera.
2. Go to tools - open terminal and open a serial terminal to your OpenMV Cam at 115200 baud.
3. Then press enter and you should get REPL prompt.
4. Edit code in the main window and then hit the green run button in the open Terminal window to send the script to the camera over REPL.
5. To see what the camera sees add: print(img.compressed_for_ide()) at the end of the while loop.

Doing this bypasses our serial debug stack which seems to have an issue on MAC. I'd like to solve this problem but I can determine the exact issue because I can't reproduce it. All my MACs I have work fine. I did write some code to solve what I thought the problem was... But, I don't have a computer that has that same problem to test with.

If you're willing to help me I can release a version of the IDE and the firmware with the fix I think will solve this issue. But, I need you to test and confirm it. The last guy didn't respond after I put the work into fixing this issue to let me know if my fix worked.
Nyamekye,
trevor
Posts: 5
Joined: Wed Nov 29, 2017 1:39 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby trevor » Wed Nov 29, 2017 2:43 pm

Yep, I'd love to help!

I will be able to test the work around you provided in the next hour or so. Just let me know what I need to do to help.
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Wed Nov 29, 2017 3:44 pm

I'll build the test image thingy this weekend for you. The work around should keep you okay until then.
Nyamekye,
trevor
Posts: 5
Joined: Wed Nov 29, 2017 1:39 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby trevor » Wed Nov 29, 2017 5:16 pm

Okay I might need a little clarification on the work around.

My step 2 looks like: open terminal>new terminal>connect to serial port>[select serial port]cu.Blutooth-Incoming-Port>[baud rate] 115200

This opens up what looks like another serial terminal. But from here, when I connect the camera and click the green play button I don't see any option to send the script to the camera over REPL. I think there is something I'm missing here.
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Wed Nov 29, 2017 6:19 pm

The port you are selecting should not be a Bluetooth port. It should say cu.usb something.

Note that the secondary serial terminal overrides the ability for the main IDE to connect. It it connected correctly you should be able to press the enter button in the serial terminal window and get a repl prompt.

Um, see the OpenMV Video channel about the Open Terminal feature to see how to use this.
Nyamekye,
trevor
Posts: 5
Joined: Wed Nov 29, 2017 1:39 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby trevor » Sat Dec 02, 2017 3:46 pm

That was the only port available from the drop down. No usb option. In the mean time, we have been using my teammate's Windows machine which runs the same scripts just fine. I'll check out the video channel in the mean time. Thanks!
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Dec 02, 2017 4:38 pm

Weird... OpenMV IDE shouldn't have been able to find the camera in the first place.
Nyamekye,
User avatar
falko
Posts: 7
Joined: Mon Dec 04, 2017 9:23 am
Location: Germany
Contact:

Re: Camera disconnects from IDE when running script (sometimes)

Postby falko » Mon Dec 04, 2017 9:28 am

I'm also having trouble running a script from the main window on my mac.
The workaround via a serial terminal works fine, so I'm not blocked from continuing my work.
But let me know if I can help resolving this issue. :)
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Mon Dec 04, 2017 10:48 am

Okay, I'm almost done releasing the next version of the IDE. It has the possible bug fix built-in but it will only activate it on firmware with a version of 9.9.9 so I'll build a version of the firmware once I'm done releasing with the possible fix.
Nyamekye,
trevor
Posts: 5
Joined: Wed Nov 29, 2017 1:39 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby trevor » Tue Dec 05, 2017 1:29 pm

Awesome, thanks for your help. Will the new IDE version be on this site or the openmv github?

Also, do you think this is something that could be remedied by updating OSX to the latest version? Or is it more likely a usb port issue?
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Tue Dec 05, 2017 11:47 pm

I honestly don't know what causes this error. I believe it's a hardware issue. As far as I understand the problem comes from the MAC hardware dropping USB bytes since we're not using the USB CDC class in exactly the way it was designed for. I got a hint of the possible problem when a person showed me the USB trace and I saw the camera delivered the bytes to the application but the application did not receive all the data... meaning that bytes were dropped in the kernel.

So, I implemented some MTU learning code in the IDE. On first connect it will learn the max transfer size through the kernel and break up image reads by that. This trick doesn't affect performance too much. (The IDE learns the MTU by doing image reads and checking to see how many bytes made it back... I have to bypass a lot of error checking in the IDE to make this work).

...

Anyway, please give me some time to release the next IDE version and I will then roll a version of the firmware with this fix. Please use the work around in the mean time.
Nyamekye,
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Thu Dec 28, 2017 4:01 pm

Hi, here's a version of the firmware that has the Mac MTU size fix implemented. The OpenMV Cam will send packets of data to the IDE that are of a max MTU size. Since I don't have a computer with the failing issue I can't tell if this fixes the error on my end but I can verify the code is working and breaking packets up. Load the firmware.bin file or openmv.dfu file. Either one will work.
Attachments
openmv.zip
(2.09 MiB) Downloaded 20 times
Nyamekye,
bobparadiso
Posts: 2
Joined: Tue Jan 23, 2018 5:26 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby bobparadiso » Tue Jan 23, 2018 6:16 pm

Hello, I believe I have this same issue. Running on a mac, and adding some characters to a working script (even if commented out) will cause a delay followed by the IDE disconnecting. I tried applying firmware.bin from your zip file by using the Tools->Run Bootloader option in the IDE. It seemed to update and I now have "Firmware Version 9.9.9" as reported at the bottom of the IDE. However it still does the hang disconnect. I've also attached a file I use to reproduce the issue.

Please let me know if there's any update on this issue or anything I can test or do differently.
Thanks,
-Bob
Attachments
helloworld_1.py
large comment at end of file uses enough chars to reproduce issue
(10.75 KiB) Downloaded 13 times
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Tue Jan 23, 2018 8:15 pm

I have no idea what the problem is if it isn't an MTU size issue...

The only way I can debug this is if I have a MAC which has the problem. All MAC computers that I've tried work fine without issue. I understand there's a problem but I don't know how to fix...

Um, if possible, can you install a USB trace analyzer program and send me the packet dump transaction? Yes, I understand this is a lot of work but I don't know else to determine what is wrong.

Alternatively, use Windows or Linux. A linux or windows VM should not have problems.
Nyamekye,
bobparadiso
Posts: 2
Joined: Tue Jan 23, 2018 5:26 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby bobparadiso » Wed Jan 24, 2018 3:02 pm

Thank you very much for your reply. I completely understand.
I'm going to run with the serial terminal workaround for now. If I have time down the line I'll see if I can get you those USB packet dumps.
Thanks,
-Bob
calvinmryan
Posts: 23
Joined: Tue Jan 30, 2018 11:05 am
Location: Boston, MA

Re: Camera disconnects from IDE when running script (sometimes)

Postby calvinmryan » Sat Feb 03, 2018 10:06 pm

@kwagyeman I'm seeing this issue too. I'd be happy to install a USB trace analyzer and get some data for you! Do you have any recommendations for an analyzer? I will also verify tomorrow that the new IDE + FW 9.9.9 that you posted above doesn't actually fix the issue as bobparadiso mentioned.
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Feb 03, 2018 10:19 pm

I found this:

https://aud-ios.com/2017/10/22/usb-moni ... wireshark/.

I just need to see the traffic from OpenMV IDE to the OpenMV Cam. So, start logging before you hit connect. Then hit connect, you should see the logger print some stuff going on, then click the run button, afterwards please send a text dump of the transactions.

...

Maybe try this too?

http://vusb-analyzer.sourceforge.net/
Nyamekye,
calvinmryan
Posts: 23
Joined: Tue Jan 30, 2018 11:05 am
Location: Boston, MA

Re: Camera disconnects from IDE when running script (sometimes)

Postby calvinmryan » Sun Feb 04, 2018 3:06 pm

I found wireshark last night as well. I'm getting data now - expect a message shortly 👍
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sun Feb 04, 2018 5:54 pm

Hi, I uploaded a new version of the IDE with the fix here: http://upload.openmv.io/openmv-ide-mac-1.8.0.dmg

Give it a try.
Nyamekye,
calvinmryan
Posts: 23
Joined: Tue Jan 30, 2018 11:05 am
Location: Boston, MA

Re: Camera disconnects from IDE when running script (sometimes)

Postby calvinmryan » Mon Feb 05, 2018 10:48 am

I tried the ide you just uploaded above and I'm still seeing the disconnection issue with my main.py script. However, the example scripts work as expected and don't cause a disconnection. Can you try to test on your end with the main.py script in the google drive link I sent you to confirm or deny what I'm seeing? In the meantime I'll grab some usb packet traces with the latest ide. If you rev the ide again could you update the version to 1.8.1 or something so I'm sure that I'm running the correct thing? Thanks!
User avatar
aakoch
Posts: 23
Joined: Wed Dec 06, 2017 11:07 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby aakoch » Sat Mar 03, 2018 5:51 pm

I am also having the issue. I'm using the 1.8.1 IDE and 2.7.0 firmware. Earlier in the thread it was said that your fix would look for firmware version 9.9.9. Is this still the case?

I also downloaded and ran Wireshark. I tried to upload the output but the forum doesn't allow the .pcapng extension. The other extensions I can save to are all .pcap. I don't know what those are.

Is there another way to get you the info.

FYI, it seems to break if the script size > 11,009 in size.

I can see my script being sent in the Wireshark packets. But other than that I don't know what to look for.
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Mar 03, 2018 6:01 pm

Just zip the files.

As for the problem.... It's a weird subtle issue with how we're using the serial port. Please upload the files in zip form and I can confirm.

>> FYI, it seems to break if the script size > 11,009 in size.

Yeah, so, the issue is that our USB serial protocol requires serial packets which are delivered 64 bytes at a time to not have two commands in them. However, we're using the VCP protocol to send data which merges byte streams as efficiently as possible. Breaking this requirement.

So, to ensure this in OpenMV IDE we repeatedly flush the serial port until all bytes are sent. On linux and windows this means the bytes made it to the target. On Mac this means they made it to the OS kernel... which is not what we want.

For the latest IDE I put a small 3ms delay on Mac. I can increase that again to make this problem stop happening.

...

A better fix would be to make the USB DBG code better. But, going there is an issue.

...

What OS are you using? I can cut a new IDE for it very quickly.
Nyamekye,
User avatar
aakoch
Posts: 23
Joined: Wed Dec 06, 2017 11:07 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby aakoch » Sat Mar 03, 2018 6:19 pm

Files? Plural? Wireshark just saved it as one file. I'll upload that. If there is something else, let me know.

I'm using macOS High Sierra 10.13.3.
Attachments
wireshark.pcapng.zip
(90.69 KiB) Downloaded 13 times
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Mar 03, 2018 7:39 pm

Yeah, the same issue is still happening. Really need to get this fixed on the camera.

Okay, I'll get a Mac IDE really quick with a fix.
Nyamekye,
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Mar 03, 2018 8:13 pm

http://upload.openmv.io/openmv-ide-mac- ... -1.8.2.dmg

Let me know if the fixed worked. I just increased some command timeouts. Not a great fix... but, it should do the trick.
Nyamekye,
User avatar
aakoch
Posts: 23
Joined: Wed Dec 06, 2017 11:07 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby aakoch » Sat Mar 03, 2018 8:44 pm

Yes! It works! :D Thanks!
User avatar
kwagyeman
Posts: 1736
Joined: Sun May 24, 2015 2:10 pm

Re: Camera disconnects from IDE when running script (sometimes)

Postby kwagyeman » Sat Mar 03, 2018 8:57 pm

Okay, I'll just leave this IDE up here then.

I'll commit these changes.
Nyamekye,

Return to “Technical Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests