Advanced Iris Tracking Software

I have recently encountered a problem in my attempts to create my project. I’m trying to create a program that uses the iris tracking software to map the users eye gaze. Essentially I’m trying to:
-Find the users gaze
-Mark where it is
-Draw/Display it on the viewed image
Currently I have the basic eye tracking program working but I’m confused as to what the next steps would be. If you could provide any help as to what I would do next, I would be extremely grateful.

Hi, I can’t quite tell what you’d like me to help with. The current eye tracking program tells you where the eye is looking and marks where it is on the image… that said, you have to mount the camera very close to the person’s eyes.

Can you go into more details?

Hi there,
I’m sorry if my first reply didn’t make much sense. What I’m trying to achieve is a program that makes you give the program an image, tracks where you’re looking on that image, and then proceeds to mark where you were looking on the given image. In short, it’s a program that tracks and marks the users gaze at any given point. I guess I’m confused as to what the code would be needed in order to draw the gaze of the user on the given image. You also said that the current program already marks the image with the users gaze. Could perhaps send me the code that does this? If you could help me I would be extremely grateful.

Thanks for the time

The current method draws the position of the persons pulpils given the cameras field of view.

You can to determine where the person is looking though. This is beyond just the OpenMV Cam. So, you want to send the detected pupil positions from the OpenMV Cam out to a serial port. Once you do this… then you need to send that data to the PC. After which, process it and draw on the screen.

So, you’re going to need to some knowledge of geometry to setup a vector transformation to run the x/y position the camera sees into an x/y position on the PC. I guess… start with this. Try mounting the camera in a fixed position and working back the match to transform the coordinates.

Thank you so much for all the help. I have just one last question. How do I send the data out the port, and where(what software) do I send it to? Once again thank you so much for your time and help.

Just use the USB_VCP module or UART module on the camera. There’s documentation online about how to send serial data.

Thank you for your response. However, I guess my next question would be, how do I set these steps up and what would the code look like? I have the camera sending me coordinates of only the eye right now but I’m rather perplexed as to what would the next steps look like. Could you please clarify this in more detail for me.
Thanks again for all the help :slight_smile:

I have a script for this on my desktop… Will post later tonight. Also, this question has been asked before on the forum. Just search a bit to find it.

Thank you very much for your help with this issue.

so I took a look your work with the French class on the computer mouse controlling software. I guess the best way to visualize my project is to think of something like that, but it draws the current location of the eye gaze on the screen. If you could help me out with the Code for this I would appreciate it greatly.

I can give you the OpenMV Cam code. As for the PC side code… Um not sure what library to use for drawing anywhere on screen.

I know how to write Qt desktop programs. I recommend using Qt for doing the desktop program.

That would be awesome. If you could send me those files and perhaps tell me how to work the QT code I would be extremely grateful. Thanks again for your help.

Attached are the starting programs.

As for Qt… just google getting started with that. It’s not something you learn in a week though. It took me two years to get good. (1.72 KB) (1.75 KB)

Once again thanks for your help. I just wanted to know, could you send the serial data over to C# and does the MV cam function like a Arduino, where it runs its last saved program if it’s unplugged from the computer and plugged into a battery?


Yes and yes.

I have recently come up with the question about whether I can use two OpenMV cams at once. I wanted to use this to have one camera showing a forward facing live time shot, while the other watches the persons eye and records it on the view of the first one. If this is possible please let me know.

Sure, you just have to do the protocol interfacing between the boards.

Is there an easy way to do that?

Async serial is the easiest way to send commands from one board to another. You have to define the command protocol. You can also just use GPIO wires too if you want to sync things.

Would I need to write any code for that and if so, what libraries would I need to use?