Is there any way to do on device training for the Cam H7 Plus in tensorflow?
If it is not currently supported, if I wanted to dig into the camera code and try to implement it myself, do you have any advice on where to start / where in the codebase such a thing would go?
To be clear, I would like to be able to do training and inference on device (not train the model elsewhere, then just load the model onto the cam for inference only).
I am ok if the capabilities are limited / training is slow.
Hi, I appreciate you thinking this is possible. However, what you want to do is beyond challenging.
Instead re-work the issue. CNNs are great at generating feature maps. So, use a CNN that outputs a feature map. Then use a K-Nearest Neighbors algorithm to map that feature map to something. This will allow the camera to learn on the fly.
So, this is like a mix of traditional ML and CNNs.
Thank you for your response.
I was wondering if you could explain a little bit more about why it is “beyond challenging”?
Can you also explain in more detail how your CNN + KNN allows the camera to “learn on the fly”? I am intrigued.
Hi, you should Google how back-propagation and training a CNN works. It requires an extreme amount of compute.
See this forum thread: CNN in combination with KNN - Deep Learning - Deep Learning Course Forums
Thanks again for the followup.
I am familiar with cnn training and back-propagation, and I still feel that given what the cam can currently do with the tf library / operations, compute should not be the primary barrier.
I understand that large networks are definitely compute intensive for backprop, but specifically I was interested in something akin to transfer learning or personalization, where most of the training work of a large network occurs on a stronger piece of hardware, but a few layers (less than 3) at the end (not that deep, not many neurons) could be trained on-device on the camera (distinct and modular from rest of much larger network, would not have to backprop through entire pipeline).
Of course, I defer to your much greater expertise and familiarity with the cam hardware, but I would like your opinion on the cam’s ability to train even very small networks on-device (which alone would not be complex enough to be useful, but combined in the aforementioned manner, is useful).
I don’t really have knowledge with how to use the tensorflow operations to do learning. So, you are on your own there. However, I’ve only seen people use the CNN to produce a rich feature map and then they have a KNN as the final output that can be modified to produce the classification results.
This is essentially want you want to do.