Detecting nerf darts on carpet in low light conditions

Dear all,

I just got my H7 Plus camera last week and I’m loving it so far! I have tried the examples and wrote some small python apps to do some light work and I am very happy with it all. However, I do have a few (basic?) questions that I’d like to ask to see if what I hope to achieve is feasible or if I am too naive as a beginner.

Basically, my cat loves to play with nerf darts - the classic blue, soft darts with the orange tips that are used in nerf guns. I thought it’d be nice to create a robot that would be able to detect a nerf dart on the floor, grab it, and throw it again. Since I am new in robotics I realise it’s probably ambitious to do this as a starting project, but you never know. :slight_smile:

I have created photographs of the nerf darts on our carpet with the normal lighting conditions in the dark (i.e. it’s really dark). I have attached an example. (0034).
00034. I’d say that these kind of images are useless for any purpose so I used my phone torch to light one which gave a somewhat better result.

I have noticed that when I do it by bright daylight, the blob detection algorithm works fairly well to detect the orange tip, but somehow the blue foam doesnt respond well with the blob detection for blue (even if I used the threshold editor).

My questions would be:

  • is detection of an object like this on a carpet doable? (most youtube videos with object detection are on white background)
  • is it true that lighting is a very decisive factor in recognition of blobs and that I perhaps should use an 8 neopixel ring to shine light on objects?
  • is it true that machine learning is probably not the way to go as this may be able to answer the question ‘is there a nerf dart in sight’ but not ‘this is where the nerf dart is’ which I will need for alignment with a robot arm?

Thanks in advance!

Yeah, lighting really affects color tracking. The issue is that the difference between colors shrinks when there’s less lighting and everything turns into grayscale.

There’s the new FOMO algorithm by edge impulse. It kinda does what you want. Please look into it.

Hi there!

Thank you so much for steering me into the right direction by confirming the lighting issues. I’ll be heading back to the shop to see if I can find something suitbale!