Hmm, um, how many different exposure values did you try? Note that the scene at which you’re pointing the camera isn’t particularly suited for vision to have an easy job doing color tracking on. For example, you have way too much going on in the image and you’re pointing the camera into the sky.
In this situation there are too many variables and things aren’t going to work well. Can you constrain the problem in any way?
Also, note that it’s impossible to really ever select one colored object given thresholds. You will get noise, so, given that you’re going to want to do filtering on the objects to reduce the noise to make them easier to track. Like, for example, in one of your pictures you’ve clearly selected the cotton blobs. Given you’d done that the thing to do now would be to toss all blobs with less than a certain number of pixels and area.