So the thing I found myself doing was I had adopted a sort of downward strike to my pluck. I was still plucking directly through the string like we are supposed to, but due to how I was plucking my finger would come down with kind of a percussive tap on the string before i plucked it.
It actually sounds kind of cool. But not what I wanted in the context of normal plucking 
Anyway, like Gio said, just being aware of it helps a lot towards avoiding it.