100% agree on everything you said, except…as you listen, and perhaps get a better turntable or cartridge, you do start to hear differences. Better cartridge means you can start to tell the difference between pressings, etc. - and thus the rabbit hole gets deeper, wider, darker and nuttier.
I follow two KISS vinyl groups on FB and shake my head at the folks that collect every single release of every single album - and then discuss the merits of each. I stay out of that end of the pool and mostly just enjoy the experience like you.
That said, I have ‘upgraded’ a few albums to better pressings as they are so dear to me, but generally stay away.
Enjoy it and if you put your collection on discogs, share your collection here and make friends too: