Actually the 504 might not be lighter than the 304 - good point.
My 604 is a pound lighter than my 304 was.
The pickups are much more rich and less sharp sounding to me, and having a midrange control on the EQ is more useful than the selector switch on the 304.
The big differences:
304 - great intro bass for features, comes in some very cool gloss colors
504 - adds AlNiCo V pickups, Active/Passive switch, midrange control on EQ, loses preset selector, comes in subtle, more “woody” satin colors
604 - adds Alder body (much lighter and slightly brighter), flame maple top (looks amazing IMO), different preamp voicing
I would say the 504 is definitely worth the price difference over the 304. At least those are all features important to me. And of course the price difference is much, much less if you buy used (always a good idea).
Whether or not the price difference between the 604 and 504 is worth it to you is a good question. I definitely enjoy its light weight and looks.
I was saving up for a new or used 504 when my 604 went up for sale. I got it for $350, used, in perfect condition. You should be able to find used 504s in the $350 price range without issue, and may get lucky like I did with a 604.