I think an under considered but vital aspect of fetishization is the dehumanizing of its subject. Imo, if you're avoiding that then you're not doing anything wrong. The term unfortunately leaves a lot of people going "if i find an aspect of someone/myself hot that's evil". It's a little silly
