I'm with Will here, I would never volunteer the info unless I feel like I know the person pretty well. It just leads to more questions. Given that I haven't regularly watched WWE in like 10 years and that's increasingly been seen as the entire scope of what wrestling is, it's just easier to avoid the subject than try to explain. Sometimes I will just say that I watch old 80s stuff on DVD and don't watch the TV stuff. This oversimplifies things so as not to explain about the indies & Japan. Back when I used to go to Japan regularly, I would use that as saying I preferred that over American wrestling but then that would lead to the inevitable questions about if it was sumo. So yeah, it's a lot easier to stay closeted than to begin opening the door to explain just how of a deep niche you are involved in.
Edit: on second thought at my old job where I could wear t-shirts to work, I'd routinely wear ROH or PWG shirts or whatever so I wasn't trying to hide anything. Largely I see it as no one's business as I'm a pretty private person but if someone were to ask or know the significance of what was on the shirt I would engage them. I guess it's pretty complex with me as I would wear some wrestling shirts in public that don't openly have like pictures on them but reading wrestling books in public, probably not, and watching wrestling on a laptop in like an airport or on a plane, most definitely not. In fact when a friend of mine & I were sitting together on a plane going to a wrestling convention, I was a bit embarrassed to be sitting by him just blantantly watching CZW on his laptop, but I guess that was mainly just due it being no one's business what I would be watching. I would read an Observer on a laptop on a plane though but I'd be sure to angle it such that no one was really looking at what I am reading.