I've had a gloryhole at my condo (a glorified apartment) for the last 9 years.
There are only 4 units in the building. Up until earlier this year, I was the only renter in the building, the other units were owned by the occupants of their units. The neighbor in front of my unit had been there for almost 30 years but he sold the place a few months ago. I never had any problems with him. The lady who bought his unit is polite and nice, but she is too friendly/talkative for my introverted self. She has made a few comments about my "friends" using the shared driveway. (I tell everybody to park on the street.) If she was ever to question about the different guys coming over, I have already planned on telling her they are coworkers or they work for me. (I travel a lot for work and she knows that I am gone a lot.)
The neighbor on the other side has lived there for about 20 years. They also sold their unit a few months ago. Our front doors face opposite sides of the building and the ONLY time I see them is when they are in the backyard. The family had moved out about 4 years ago, but their adult son was living there with roommates. 2 years ago, one of the roommates was on Grindr all the time. We never talked, but it was odd to see a guy was "20 feet away" all the time. And on the nights he would host gay parties, my Grindr would be filled up with "20 feet away" guys. (And I did a few of those party visitors over to my gloryhole a few times!) The family that bought that unit is from Chicago and they only stay in the unit 2 or 3 times a month. It is super quiet over there.
As for the other unit, the opposite corner from mine, the old lady that lived there for 30 years died a few years ago. The place was sold last Spring. The remodeled the place and had new tenants in there. It was a really hot black guy and his daughter. He was always outside and I would talk to him a lot. He never said anything about the stream of guys coming and going from my place. He lived there for a few months and then disappeared. The place is empty again.
The rest of my neighborhood is apartment buildings. They are mainly filled with graduate students or small/new families. 20 feet outside my front door is the end of an apartment building. The guy who lives in the first apartment (closest to my door) is introverted too. He has been there 5 years and I have talked to him 1 time, when his mom was visiting him and she came over and introduced them. He keeps quiet and stays in his apartment. There are 2 more apartments on that end of their building. Those tenants change every year or two. I have become friends with a couple of them, and they never asked or said anything. The rest keep to themselves and I've never had issues with any of them.
As for my landlord, she has been a friend of mine since college. She was the straight girl that hung around all the gay guys. I'm 90% sure she knows what happens, but she is cool and doesn't say anything. And I'm not sure there is anything she could legally say. I'm not doing anything illegal. And if she had any questions or concerns, she would have asked me.
I say go for it. Just try to be considerate of your neighbors and don't cause problems. And HAVE FUN!