I wouldn’t assume any gay man knows you are a lesbian unless you tell them. In regard to your neighbors, being friends with someone you share a wall with can turn out bad, so it is healthy to keep some distance.

That all said, they could still not be great people (for you) and that’s about as fine as it’s going to get.