Annoying how? Like are they going to come over, and borrow my lawn mower, but then not return it for a week and half because it was too much effort to bring it back?
Wait.....no, that's my neighbor. Is my neighbor a human sized wasp?
They're pretty chill as far as wasps go. They don't go looking for fights but they'll bring it if you antagonize them enough. You can swat one away and they'll usually leave you alone. Unlike an asshole yellow jacket that will make it their life's mission to bring you pain if you look at it wrong.
My grandmother used to tell me they don't sting when I was a little kid. I also suspect she used to identify all wasps as mud daubers so I'd stop freaking out about them.
