When I had my last black Labrador male, named Number Seven... he was not neutered for the longest time. I think the vets around me were obsessed with cutting that dog. Every time I took him in for anything... they always suggested I neuter him.
He was almost four years old before I finally did have him neutered. I think the rough play can be curbed by training and reminding the kids that they should not treat the dog in a rough manner. I never allowed rough housing with Number Seven and he was as mellow and calm an animal I ever owned.
Actually the reason I had him neutered was the vet claimed.. there might be a possible growth in his testicle during an exam. So for the sake of his health and the fact I was not going to breed him, I finally gave in and had him neutered. Turns out the Vet lied. All she wanted to do was to cut that dog. I swear.
The benefits of neutering are more then just a possible mellow attitude in the dog. There are definite health benefits to neutering for the dog. If he is not going to be a breeder, then he don't need them. The other benefit is when them gone before the urges hit.. they males don't run off in the middle of the night. That is a real plus. Also they tend to me more dog friendly when meeting other dogs in the field. Although, mine.. even neutered was NOT dog friendly at all.
In the near future I am going to get another pedigree dog. It will be a German Short haired Pointer. I tend to lean towards the males but I like the females for being in the house. I do not think I will have the dog fixed. Until it is a medical emergency. I know this is selfish of me. But I am like you, I hate making a him or her, an it.