I don't wear a uniform, and despite being a big guy, I'm not interested in a confrontation over street photography. In similar circumstances, I'd just show him the pics on my camera. And if I had taken a picture of his little girl, I'd erase it. I don't see this as an issue about photographer's rights. The guy in your scenario was misguided, but his primary interest was in protecting his little girl. Nothing wrong with that at all. I want to enjoy taking pictures, but not at the expense of making someone else uncomfortable.
Have to agree! I personally don't feel comfortable pointing a camera at a stranger without their permission. I guess I always tend to put myself in their shoes and how I would feel. If you are going to do street photography just be sensitive to the person on the othee side of the lens. It might save you an embarrassing confrontation.