Ask yourself if it's worth the hassle - for example, taking a picture of someone wiping a baby's bottom is bound to get you in trouble, as is photographing a drug deal. I have a gauge of the people I'm going to photograph and if it's worth it. I used to try to photograph fights when I saw them but I don't now - it's not worth aggravating two people whose adrenaline levels are soaring. All the attention can easily be turned to you, the person with the camera.