I've enjoyed milk for the flavour and texture only once: the semester I lived across the street from a dairy. To this day, in the weird circumstance I have milk in the fridge for cooking, I shake it vigourously.
Much of it is about sourcing and practices. Raw milk is not inherently dangerous, and we'd be out a whole world of cheeses if they had to be made from pasteurized.
Unless there's some sort of deadly disease commonly involved in it (which, in the American food supply right now, is what's up.)
I mean bullets are not inherently dangerous either, but if the environment you're going into involves them moving at high speeds sometimes, you probably want to take some precautions about it.