In an episode of DS9 I heard some of the characters mention that they not only have deflector shields, but also “structural reinforcement shields.” So whatever science-fictiony force field is used to protect them from phasers and micrometeorites is also coursing through the skeletal structure of the ship.
When I heard this it immediately clicked in my mind: whenever the ship is hit with phaser fire the explosions happening inside are recoil from these internal shields. Perhaps the catastrophic damage prevented by structural reinforcement shields outweighs the localized damage of potentially fatal recoil.
That is my favorite explanation, anyway.
(This assumes all ships have structural reinforcement shields, and not just the Defiant.)