Rich people are affected by inflation. If your return on investment is 4%, but inflation rose 8%, you lost money.
The detachment of productivity gains from average wages is a much stronger argument. They more or less matched up through the 70s, but then a stark difference settled in as the extra money made from things went to the investor class rather than the working class.