I'm confident that a lot of them -- probably most of them -- do indeed work very hard, in every sense. Whether effective at their tasks or just fucking lucky (hint: it's the latter) the ones we know about do put in the hours. Hording wealth is their profession, hobby, and entire life all wrapped up in one.
But if the average lifetime earnings of the typical American is around $1 mil you would need to work 1,000 times harder than them, saving and investing every bit you earn and not spending it on anything including necessities, to just barely qualify as a billionaire in America by the time you retire. The average American works somewhere around 35 hours a week, so was the minimum billionaire working 3,500 hours a week? I suspect not.
There's no point or value in calling a billionaire lazy. It may or may not be true, but it ignores the point. It does not matter how hard anyone works. You don't get to a billion ethically.