imo (and I'm not a manager, though I occasionally do manage people) there's different levels of collaboration. If you get the whole company in one room, that's one level of energy. If you get 5 separate but related teams in a room, that's another level. And if you get just 5 random people from the company together, that's another level.
There's definitely a balance to be had, but having your office be empty every Friday is not efficient, even if you want to increase socializaion and collaboration.
Plus there's some benefit to providing workers a place that isn't their house to work. Some people just need to go somewhere else to be at their most productive, not necessarily just for socializing.