Because “relativity” isn’t even close to your biggest problem with time. The way we communicate time changes historically, unpredictably, without obvious record. The only way to know what time you’re talking about is to know exactly how you got your information. What location, measured at what time relative to recorded changes in the local time zone, with how much drift relative to the last time you synchronized to which ntp server, and so on. These things easily account for hours or days of error, not just nanoseconds.
I don’t hate it as a concept, but I recognize that it contributed to my burnout during the pandemic. I would personally prefer a hybrid schedule over pure WFH.