Go far enough back and there were no passports but you were near certain to be killed if you wandered into another tribes space. Horrific violence is incredibly common. Go further and you might be able to wander in to trade but not welcome to settle and only if a handful of assholes who run everything hadn’t personally decided they had a dispute you ordinary folks ought to die over with zero recourse to pick new assholes.
Looking backwards is almost always a mistake because most of history is pretty awful.