I use pfsense's HAProxy integration and a combination of Cloudflare or Lets Encrypt certificates for external stuff. For internal-only stuff I have a root CA I distributed to my computers that I use to sign certificates. My docker box that serves most of my internal stuff has an nginx-proxy-manager container with a wildcard certificate so that I don't have to sign one for every new subdomain on my docker host, and the various containers with services in it talk to it over a private docker network. Buying a cheap domain and managing it through Cloudflare simplifies a ton of stuff.
Yeah setting up a reverse proxy can be confusing, I still struggle with it sometimes. You can get away without it, but when you have your basic setup working then it's great to have.
Other people have said lots of useful things so I wanna just add on: nginx proxy manager is really useful for this. It's a webui that automates reverse proxying with Nginx (so that you can host multiple pages on the same machine/port) and also centralizes managing SSL certificates, including automatically obtaining them from Let's Encrypt.