I just reread your OP and noticed you don’t currently have an SSD. You may consider swapping one in as the boot drive and running the rest of your current hardware (and possibly buying a new case for that “new PC” feeling) since they’re so cheap. This will, hands down, give you the biggest performance boost out of anything.
Realistically, the main benefit you’d get from buying all new hardware is better power efficiency as your current hardware is more than adequate for a media server, but it would possibly take years to ‘break even’ on operating costs depending on how much electricity costs where you live.