Is this being hosted in a VM that would be on-premises or cloud hosted? Is this going to be Windows-based or Linux/BSD based? It generally does not take a whole lot of computational horsepower to handle a lot of DNS requests. If you’re doing to handle DNS using Linux and don’t need dynamic update capability, you would be just fine using something like Unbound or NSD. I’d recommend maybe 2G of RAM and minimum 2 cores allocated. Now Windows will be much different. You’ll need more horsepower than that.
Okay, then I think you will be good with around 2GB and 2 cores. I cannot speak highly enough for using Unbound or NSD. As of about two or three years ago, an authoritative DNS capability was added to Unbound and I cannot speak highly enough of it. I’ve used Unbound in production reliably for that long. After moving my domains to Cloudflare though, I found DNS resolution to be speedier simply because the DNS servers sit on an isolated segment instead of mixing traffic so I went that way. But I would go back to using Unbound in a heartbeat if Cloudflare decided to do something stupid. Which is always a distinct possibility because, well, it’s Cloudflare.
Thanks for the detailed response! I will definitely check out unbound. Sounds perfect. And 2 cores and 2 GB RAM is exactly what I was hoping would be enough, so I can keep costs down.