You could restructure your network, but it’s probably not necessary. My phone is always behind NAT on the secondary router’s wi-fi. I got it working by:
Reserving a static IP address for my phone in the secondary router’s DHCP server.
Forwarding incoming ports 1714-1764 on that router to the phone’s IP address.
Allowing outgoing traffic from that router’s network to ports 1714-1764 on my workstation’s IP address (on the primary network).
Adding a device by IP address (my workstation) in my phone’s KDE Connect app.