is your ISP double NATing you? specifcally, are they providing a public IP address to your EdgeRouter? ISPs will often provide a residential router/modem combo and their router will distribute a private IP to your router - a double NAT. double NAT is usually not a huge issue, but its not nice to live behind one. if the ISP confirms that their equipment is bridging (and not routing), then the double NAT issue goes away.
all of the symptoms you describe sound like intermittent packet loss and there are lots of possible reasons for this including poor signalling from your ISP and misconfigured routers (possibly your ISP customer premesis equipment)
as long as your have verified good LAN cabling, including the ISP CPE to your router (you indicated a switch to GigE+ speeds - that needs good copper), the fact that you get this possible packet loss from devices behind your router but not between your router and the ISP CPE may mean your router and the ISP CPE are not playing well together due to double NAT.