I’ve seen something like this years ago. The cause was that Ethernet ports on the WAN gateway were re-autonegotiating.
Try setting your WAN port(s) on your gateway to fixed speed and see if that helps. Took me ages to solve that one - and in the end it was a gnarly old network engineer who...