Router A would also need to bridge in order for that to work
Why would Router A also need to be a bridge? Router B is configured to bridge its devices to Router A’s network, so, from what I understand, its devices are treated as if they are on Router A’s network – bridging is layer 2, and mDNS is layer 3 (afaik), so Avahi should be able to resolve across the bridge.
On the other hand, there are ways of setting up Multicast Forwarding if the router supports it, or you could have a device in both networks that does Avahi/mDNS Reflection.
Wouldn’t this only matter if Device A, and Device B were on two separate vlan’s?