I don’t see it happening unless he’s actually convicted in the 1/6 or Georgia trials.
Superficially, it seems obvious he broke his oath of office, but using superficial observations isn’t how our court system is supposed to work. “BUT HE LOOKS GUILTY!” ;)
The Cowboys For Trump guy who got bounced out based on the 14th Amendment was actually convicted first.