Do you think Hitler made it so far because his speech wasn't silenced earlier or because he silenced his critics speech? I think throughout history, the side trying to silence speech were the bad guys.
Part of me thinks you're right about the guardrails in place. The other part of me thinks it's still possible if you corrupt those guardrails over decades. In my option, it's more likely that the corruption's goal is to destroy America than it is to take over the world.
Do you think Trump has been silenced? Or any other right-wing personality? Fox is the most popular cable news station in the world. The Top 20 list of political podcasts is littered with right-wing content. One of the largest social media platforms in the world is run by someone who has endorsed Trump, spent 100s of millions of dollars supporting Trump super PACs, and is running Trump's ground game in swing states. If there is much silencing going on, it doesn't seem to be working.
True examples of silencing include things like book bans (which both sides have occasionally done throughout history, but the right has taken a particular shine to recently), or the government (i.e., FEC) threatening to pull broadcast licenses for content they don't like (which Trump has proposed), or political figures describing the free press as the "enemy of the people" (something Trump has done more times than you can count).
The press is broken right now. There are a bunch of reasons for that, we don't need to go into all of them. But even in its broken state, the press always has been, and still is, protection from government overreach. All authoritarians have to convince the populace that the free press is the enemy, so they have carte blanche to do whatever heinous shit they want without people uprising. I have seen some on the left trying to control/influence/delegitimize what the free press says: e.g., relative to Biden's decline or Trump's craziness. They justify this by stating Trump is a uniquely dangerous threat, so everyone needs to be all in on stopping him, the ends justify the means. I don't buy that shit at all...because as soon as you go down that road, you are giving permission to the other side to do the same stuff.
To summarize, I don't think either side has a monopoly on trying to silence dissent, but I definitely wouldn't say the right is being silenced in an uneven way.