I put a large cable tie around the rubber hangers above the mufflers. It seems to have stiffened everything up so there is a lot less sway.
It's been raining here so I haven't been able to test it out yet. I want to do it with the top down so that I can hear everything.