Red to yellow swap will only affect the internal memory - battery vs accessory power. Most likely you don't have a power wire (blue wire stryfox mentioned) hooked up for the amp.