You are correct in a sense, but things like camber and toe-in or out do not affect how the car drives in a straight line but will eventually eat up the tires. The best thing to do is to keep an eye out for strange wear patterns on the tires. Such as it wearing more on the inside edge rather than the outside and vise-versa. Tire wear patterns will tell if you need an alignment.

That's just my Dos Centavos