This is fairly common practice, actually, and the only downside to it is if you have really hard water you could end up with little trails of water spots that need to be given a once over with a microfiber towel and some quick detail spray. At our old house the water was so hard that this method really wasn't an option - the little water spot trails were horrible. But in our current place I use this method just for the wheels, grille, etc.

As for inducing swirls in your drying process, if you do a final rinse with the nozzle off the hose and just let the water flow over the car you'll sheet off the vast majority of the water and can then basically just blot the rest dry with a good waffle weave microfiber drying towel. That's what I do, then use the leaf blower for the areas mentioned above.

But if your water is such that you can dry a dark colored car fully with a leaf blower, go for it!