Contrast is good, but if you switch the surround, handles and other trim to black your eye will be drawn to that point on your car.
That is why I got rid of all silver trim on my black SRT (essentially the opposite of what you want to do) with the exception of the ring around my wheels and exhaust tips. I recently added a 3/4" red stripe around the lower section of the car as well.
Look close and you will see that all the chrome and silver parts are now finished in a glossy black CF, but from a distance it all looks black.
Accent colors look good but because of their contrast, that is where your attention is drawn. You might want to use some electrical tape (peels off easy) to cover the silver trim and stand back to see if you accoplish what you wanted before permanently choosing a contrast color (wheels not included!). I did that when I decided to get rid of the silver. I think it worked out pretty bood. Bottom line, do what YOU like, it's your car. It's kinda fun making a unique car even more so....
Before......
And after
