Next time it happens, turn the key on *NOT START* and let the fuel pump prime. Then turn the key off, then turn it on, and then turn it off. Do this a few times to get it good and primed. Then, turn the key all the way and start it. See if it runs rough.

My first SRT did something similar with 1300 miles on it. The fuel system wouldn't hold a prime. They replaced the fuel pump assembly (under warranty) and all was well. It would also do a 'long crank' before starting, which was associated with the rough idle.