When you turn the key to "ON", the pump runs for a short time, if the ECU detects that the engine has not started, the pump is disabled.
It is also disabled when you turn the key to "OFF". The logic is as follows: Key must be ON + engine must be running = fuel pump running.
The only exception is at start up, the pump is allowed to run a short time to build pressure for engine starting.

You are right, fuel pressure remains in the rail for some time after engine shutoff; however, the pump can build up to 56psi almost instantly. To witness this yourself, simply connecct a fuel pressure test kit to the schrader valve on the end of the fuel rail.

This kit is awesome, it comes with everything you need to verify fuel pressure in the Crossfire and (oddly enough) the Ford Ranger - the two vehicles I own:
https://www.summitracing.com/parts/otc-5630

It makes fuel pressure testing as easy as checking your tire pressure (Just don't use it with a hot engine).

I still don't know what sound you are hearing......