The fuel pump does indeed have a check valve.
I should have spent more time with my previous answer. My mistake! The filter is primarily the fuel pressure maximum regulator, filter, and with a return line to the fuel tank to bleed off excess pressure.
The pump is designed to hold fuel in the line at about 36 lbs. pressure after the car is shut down. So it would be normal for the pressure to decrease at the fuel test port to 36lbs after shut down. The pump's check valve should hold that pressure in the line until the ignition switch is turned on and thus the pump should bring the pressure back up to 55~60 PSI at the test port. With fuel being held in the line the pressure will rise very quickly and allow ignition without hesitation.
There is also a possibility that a fuel injector is leaking and allowing the fuel pressure to drop. You could distinguish between the two problems by were you place the pressure testing gauge.
Sorry about my post #14 above being incorrect/incomplete.