After I wrote that I got more aggressive and the plastic connector just popped right out - but I had to pull much harder than I was initially comfortable with.

Thanks for the reply.