I wear contact when i dive, and i find no problems with it. What i do is this, i bought an extra contact case, adn i save the previous months lens in the extra case that i marked as DIve. then when i dive i wear the saved pair and bring my regular pair with me on the boat.
If i lose a contact i have a backup pair, and in truth in the two years i've been diving i haven't had a issue with losing them.
You should be just fine, just cloe your eyes when you flood your mask, you'll be fine.
IF you have more questions you can contact me by email or at scubadivernet.com