I don't think it would be much of a problem if you rinse them well with clean water and dry.
If you are worried you can rub your hands in dechlor, clean under your fingernails, rinse well and allow to dry. Maybe even soak your hands in some water so that the residue in your pores and fingernails oozes out. Or add a little extra dechlor after the Wc, but I sincerely doubt that after 3-4 times washed you still have enough on you to harm them