Can the wheels stay on while the boat is in the water?

No, they are intended for launching and recovery on land and should be locked in the raised position or removed before extended use in the water.