How can you make sure your baby's umbilical cord stump stays clean and dry? Clean the stump and area around it with water when it gets sticky or dirty. Keep your baby's diaper from covering the stump. Use sponge baths until the stump falls off. All of the above.