If a baby with galactosemia is given milk, its body cannot metabolize galactose due to a deficiency of certain enzymes. Unwanted substances build up in the baby's body and act like poison. This can cause damage to the liver, brain, kidneys and eyes. Sometimes galactosemia can even be fatal.