There are two sets of veins, or blood vessels. One is on the inside and the other on the outside of the rectum.