Dietary fat also plays a major role in your cholesterol levels. But not all fats are the same. You should try to avoid saturated fats and trans fat. These unhealthy fats can be replaced with healthier unsaturated fats.