Nearly all of history's worst diseases have come from animals, and animals have those diseases whether we eat them or not. Factory farming certainly enhances the danger, but cows don't stop existing just because we don't eat beef. There is, however, also an upside to meat consumption: being around/eating animals all the time also builds up your immune system's defenses against diseases that originate in those animals. See: indigenous people in the Americas dying in droves to diseases they had no immunity to because they didn't farm/ranch animals. I mean and also the smallpox blankets, but you get my point.