The combination of up and down votes serve little purpose other than to enforce popular opinions because they ultimately boil down to like and dislike buttons most of the time. Combine that with a dumb karma system (or people paying attention to a summation score) and you get tons of reposts guaranteed to farm upvotes, bot farms using up and down votes to make their opinions look more popular and opposing opinions look less popular, and vapid le ebic meme posts that add nothing to the conversation but make people blow air through their nose and thus "have my upboat".
Like it or not, people often get swayed seeing numbers go in either direction. I'd rather not see people try to conform to popular sentiment just to play it safe in case they care about this silly score. Disabling downvotes at least means that a post receiving few upvotes could've done so for a variety of reasons so people can't assume more upvotes automatically means better and there's no red number that people use to automatically disregard an unpopular opinion. If you wish to point out that a post is bad, you can do so with words, then you actually have to provide a reason instead of "I don't like this post".