The main reason that I quit antitheism was that I slowly realized that religion is only an excuse rather than a cause of oppression. The Crusaders did not simply read the Bible and immediately go out on a rampage. That would almost be like saying that violent video games make innocent people become violent themselves. Underneath the façade of religion are causes far more complex than that.
In the Crusaders’ case, the European ruling classes were interested in grabbing land and other resources from Palestine and the inhabitants therein and thereabout. The situation was similar for chattel slavery and serfdom: they merely used religion as an excuse. It was not the cause (the cause in this case originating in upper‐class dominance). There were far more examples of religiously justified oppression than those, but I think that you understand what I mean: the causes go deeper than what the surface suggests.
More recently, heterosexists and cissexists can distract us from their attitudes by pointing to scripture, but careful hermeneutics and human reason (of which, I’m sure monotheists would agree, the Almighty created) suggest that scripture is not the problem. It is the neopatriarchal attitude that is the problem, and mistaking scripture for the source—as if the scripture’s own stances came ex nihilo—is a classic case of confusing cause and effect. Religions are cultural phenomena; they are subordinate to culture, and cultures mutate regularly. That is why I used to liken religions to clay.
Even if our oppressors never had religion, they still could have appealed to pseudoscience, and most already do. Quite a few irreligious men are misogynists who’ll justify their oppression by claiming that women were genetically programmed to be caretakers or whatever. In fact, one could argue that the pseudoscientific excuse is more dangerous than the religious one because respect for science is almost universal whereas religions and spiritualities are very culturally specific.
Lastly, lower‐class monotheists can and have rebelled against their oppressors before. I believe that quite a few poor peasants during the Protestant Reformation were monotheists and yet they rebelled against their masters anyway. Religion is not a perfect tool for mind control; lower‐class monotheists can be just as revolutionary as lower‐class atheists, and upper‐class atheists can be just as troublesome as upper‐class monotheists.
That is why I gave up on antitheism: its explanatory power is woefully inadequate. There were a few other reasons, but the ones that I described above were the most important.
One of the minor reasons that I resigned myself from antitheism was that I was becoming pretty disgusted with how other antitheists were misbehaving. For example, a dozen years ago I watched a video of some white guy destroying a Qurʻan with a chainsaw and then drinking the shreds in a milkshake. (???)
More infamously, dozens of artists would make intentionally provocative and insulting depictions of an Abrahamic prophet, dirty Qurʻans or images of a prophet, intentionally destroy undamaged Qurʻans, and so forth… I found those actions and depictions to be very immature and nauseating in their own right even though I still disliked Islam, and that they portrayed their immaturity as being for a good cause—‘free speech’—was just embarrassingly pompous and melodramatic.
When I read about how the Crusaders treated Jewish artifacts, I thought back to those times of antitheists and other Islam‐haters behaving so immaturely.
Lastly, I think that for many of us in the lower classes, religion and spirituality are simply ways in which we cope with trauma. They may crude ways of handling a serious problem, but sometimes they’re the best that we can afford in a hyperindividualist, largely uncaring society with inadequate mental healthcare. Taking away a poor person’s faith wouldn’t fix anything.
There's a big difference between video games and religion (except in Dark Souls, of course), especially religion as practiced during the Crudades. Religion has true believers in a way other cultural practices rarely do.
Your overall point about material reasons largely driving the type of oppression/aggression we attribute to religion is a good one, but it's a bit overstated. People make religious decisions that harm their material interests all the time -- look at tithing, or those evangelical "send me money for your sins" TV shows.
But that's weaponised religion, just like the crusades. It uses people's belief system to exploit them.
Religion is a means, not an end. It can be used for extreme feats of generosity, or resistance, for example. And it can be used to oppress and empoverish.
How it's used depends exclusively on material conditions for whom is in control of what the belief system is used for, for which there's extreme variety from the micro to the macro scale.