Before nationalism and monotheism, as far as warlords were concerned a peasant was a peasant; what language they spoke did no impact their ability to plow. As such, Settler Colonialism was rare but not unheard of. Best examples are the crusades, Germany's colonization of Prussia (see Baltic crusades), Japan vs the Ainu, and the Bible (Jewish conquest of philistine).
The far more common occurrence was that invaders would replace the ruling/warrior class with their own people and the commoners would be left to work. Sometimes the two classes would form a common culture (Saxon invasion of England, Norman invasion of England, Roman Europe, French invasion of Gaul), other times they would not (Ottoman Turks outside Anatolia, the Selucid Empire).