As someone have said, lemmy.world blocks VPNs.
But I'm curious, why are you sometimes using VPN, but other times exposing your real IP? Doesn't that defeat the purpose, since your instance admins would still have your real IP, once you have accessed your account from your real IP, even once.
Either use VPN all the time, or just don't bother. A half-assed approach doesn't really do anything.