The only reason ssh client would "hang" without any output is when it's waiting for external key storage to allow access. It's designed that way to give user some time to approve access to key storage.
It sometimes happen that the installed key storage is broken in a way that it fails to show user modal, for any reason (showing on wrong screen, wrong desktop, wrong activity, wrong framebuffer, ....)
One solution (that you already did) is to change the SSH agent env variable to point to different key storage.
Another would be (if possible) to uninstall the broken key storage if you don't use it. But it is sometimes needed/used by other apps.
It's overall good to notify/open bug on your distro issue tracker to notify that some packages are missconfigured (maybe have missing dependencies) or conflicts with other ones.