If you mean the "passkeys" that are becoming popular as a "password replacement", it's basically speaking a public private keypair. What makes it more secure is that, under normal conditions (aside from backing up the passkey), the private "secret" part of the keypair never leaves the app or device it's stored on. It's only used temporarily to sign messages and prove that you have the secret key, unlike a password which needs to be sent securely to a server to validate.
You could in theory store a backup on a USB drive but since passkeys are new, it highly depends on the password manager you use to store the passkey. Since passkeys are more complex than something you can memorize/type, it has to be stored in a password manager of some sort to be useful, so you would need to check that password manager allows backing up passkeys. There is currently work being done to standardize the formats/protocols to transfer passkeys so it seems this is very much up in the air. For example, I use BitWarden which stores passkeys, ~~but it seems like I can only add or delete passkeys to an entry, not export them~~ and apparently they get exported with the passwords when the vault is exported. BitWarden also syncs your vault to every logged in device though so you could see that as a form of backup. Going one step further, ~~even though BitWarden doesn't have a passkey export/backup feature yet~~ (in addition to Bitwarden's vault export), the self-hosted server also stores all your passwords including passkeys in regular files which also can be backed up (this is how I back up my VaultWarden instance) - although it would probably be hard to use that backup in any other way besides restoring it onto a BitWarden server instance.
Edit: I didn't realize passkeys were exported with the vault export, since I haven't used it and noticed that editing an entry doesn't allow you to view passkey data - only remove, updated my comment to reflect that.
The key difference is that during normal use, the private key of the passkey doesn't leave the device (or password manager). The passkey basically comes in 2 parts, the public and private (secret) part. In order to log in, the website presents a cryptographic challenge that is only solvable using your private key - and crucially you can solve the challenge without revealing your private key. An attacker could get your answer to the challenge and still be unable to solve additional challenges without the private part of your passkey.
This of course makes it basically impossible to manually log in using a passkey and a keyboard, without any password manager to do the cryptographic calculations (unless you have a LOT of paper and time), but the security advantage of making it near impossible to be phished is generally regarded as a net positive. In order to steal a passkey there would need to be a vulnerability in the software, since passkeys make it much harder to trick a user into giving it away (since tricking the user into logging in on a fake website doesn't work due to the aforementioned cryptography, the main way to steal a passkey would be to trick the user into exporting it - which is a much higher bar).