Wallets
You usually store cryptocurrencies in wallets. A wallet is an application that can be installed on your computer, your mobile phone or on a website. The wallet connects you to a blockchain and with the help of a private and a public key you can see your assets and send them to other accounts.
An exchange keeps wallets for you, where you get access to the public key (your address), but never the private key. It is easy and painless to keep the money with a exchange, but you then give away control to them. It is always recommended that you keep your own private key in your own wallet.
Security
If you somehow lose your private key, your assets are gone. If someone else get hold of your key, then the person can access your funds and send them on to themselves or elsewhere. That is why it is essential that your private key is always kept safe and secret.
The worst thing to do is saving your private key in a file somewhere on your computer. It can then be lost if the computer crashes and you also risk being hacked. A better option is to keep your private key in a safe place such as LastPass or 1Password.
But the absolute safest way is to buy yourself a hardware wallet, such as the Ledger Nano X. Such a key requires you to plug it into your computer and unlock it with a PIN. If your PIN code is lost, or if the device burns down, you can always buy a new one and reset it with a bunch of words in a specific sequence. These words should always be stored securely, for example in a safe or safe.
These words create your private key using an algorithm. The same algorithm can create all the private keys for you that are used by a multitude of different cryptocurrencies.