You can disable the public key authentication and allow password authentication (you have received your password in the server installation finished e-mail)
- Log into SSH
- Edit the file with your favorite editor:
/etc/ssh/sshd_config - Lookup the variable: PasswordAuthentication and change 'no' to 'yes'
- Save and close the file
- Run this command:
service sshd reload
Now you will be able to SSH into your server without public keys.
