I am trying to log in to my server with ssh keys (I use putty to do it) but every time I try to connect it says Network error: Connection refused. I believe it's something wrong with the public key not sure though. The key looks like this:

rsa-key-public AAAAB3NzaC1yc2EAAAABJQAAAQEAx+KoPhVxfBrnN8cFb+hG9MveY0cfNpn9mAcN
hsfkEvxeG2EqLRYtaXUBXPgl3uILvXYbqG7HSBq/kZe/AICn/aK89rCGAozEepde
aYmy9EtmfPU8pFgTrgMils8X6b5kPPxCBZ2pfeL/q4SUke+/xpV1x98py6PHM8Vm
JaBciqvaa89QLvWf3IUuxm7798WvGUPlSMtuE2wnYsyJ4W65nBCs4PCROpaPmcmq
iP0VF+Vm5vC3W/F00PC1w3R3BMdDoS2VJj7jQTR1Ralbn9cM185/pZY8lvkX4lEQ
MJvvwRM1Oy/g+J7+RbPR/XTrrRmKQq5mnWU0ICV5qvTnsc+Lyw==

but all in one line. I used puttygen to create the key and it looked like this by deafult:

---- BEGIN SSH2 PUBLIC KEY ----
Comment: "rsa-key-public"
AAAAB3NzaC1yc2EAAAABJQAAAQEAx+KoPhVxfBrnN8cFb+hG9MveY0cfNpn9mAcN
hsfkEvxeG2EqLRYtaXUBXPgl3uILvXYbqG7HSBq/kZe/AICn/aK89rCGAozEepde
aYmy9EtmfPU8pFgTrgMils8X6b5kPPxCBZ2pfeL/q4SUke+/xpV1x98py6PHM8Vm
JaBciqvaa89QLvWf3IUuxm7798WvGUPlSMtuE2wnYsyJ4W65nBCs4PCROpaPmcmq
iP0VF+Vm5vC3W/F00PC1w3R3BMdDoS2VJj7jQTR1Ralbn9cM185/pZY8lvkX4lEQ
MJvvwRM1Oy/g+J7+RbPR/XTrrRmKQq5mnWU0ICV5qvTnsc+Lyw==
---- END SSH2 PUBLIC KEY ----

Output of cat /etc/ssh/sshd_config:

# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
#Port 2222
# Use these options to restrict which interfaces/protocols sshd will bind to 
ListenAddress 192.168.1.20
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key 
KeyRegenerationinterval 3600
ServerKeyBits 2048

# Logging
SyslogFacility AUTH 
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin prohibit-password 
StrictModes yes

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile ~/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts 
RhostsRSAAuthentication no
# similar for protocol version 2 
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication 
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED) 
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with 
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords 
PasswordAuthentication yes

# Kerberos options
#KerberosAuthentication no 
#KerberosGetAFSToken no 
#KerberosOrLocalPasswd yes 
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

Xl1Forwarding yes 
Xl1DisplayOffset 10 
PrintMotd no
PrintLastLog yes 
TCPKeepAlive yes 
#UseLogin no

#MaxStartups 10:30:60 
*Banner /etc/issue.net

# Allow client to pass locale environment variables 
AcceptEnv LANG LC *

Subsystem sftp /usr/lib/openssh/sftp-server

# Set this to 'yes' to enable PAM authentication, account processing, 
# and session processing. If this is enabled, PAM authentication will 
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass 
# the setting of *PermitRootLogin without-password*.
# If you just want the PAM account and session checks to run without 
# PAM authentication, then enable this but set PasswordAuthentication 
# and ChallengeResponseAuthentication to 'no'.
UsePAM no

Output service ssh status / systemctl status ssh.service:

peter@PM-server:-$ service ssh status
● ssh.service - OpenBSD Secure Shell server
 Loaded: loaded (/lib/systemd/systemissh.service; enabled; vendor preset: enabled)
 Active: active (running) since Mon 2017-04-03 16:02:13 CEST; 3h 37min ago
 Main PID: 1577 (sshd)
 Tasks: 7 (limit: 4915)
 Memory: 23.1M
 CPU: 3.774s
 CGroup: /system.slice/ssh.service 
 ├─1577 /usr/sbin/sshd -D 
 ├─2351 sshd: peter [priv] 
 ├─2359 sshd: peter@pts/0 
 ├─2360 -bash
 ├─2395 systemctl status ssh.service
 └─2400 pager

Apr 03 16:25:11 PM-server sudo[2030] :pam_unix(sudo:session): session opened for user root by peter(uid=0)
Apr 03 16:32:45 PM-server sudo[2030] :pam_unix(sudo:session): session closed for user root
Apr 03 16:50:45 PM-server sshd[2068] : Accepted password for peter from 192.168.1.19 port 57813 ssh2
Apr 03 17:19:14 PM-server sudo[2135] peter : TTY=pts/0 ; PWD/home ; USER=root ; COMMAND/bin/chown peter:peter peter
Apr 03 17:19:14 PM-server sudo[2135] :pam_unix(sudo:session): session opened for user root by peter(uid=0)
Apr 03 17:19:14 PM-server sudo[2135] :pam_unix(sudo:session): session closed for user root
Apr 03 19:25:09 PM-server sshd[2351] : Accepted password for peter from 192.168.1.19 port 56635 ssh2
Apr 03 19:26:05 PM-server sudo[2372] peter : TTY=pts/0 ; PWD/home/peter ; USER=root ; COMMAND/bin/systemctl
Apr 03 19:26:05 PM-server sudo[2372] :pam_unix(sudo:session): session opened for user root by peter(uid=0)
Apr 03 19:28:14 PM-server sudo[2372] :pam unix(sudo:session): session closed for user root

Here is the out put of sudo systemctl.

Output of sudo sshd -T

 port 22
 protocol 2
 addressfamily any
 listenaddress [::]:22
 listenaddress 0.0.0.0:22
 usepam yes
 serverkeybits 1024
 logingracetime 120
 keyregenerationinterval 3600
 x11displayoffset 10
 maxauthtries 6
 maxsessions 10
 clientaliveinterval 0
 clientalivecountmax 3
 streamlocalbindmask 0177
 permitrootlogin without-password
 ignorerhosts yes
 ignoreuserknownhosts no
 rhostsrsaauthentication no
 hostbasedauthentication no
 hostbasedusesnamefrompacketonly no
 rsaauthentication yes
 pubkeyauthentication yes
 kerberosauthentication no
 kerberosorlocalpasswd yes
 kerberosticketcleanup yes
 gssapiauthentication no
 gssapikeyexchange no
 gssapicleanupcredentials yes
 gssapistrictacceptorcheck yes
 gssapistorecredentialsonrekey no
 passwordauthentication yes
 kbdinteractiveauthentication no
 challengeresponseauthentication no
 printmotd no
 printlastlog yes
 x11forwarding yes
 x11uselocalhost yes
 permittty yes
 permituserrc yes
 strictmodes yes
 tcpkeepalive yes
 permitemptypasswords no
 permituserenvironment no
 uselogin no
 compression delayed
 gatewayports no
 usedns no
 allowtcpforwarding yes
 allowagentforwarding yes
 allowstreamlocalforwarding yes
 streamlocalbindunlink no
 useprivilegeseparation yes
 fingerprinthash SHA256
 pidfile /var/run/sshd.pid
 xauthlocation /usr/bin/xauth
 ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256- ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
 macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256- etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac- 64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
 versionaddendum none
 kexalgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2- nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman- group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14- sha256,diffie-hellman-group14-sha1
 hostbasedacceptedkeytypes ecdsa-sha2-nistp256-cert- v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521- cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert- v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh- ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
 hostkeyalgorithms ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2- nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh- ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2- nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa- sha2-256,ssh-rsa
 pubkeyacceptedkeytypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa- sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh- ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2- nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa- sha2-256,ssh-rsa
 loglevel INFO
 syslogfacility AUTH
 authorizedkeysfile .ssh/authorized_keys .ssh/authorized_keys2
 hostkey /etc/ssh/ssh_host_rsa_key
 hostkey /etc/ssh/ssh_host_ecdsa_key
 hostkey /etc/ssh/ssh_host_ed25519_key
 acceptenv LANG
 acceptenv LC_*
 authenticationmethods any
 subsystem sftp /usr/lib/openssh/sftp-server
 maxstartups 10:30:100
 permittunnel no
 ipqos lowdelay throughput
 rekeylimit 0 0
 permitopen any

Here you go short manual covering the whole process:

How to SSH Ubuntu from Windows via PuTTY, using keys

I. Generate SSH 'key pair' in Ubuntu and create authorized_keys file

$ ssh-keygen -t rsa -b 4096 Enter
Generating public/private rsa key pair.
Enter file in which to save the key (/home/$USER/.ssh/id_rsa): Enter
Created directory '/home/$USER/.ssh'.
Enter passphrase (empty for no passphrase): type your passphrase Enter
Enter same passphrase again: retype your passphrase Enter

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys Enter 
$ chmod go-w ~/ Enter
$ chmod 700 ~/.ssh Enter
$ chmod 600 ~/.ssh/authorized_keys Enter

$ ls -la ~/.ssh Enter
drwx------ 2 user user 4096 апр 2 17:21 .
drwxr-xr-x 3 user user 4096 апр 2 17:40 ..
-rw------- 1 user user 738 апр 2 17:21 authorized_keys
-rw------- 1 user user 3243 апр 2 17:15 id_rsa
-rw-r--r-- 1 user user 738 апр 2 17:15 id_rsa.pub

Please note we don't need to use sudo. If authorized_keys already exists the output redirection >> just will append a new entry.

Make a test - SSH to localhost using the username of the current user:

$ chmod 600 ~/.ssh/id_rsa.pub
$ ssh $USER@localhost -i ~/.ssh/id_rsa -p 22 -v

Where: (1) you can omit the options -i ~/.ssh/id_rsa -p 22, because these are the default values, and (2) -v will turn on verbose mode. Further information can be found in man ssh.

Please note, this test will pass "only" with the default configuration of /etc/ssh/sshd_config. Here is the default sshd_config of Ubuntu 16.04.

II. Convert id_rsa private key into .ppk format and use it

Where .ppk means PuTTY Private Key.

Approach 1: Using puttygen for Linux:

The idea for this additional editing came from this topic where was discussed the conversion from .ppk into an OpenSSH compatible format.

1. 
   

   Install putty-tools in Ubuntu. Open a terminal and type:

   
   
   

   sudo apt install putty-tools
   

   
   


2. 
   

   Convert the private key:

   
   
   

   puttygen ~/.ssh/id_rsa -O private -o ~/.ssh/converted_id_rsa.ppk
   

   
   



3. Copy the converted private key (converted_id_rsa.ppk) into Windows.




4. Use this .ppk key with PuTTY to connect to Ubuntu. Screenshot.

Approach 2: Using puttygen for Windows

1. Copy the private key (id_rsa) into Windows.




2. Run 'PuTTY Key Generator' (puttygen.exe) and click on Load button. Screenshot.




3. Switch to All Files (*.*) and Open your generated in Ubuntu (id_rsa) private key file. Screenshot.




4. Enter the passphrase if there is one, then click on OK. An notification will be appeared - click on OK once again. Screenshot.




5. Edit Key comment and Key passphrase if you need it and click on Save private key. Screenshot.




6. Save your new .ppk key in a convenient location. Screenshot.




7. Use this .ppk key with PuTTY to connect to Ubuntu. Screenshot.

References:

• Simple explanation about the conception of 'public/private key pair'




• SSH and Encrypted Home directory




• Ubuntu Documentation: SSH/OpenSSH/Keys




• How to convert .ppk key to OpenSSH key under Linux?

Usually, Network error: Connection refused means that the server refused your SSH connection entirely. It didn't have a problem with the key PuTTY used with it because it didn't get that far in the connection process to even bother with keys. Instead, it refused the connection immediately. This is usually due to the server not running an SSH server (or the SSH server being broken), you having the wrong port, or you having the wrong server address.

However, you say that you only receive that error message when connecting using a public key while connecting using a password works fine. To add a public key for incoming connections, add the following line to ~/.ssh/authorized_keys on the server (create the file if it doesn't exist):

ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAx+KoPhVxfBrnN8cFb+hG9MveY0cfNpn9mAcNhsfkEvxeG2EqLRYtaXUBXPgl3uILvXYbqG7HSBq/kZe/AICn/aK89rCGAozEepdeaYmy9EtmfPU8pFgTrgMils8X6b5kPPxCBZ2pfeL/q4SUke+/xpV1x98py6PHM8VmJaBciqvaa89QLvWf3IUuxm7798WvGUPlSMtuE2wnYsyJ4W65nBCs4PCROpaPmcmqiP0VF+Vm5vC3W/F00PC1w3R3BMdDoS2VJj7jQTR1Ralbn9cM185/pZY8lvkX4lEQMJvvwRM1Oy/g+J7+RbPR/XTrrRmKQq5mnWU0ICV5qvTnsc+Lyw==

Note that this:

1. Starts with ssh-rsa
   


2. Has no line breaks in the middle of it (remove them if present)


3. Should be the public key (should start with AAAAB3) you're currently using (replace it with the new one if you've generated a new one)

This is a bit different format of the keys. Proper way to convert one to the other is to use ssh-keygen:

 ssh-keygen -i -f /path/to/public.key

The result for your key looks like this:

ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAx+KoPhVxfBrnN8cFb+hG9MveY0cfNpn9mAcNhsfkEvxeG2EqLRYtaXUBXPgl3uILvXYbqG7HSBq/kZe/AICn/aK89rCGAozEepdeaYmy9EtmfPU8pFgTrgMils8X6b5kPPxCBZ2pfeL/q4SUke+/xpV1x98py6PHM8VmJaBciqvaa89QLvWf3IUuxm7798WvGUPlSMtuE2wnYsyJ4W65nBCs4PCROpaPmcmqiP0VF+Vm5vC3W/F00PC1w3R3BMdDoS2VJj7jQTR1Ralbn9cM185/pZY8lvkX4lEQMJvvwRM1Oy/g+J7+RbPR/XTrrRmKQq5mnWU0ICV5qvTnsc+Lyw==

Therefore on the single line with key identifier ssh-rsa, instead of the one you used. Also the newlines matter (there can be none).