Oxidized 0.29.1 with net-ssh >7.2 crashes when offered rsa-sha2-512

[Atroskelis]

Oxidized 0.29.1 with net-ssh >7.2 crashes when offered rsa-sha2-512

[Atroskelis]

will also make a ticket to net-ssh as their code makes the error not oxidized itself. also does same error with sample code.

[ytti]

It seems to me that someone is offering very new type of keys, newer than ed25519, and these new key types are not supproted by net-ssh.

Man page for 'ssh-keygen' says this:

-t dsa | ecdsa | ecdsa-sk | ed25519 | ed25519-sk | rsa
               Specifies the type of key to create.  The possible values are “dsa”, “ecdsa”, “ecdsa-sk”, “ed25519”, “ed25519-sk”, or “rsa”.

But it still accepts to generate rsa-sha2-256:

╰─ ssh-keygen -t rsa-sha2-256
Generating public/private rsa-sha2-256 key pair.
Enter file in which to save the key (/home/ytti/.ssh/id_rsa): ^C

There is new RFC RFC8332 which introduces rsa-sha2-256and rsa-sha2-512, which net-ssh does not understand.

Just to be sure, you do not have these new type of rsa-sha2-xx keys generated in ~/.ssh? It is fortigate which is offering these new type of keys, which breaks net-ssh? If fortigate offers those key types, can fortigate be changed? Can you tell fortigate to offer ed25519 or rsa or dsa?
For example, can you jump to fortigate shell, and delete the rsa-sha2-X keys?

[ytti]

I can't reproduce this problem on my server, I added to /etc/ssh/sshd_config:

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa2-512_key ## thisline

And it didn't break net-ssh, I can use net-ssh just fine to interact with the server.

If I ask buffer.rb to print types it sees:

╰─ ./ssh.rb
ecdsa-sha2-nistp256
ecdsa-sha2-nistp256
ecdsa-sha2-nistp256
ssh-ed25519
ssh-rsa
ssh-rsa
ssh-ed25519

I don't see ssh-rsa2-512 here offered. But I do see with ssh -vvvvv server this:

debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

I suspect fortinet may have misunderstood the RFC, and is offering wrong string in a place where it should offer just rsa-sha.

At least because I can't reproduce, I can't fix, if this was legitimate problem in net-ssh, it seems like I might be able to add support for those hostkeys.

As I can't reproduce it, I'd like somieone to try to change line when /^ssh-rsa$/ into when /^sha-rsa$/, /^rsa-sha2-(256|512)$/
If that works, I suspect we have some support that fortinet is just marking the key types incorrectly.

[Atroskelis]

From my understanding Fortigate also offers ssh-ed25519 as seen with ssh -vvvv but only the first option is taken into account

debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,ssh-ed25519
debug2: ciphers ctos: aes256-ctr,[email protected]
debug2: ciphers stoc: aes256-ctr,[email protected]
debug2: MACs ctos: hmac-sha2-256
debug2: MACs stoc: hmac-sha2-256

I could try to alter the ssh config on the forti but i doubt i can not make it offer sha2-512

client-ftw (global) # set ssh
ssh-enc-algo    Select one or more SSH ciphers.
ssh-kex-algo    Select one or more SSH kex algorithms.
ssh-mac-algo    Select one or more SSH MAC algorithms.
 
client-ftw (global) # set ssh-enc-algo 
aes256-ctr                aes256-ctr
[email protected]    [email protected]
 
client-ftw (global) # set ssh-kex-algo 
diffie-hellman-group-exchange-sha256    diffie-hellman-group-exchange-sha256
[email protected]            [email protected]
ecdh-sha2-nistp256                      ecdh-sha2-nistp256
ecdh-sha2-nistp384                      ecdh-sha2-nistp384
ecdh-sha2-nistp521                      ecdh-sha2-nistp521
 
client-ftw (global) # set ssh-mac-algo 
hmac-sha2-256                    hmac-sha2-256
[email protected]    [email protected]
hmac-sha2-512                    hmac-sha2-512
[email protected]    [email protected]

client-ftw (global) # set strong-crypto disable 

client-ftw (global) # set ssh-enc-algo  
[email protected]    [email protected]
aes128-ctr                       aes128-ctr
aes192-ctr                       aes192-ctr
aes256-ctr                       aes256-ctr
arcfour256                       arcfour256
arcfour128                       arcfour128
aes128-cbc                       aes128-cbc
3des-cbc                         3des-cbc
blowfish-cbc                     blowfish-cbc
cast128-cbc                      cast128-cbc
aes192-cbc                       aes192-cbc
aes256-cbc                       aes256-cbc
arcfour                          arcfour
[email protected]      [email protected]
[email protected]           [email protected]
[email protected]           [email protected]
 
client-ftw (global) # set ssh-kex-algo 
diffie-hellman-group14-sha1             diffie-hellman-group14-sha1
diffie-hellman-group-exchange-sha1      diffie-hellman-group-exchange-sha1
diffie-hellman-group-exchange-sha256    diffie-hellman-group-exchange-sha256
[email protected]            [email protected]
ecdh-sha2-nistp256                      ecdh-sha2-nistp256
ecdh-sha2-nistp384                      ecdh-sha2-nistp384
ecdh-sha2-nistp521                      ecdh-sha2-nistp521
 
client-ftw (global) # set ssh-mac-algo 
hmac-md5                          hmac-md5
[email protected]          [email protected]
hmac-md5-96                       hmac-md5-96
[email protected]       [email protected]
hmac-sha1                         hmac-sha1
[email protected]         [email protected]
hmac-sha2-256                     hmac-sha2-256
[email protected]     [email protected]
hmac-sha2-512                     hmac-sha2-512
[email protected]     [email protected]
hmac-ripemd160                    hmac-ripemd160
[email protected]        [email protected]
[email protected]    [email protected]
[email protected]               [email protected]
[email protected]              [email protected]
[email protected]           [email protected]
[email protected]          [email protected]

[ytti]

From my understanding Fortigate also offers ssh-ed25519 as seen with ssh -vvvv but only the first option is taken into account

debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,ssh-ed25519
debug2: ciphers ctos: aes256-ctr,[email protected]
debug2: ciphers stoc: aes256-ctr,[email protected]
debug2: MACs ctos: hmac-sha2-256
debug2: MACs stoc: hmac-sha2-256

I could try to alter the ssh config on the forti but i doubt i can not make it offer sha2-512

My openssh server is offering rsa-sha2-512, and everything works just fine with net-ssh. So I think the problem may be, that rsa-sha2-512 and 256 keyformat is 'ssh-rsa', but fortigate incorrectly claims the keyformat is 'rsa-sha2-512', which is not something you see in the openssh debug shown, as it's not hostkey problem at all, but hostkey format problem.
If it was hostkey problem, I should see the problem with net-ssh + openssh, and I should be able to fix it.

Please do try to modify the buffer.rb to include ,/^rsa-sha2-(256|512)$/, to see if that magically ´fixes´the problem.

[Atroskelis]

Modified buffer.rb in 2 places replacing when /^ssh-rsa$/ into when /^sha-rsa$/, /^rsa-sha2-(256|512)$/

result running test code

7.1.0
/usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/buffer.rb:342:in `read_keyblob': unsupported key type `ssh-rsa' (NotImplementedError)
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/buffer.rb:248:in `read_key'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/key_factory.rb:102:in `load_data_public_key'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/key_factory.rb:83:in `load_public_key'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/key_manager.rb:267:in `block in load_identities'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/key_manager.rb:264:in `map'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/key_manager.rb:264:in `load_identities'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/key_manager.rb:110:in `each_identity'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/methods/publickey.rb:18:in `authenticate'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/session.rb:87:in `block in authenticate'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/session.rb:71:in `each'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh/authentication/session.rb:71:in `authenticate'
        from /usr/local/share/gems/gems/net-ssh-7.1.0/lib/net/ssh.rb:259:in `start'
        from test.rb:7:in `<main>'

[Atroskelis]

Solution works with

when /^ssh-rsa$/, /^rsa-sha2-(256|512)$/

[ytti]

Solution works with

when /^ssh-rsa$/, /^rsa-sha2-(256|512)$/

I suspect this means, fortinet is doing a wrong thing, it is using wrong string to describe key format. And in all of these cases the key format is just 'ssh-rsa'.

But I'm not 100% confident.

Possibly if you kill ed key from fortinet, you cannot login to it with openssh. Which would further support that the rsa-sha2 is broken in fortinet.

[Atroskelis]

I unfortunately cannot remove the ed key, in the next software version they will remove the sha key entirely and leave only the ed key which works.

[ytti]

I unfortunately cannot remove the ed key, in the next software version they will remove the sha key entirely and leave only the ed key which works.

Ok. For now, I'm thinking this is purely Fortinet incorrectly implementing RSA, and the fix proposed is non-sensical against any non-broken implementation. But I don't have 100% confidence.

Others have concluded same as I have:
https://forum.rebex.net/22697/exchange-issues-negotiation-failed-with-fortigate-firewalls?show=22698#a22698 - that Fortinet is just sending nonsensical information about RSA, and it can't possibly work with any RSA compliant client, only problem is that net-ssh doesn't fail gracefully to working EC.

[Atroskelis]

Here is the SSH app debug on the firewall side if it helps

SSH: fd 7 is not O_NONBLOCK
SSH: Forked child 17227.
SSH: Client protocol version 2.0; client software version Ruby/Net::SSH_7.1.0 x86_64-linux
SSH: no match: Ruby/Net::SSH_7.1.0 x86_64-linux
SSH: Enabling compatibility mode for protocol 2.0
SSH: Local version string SSH-2.0-6PB-lc-DHYPxG
SSH: fd 7 setting O_NONBLOCK
SSH: Proposal: 0, Ciphers: 'diffie-hellman-group-exchange-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521'
SSH: Proposal: 2, Ciphers: 'aes256-ctr,[email protected]'
SSH: Proposal: 3, Ciphers: 'aes256-ctr,[email protected]'
SSH: Proposal: 4, Ciphers: 'hmac-sha1'
SSH: Proposal: 5, Ciphers: 'hmac-sha1'
SSH: list_hostkey_types: rsa-sha2-512,ssh-ed25519
SSH: SSH2_MSG_KEXINIT sent
SSH: SSH2_MSG_KEXINIT received
SSH: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
SSH: kex_parse_kexinit: rsa-sha2-512,ssh-ed25519
SSH: kex_parse_kexinit: aes256-ctr,[email protected]
SSH: kex_parse_kexinit: aes256-ctr,[email protected]
SSH: kex_parse_kexinit: hmac-sha1
SSH: kex_parse_kexinit: hmac-sha1
SSH: kex_parse_kexinit: none,[email protected]
SSH: kex_parse_kexinit: none,[email protected]
SSH: kex_parse_kexinit: 
SSH: kex_parse_kexinit: 
SSH: kex_parse_kexinit: first_kex_follows 0 
SSH: kex_parse_kexinit: reserved 0 
SSH: kex_parse_kexinit: ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
SSH: kex_parse_kexinit: ssh-rsa,[email protected],[email protected],[email protected],ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,[email protected],ssh-rsa-cert-v00
SSH: kex_parse_kexinit: aes256-ctr,aes192-ctr,aes128-ctr
SSH: kex_parse_kexinit: aes256-ctr,aes192-ctr,aes128-ctr
SSH: kex_parse_kexinit: [email protected],[email protected],hmac-sha2-512,hmac-sha2-256,hmac-sha1
SSH: kex_parse_kexinit: [email protected],[email protected],hmac-sha2-512,hmac-sha2-256,hmac-sha1
SSH: kex_parse_kexinit: none,[email protected],zlib
SSH: kex_parse_kexinit: none,[email protected],zlib
SSH: kex_parse_kexinit: 
SSH: kex_parse_kexinit: 
SSH: kex_parse_kexinit: first_kex_follows 0 
SSH: kex_parse_kexinit: reserved 0 
SSH: kex: host key algorithm: rsa-sha2-512
SSH: kex: client->server aes256-ctr hmac-sha1 none
SSH: kex: server->client aes256-ctr hmac-sha1 none
SSH: expecting SSH2_MSG_KEX_ECDH_INIT
SSH: set_newkeys: mode 1
SSH: SSH2_MSG_NEWKEYS sent
SSH: expecting SSH2_MSG_NEWKEYS
SSH: set_newkeys: mode 0
SSH: SSH2_MSG_NEWKEYS received
SSH: KEX done
SSH: userauth-request for user support service ssh-connection method none
SSH: attempt 0 failures 0
SSH: input_userauth_request: setting up authctxt for support
SSH: input_userauth_request: try method none
SSH: error: Could not get shadow information for support
SSH: Unrecognized authentication method name: none
SSH: userauth_finish: failure partial=0 next methods="publickey,password"
SSH: userauth-request for user support service ssh-connection method publickey
SSH: attempt 1 failures 0
SSH: input_userauth_request: try method publickey
SSH: userauth_pubkey: unsupported public key algorithm: [email protected]
SSH: userauth_pubkey: authenticated 0 pkalg [email protected]
SSH: userauth_finish: failure partial=0 next methods="publickey,password"
SSH: userauth-request for user support service ssh-connection method publickey
SSH: attempt 2 failures 1
SSH: input_userauth_request: try method publickey
SSH: test whether pkalg/pkblob are acceptable
SSH: temporarily_use_uid: 0/0 (e=0/0)
SSH: trying public key file /etc/ssh/support_auth_keys
SSH: Could not open authorized keys '/etc/ssh/support_auth_keys': No such file or directory
SSH: restore_uid: 0/0
SSH: temporarily_use_uid: 0/0 (e=0/0)
SSH: trying public key file (null)/.ssh/authorized_keys2
SSH: Could not open authorized keys '(null)/.ssh/authorized_keys2': No such file or directory
SSH: restore_uid: 0/0
SSH: userauth_pubkey: authenticated 0 pkalg [email protected]
SSH: userauth_finish: failure partial=0 next methods="publickey,password"
SSH: userauth-request for user support service ssh-connection method publickey
SSH: attempt 3 failures 2
SSH: input_userauth_request: try method publickey
SSH: userauth_pubkey: unsupported public key algorithm: rsa-sha2-256
SSH: userauth_pubkey: authenticated 0 pkalg rsa-sha2-256
SSH: userauth_finish: failure partial=0 next methods="publickey,password"
SSH: userauth-request for user support service ssh-connection method publickey
SSH: attempt 4 failures 3
SSH: input_userauth_request: try method publickey
SSH: test whether pkalg/pkblob are acceptable
SSH: temporarily_use_uid: 0/0 (e=0/0)
SSH: trying public key file /etc/ssh/support_auth_keys
SSH: Could not open authorized keys '/etc/ssh/support_auth_keys': No such file or directory
SSH: restore_uid: 0/0
SSH: temporarily_use_uid: 0/0 (e=0/0)
SSH: trying public key file (null)/.ssh/authorized_keys2
SSH: Could not open authorized keys '(null)/.ssh/authorized_keys2': No such file or directory
SSH: restore_uid: 0/0
SSH: userauth_pubkey: authenticated 0 pkalg ssh-rsa
SSH: Failed publickey for support from 10.138.1.41 port 39778 ssh2
SSH: userauth_finish: failure partial=0 next methods="publickey,password"

[Atroskelis]

It was concluded that problem was on end device side. And it was mitigated.

Many thanks to everyone who helped

[KevinTang8888]

looks like you try to troubleshoot using public/private key for ssh connection, not the password method.
Did you load the paired private key at the SSH client device?

[KevinTang8888]

SSH: input_userauth_request: try method publickey