When OpenSSH server on the remote host is v8.8 or later, the client for Windows cannot authenticate by using 'key-based authentication with a SSH agent' .

This is triggered by changes in OpenSSH v8 which disables rsa-sha1 algorithm and requires RSA keys using SHA2 signature algorithm. See  changes in https://www.openssh.com/txt/release-8.8 for more details.

Support for authentication by SSH agent on Windows relies on nxssh, which doesn't support rsa-sha2 keys.

As a workaround, use ECDSA keys instead of RSA keys.

To generate them on Linux:

ssh-keygen -t ecdsa -m PEM