Monday Feb 28, 2011

Troubleshooting SSH setup in GlassFish v3.1

Troubleshooting SSH setup in GlassFishv3.1 Centralized administration feature in GlassFish v3.1 uses SSH for provisioning GlassFish nodes and managing cluster (or instance) lifecycle operations securely. Even before DAS can use SSH for communicating with remote hosts, SSH needs to be configured properly. Since SSH server settings vary on different platforms (and even across different flavors of Unix), GlassFish SSH setup command, setup-ssh may fail at times. In this blog, I've tried to capture such failure cases as well as solutions on how to address them.

Before getting into the setup-ssh command, its important to understand SSH authentication schemes. Please check this blog on GlassFish SSH authentication schemes.
 
The asadmin setup-ssh command uses password authentication to copy over the specified (or generated) public key to the remote host. Before exiting, the command also verifies that SSH public key authentication works as expected.

Please note that all of the SSH server settings mentioned below pertain to the settings on the remote host since DAS host is actually the SSH client. SSH server settings is configured through the SSH server configuration file located as follows:

Unix: /etc/ssh/sshd_config or /etc/sshd_config
Windows with Cygwin: From within cygwin terminal, /etc/sshd_config or actual path on file system would be <Cywin-install-dir>/etc/sshd_config
Windows with MKS Toolkit: Programs -> MKS Toolkit -> Configuration -> Configuration Information.

Also note that SSH server needs to be restarted whenever any of the server attributes are changed.
  • Password authentication error
$ asadmin setup-ssh sunone172
Enter SSH password for yamini@sunone172>
SSH key setup failed: SSH password authentication failed for user yamini on host sunone172
Command setup-ssh failed.

Reason: SSH user password entered is incorrect or SSH server is rejecting password authentication.

Solution: Change the 'PasswordAuthentication' setting to 'yes' in SSH server config and restart the SSH server.

Explanation: Assuming that you are typing in the correct password, first check if manual ssh works:

$ ssh sunone172
Permission denied (publickey).

If you see something like above, it means that SSH server is not even considering password authentication.

It can also be possible that password authentication appears to be working manually:

$ ssh sunone172 pwd
Password:
/home/yamini

But there is a catch here, the above appears to be using password authentication but its actually using keyboard-interactive authentication. The point is password authentication is different from keyboard-interactive. In the former case, the (SSH) client reads in the password from user and sends it across to the SSH server whereas in the latter case, the password is challenged from the SSH server. A subtle difference is in the way the prompting happens. Also notice the last debug message from 'ssh -v':

Password:
debug1: Authentications that can continue: publickey,password,keyboard-interactive
Password:
debug1: Authentications that can continue: publickey,password,keyboard-interactive
Password:
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password
yamini@sunone172's password:

First three prompts are from the SSH server whereas the last one is from the SSH client.

The attribute that turns on/off keyboard-interactive is 'ChallengeResponseAuthentication' attribute in SSH server configuration. Here is a nice explanation on the difference between password and keyboard-interactive.

In short, it doesn't matter if keyboard-interactive is turned on or not, what matters for us is the password authentication setting.
  • Connection verification failure
$ asadmin setup-ssh sunone172
SSH key not found for user yamini
Would you like to generate a SSH key pair (without a key passphrase) for yamini to access [sunone172]? [y/n]: y
Enter SSH password for yamini@sunone172>
Created directory /home/yamini/.ssh
/usr/bin/ssh-keygen successfully generated the identification /home/yamini/.ssh/id_rsa
Copied keyfile /home/yamini/.ssh/id_rsa.pub to yamini@sunone172
Connection verification failed.
Command setup-ssh failed.

Reason 1: Public key authentication is disabled on SSH server.

Solution: Change the attribute 'PubkeyAuthentication' to 'yes' in the SSH server configuration file and restart the SSH server.

Reason 2: On remote host, permissions on $HOME or .ssh or authorized_keys file is too loose.

Solution: Either disable strict checking of file permissions by setting 'StrictModes' to 'no' in SSH server config file and restart the SSH server.
          Or make sure that the following permissions are in place:
          $HOME --> 755
          $HOME/.ssh --> 700
          $HOME/.ssh/authorized_keys --> 600 or 644
About

yamini

Search

Top Tags
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today