X

Using GlassFish v3.1 SSH Provisioning Commands

Yamini Kalyandurga
Consulting Member of Technical Staff


http-equiv="Content-Type">
Using GlassFish SSH Provisioning Commands

Introduction


SSH provisioning commands in GlassFish v3.1 are used for setting up
hosts that will be used for installing and managing GlassFish
instances. Previously, in GlassFish v2.x, administrator would
explicitly perform the installations on hosts either manually or
through shell scripts and node agent would handle some part of the
provisioning (like creating instance) With the introduction of SSH in
GlassFish v3.1 as the key communication protocol, GlassFish clusters
are now completely
secure. Right from installing the software, life cycle of instances can
be managed securely.



GlassFish SSH provisioning requires some pre requisites mainly due to
the fact that SSH configuration and user management differs based on OS
(Unix, Windows) Though it may appear that SSH configuration is complex,
it is simple if the correct steps are followed. But once setup, things
work like a charm!



GlassFish v3.1 provisioning mainly consists of 5 commands, style="font-style: italic;">setup-ssh, style="font-style: italic;">install-node, style="font-style: italic;">uninstall-node, create-node-ssh,
delete-node-ssh. This blog covers each
of these commands in detail along with trouble shooting tips and
solutions.

SSH pre requisites


There are certain prerequisites before actually getting
down to using these commands. Since user running DAS will act as the
SSH client, most of these prerequisites apply to remote hosts which run
the SSH server (sshd)



cellspacing="2">


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold;">Pre
requisite


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold;">Unix


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold;">Windows
Cygwin


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold;">Windows
MKS


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold;">Comments





















































OpenSSH should be installed on
all machines hosting GlassFish
software
SSH utils/packages are usually
installed as part of base
OS. If not, install the corresponding OpenSSH ssh package for your OS
Install OpenSSH that comes along
with
Cygwin (1.7.6 and
above)

href="http://wikis.sun.com/display/GlassFish/3.1SSH+-+Installing+Windows+Cygwin+sshd">Cygwin
installation
instructions

Install OpenSSH that comes along
with
MKS Toolkit (9.2 and above)

For MKS installation, get software from http://www.mkssoftware.com/ and
follow their instructions


Ensure SSH server (sshd)
is
running on remote hosts. DAS host
will act as the SSH client, so you don't actually need SSH server
running on DAS
Solaris:

$ svcs status ssh

STATE         
STIME    FMRI

online        
Dec_14   svc:/network/ssh:default

online         14:10:10
svc:/network/nfs/status:default




Red Hat Linux:

$ service ssh status

 \* sshd is running




Any other flavor of Unix:

Run 'ps -ef|grep sshd' to figure out if SSH server is running
$ cygrunsrv --query sshd

Service            
:
sshd

Display name        : CYGWIN sshd

Current State       : Running

Controls Accepted   : Stop

Command            
:
/usr/sbin/sshd
-D
$ service query MKSSecureSH

Name:           MKS
Secure Shell Service

Service Type:   WIN32_OWN_PROCESS

Current State:  RUNNING

Controls Accepted:      ACCEPT_STOP

Check Point:    0

Wait Hint:      0

Start Type:     AUTO_START

Error Control:  IGNORE

Path:          
"C:\\Program Files\\MKS Toolkit\\bin\\secshd.exe"

Dependency:     NuTCRACKERService

Dependency:     tcpip

Service Start Name:     LocalSystem


Cygwin/MKS user home directory
should be same as
Windows user home directory. If not, there might be errors while
running some of the GlassFish commands.
Not applicable

Login as Administrator and
modify /etc/passwd to edit
home directory setting for the SSH user.



(Not sure if there is a better way to do this)

Follow tip #1 in href="http://wikis.sun.com/display/GlassFish/3.1SSH+-+MKS+sshd+tips">MKS
SSHD
tips



PATH variable should be
correctly configured

Does not require any explicit
setting

PATH should have Cygwin bin
directory as well as JDK
bin directory.
MKS bin directory would be
automatically added in PATH by
the MKS installer but JDK bin directory would need to be explicitly
added.
It is best to enable this
setting as Administrator user so that
it applies to all users.

Proper file/directory
permissions. Incorrect permissions will cause
authentication failures since SSH server may not be able to read/access
the SSH config files.



User home directory: 755

.ssh directory: 700

authorized_keys file: 644

Use chmod command

Use chmod command

Use chmod command.

chmod on Windows
work differently since Windows files do not have the same attributes as
Unix/Posix

If a key is manually generated
using ssh-keygen and Windows
user home and Cygwin (or MKS) user home is different, then the key
should be copied into Windows user home's .ssh directory. Otherwise it
will cause SSH authentication within GlassFish to fail (ex
create/delete-node-ssh) since GlassFish will look for key under Windows
user home .ssh directory.
Not applicable

Verify Windows user home .ssh
folder's contents.

Verify Windows user home .ssh
folder's contents.


SSHD configuration

Settings can be found in /etc/sshd_config
or /etc/ssh/sshd_config



StrictModes yes

PubkeyAuthentication yes


Settings can be found in /etc/ssh_config
(/cygdrive/c/cygwin/etc/sshd_config)




StrictModes yes

PubkeyAuthentication yes
Programs -> MKS Toolkit ->
Configuration -> Configuration Information



Strict mode: Click on "Secure Shell Service" tab. Go to "Advanced"
setting, click on "Login" tab, uncheck "Strict Modes"

Password auth: Click on "Secure Shell Service" tab and enable "Password
Authentication"



In most cases, the default
settings should work.



Also, some settings require a server restart.




Please see href="http://wikis.sun.com/display/GlassFish/3.1SSHAuthentication">here
on how to use the various SSH authentication schemes in GlassFish.

setup-ssh


Purpose: To setup public key
authentication between DAS host and
instance host(s). You don't need to use this command if you intend to
use password authentication for SSH




The command first checks if a key is available. It also checks if the
key is encrypted. It could also be possible that the SSH password or
key passphrase is stored in aliased form in the domain key store. Such
an alias is generally passed to the command in non-interactive mode
using the --passwordfile option.



Unix: Command generates key
pair as well as propagates it to host.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
asadmin
setup-ssh --sshuser yb113654 caitanya.india.sun.com

SSH key not found for user yb113654

Would you like to generate a SSH key pair (without a key passphrase)
for yb113654 to access [caitanya.india.sun.com]? [y/n]: y

Enter SSH password for yb113654@caitanya.india.sun.com>

/usr/bin/ssh-keygen successfully generated the identification
/home/yamini/.ssh/id_rsa

Copied keyfile /home/yamini/.ssh/id_rsa.pub to
yb113654@caitanya.india.sun.com

Successfully connected to yb113654@caitanya.india.sun.com using keyfile
/home/yamini/.ssh/id_rsa

Command setup-ssh executed successfully.






Windows Cygwin: Key already
exists in following case. So the command just copies the key to remote
host.
cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
asadmin
setup-ssh underpass.india.sun.com

Enter SSH password for Yamini@underpass.india.sun.com>

Copied keyfile C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa.pub to
Yamini@underpass.india.sun.com

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Command setup-ssh executed successfully.






Windows Cygwin: This example
demonstrates the key setup when Windows user home and Cygwin user home
is not same. Note that it requires some extra steps. Hence, it is best
to keep the home directories same.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
rm
-rf ~/.ssh



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ rm -rf /cygdrive/c/Documents\\ and\\ Settings/Yamini/.ssh



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin setup-ssh underpass.india.sun.com

SSH key not found for user Yamini

Would you like to generate a SSH key pair (without a key passphrase)
for Yamini to access [underpass.india.sun.com]? [y/n]: y

Enter SSH password for Yamini@underpass.india.sun.com>

Created directory C:\\Documents and Settings\\Yamini\\.ssh

ssh-keygen successfully generated the identification C:\\Documents and
Settings\\Yamini\\.ssh\\id_rsa

Copied keyfile C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa.pub to
Yamini@underpass.india.sun.com

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Command setup-ssh executed successfully.

Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ ls ~/.ssh

ls: cannot access /home/Yamini/.ssh: No such file or directory



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ ls /cygdrive/c/Documents\\ and\\ Settings/Yamini/.ssh

id_rsa  id_rsa.pub



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ cp -r /cygdrive/c/Documents\\ and\\ Settings/Yamini/.ssh ~/.ssh



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$







cellspacing="2">


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold; font-style: italic;">Tip:style="font-style: italic;"> If you want to run ssh commands manually
and Windows home is
different from Cygwin user home, the key needs to be copied manually to
Cygwin user home (see last command in above example). This is because
ssh commands will search for key in
Cygwin user home .ssh directory.







Windows Cygwin:
Non-interactive mode of key generation using clear text SSH password.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">C:\\cygwin\\home\\Yamini\\gf\\glassfish3\\glassfish\\bin>echo
AS_ADMIN_SSHPASSWORD=ssh-user-password
>/tmp/pass.txt

C:\\cygwin\\home\\Yamini\\gf\\glassfish3\\glassfish\\bin>asadmin
--passwordfile ./pass.txt setup-ssh --sshuser Yamini --generatekey
underpass.india.sun.com

Copied keyfile C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa.pub to
Yamini@underpass.india.sun.com

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Command setup-ssh executed successfully.







install-node


Purpose:  To install GlassFish
software on remote host(s). Either
password or key auth (with/without passphrase) can be used for
authentication.




The command first tries to use key authentication  and checks if a
key is available. In case key authentication fails, it falls back to
using password authentication. It also checks if the
key is encrypted. It could also be possible that the SSH password or
key passphrase is stored in aliased form in the domain key store. Such
an alias is generally passed to the command in non-interactive mode
using the --passwordfile option.



Windows Cygwin: Using clear
text key passphrase.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
echo
AS_ADMIN_SSHKEYPASSPHRASE=foo123>pass1.txt



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin --passwordfile pass.txt install-node --installdir
/home/Yamini/gf underpass.india.sun.com

Created installation zip
C:\\cygwin\\home\\Yamini\\gf\\glassfish3\\glassfish\\bin\\glassfish6762824754722728821.zip

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Copying
C:\\cygwin\\home\\Yamini\\gf\\glassfish3\\glassfish\\bin\\glassfish6762824754722728821.zip
(81745171
bytes) to underpass.india.sun.com:/home/Yamini/gf

Installing glassfish6762824754722728821.zip into
underpass.india.sun.com:/home/Yamini/gf

Removing
underpass.india.sun.com:/home/Yamini/gf/glassfish6762824754722728821.zip

Fixing file permissions of all files under
underpass.india.sun.com:/home/Yamini/gf/bin

Command install-node executed successfully.



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$






Windows Cygwin: Using key
passphrase alias.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
asadmin
list-password-aliases

Nothing to list

Command list-password-aliases executed successfully.



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin create-password-alias

Enter the value for the aliasname operand> ssh-keypass

Enter the alias password>

Enter the alias password again>

Command create-password-alias executed successfully.



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin list-password-aliases

ssh-keypass

Command list-password-aliases executed successfully.



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ echo AS_ADMIN_SSHKEYPASSPHRASE=\\$\\{ALIAS=ssh-keypass\\}>pass2.txt



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ cat pass2.txt

AS_ADMIN_SSHKEYPASSPHRASE=${ALIAS=ssh-keypass}



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin --passwordfile pass2.txt install-node --installdir
/home/Yamini/gf underpass.india.sun.com

Password is aliased. To obtain the real password, enter master password
for domain1's key store>

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Created installation zip
C:\\cygwin\\home\\Yamini\\gf\\glassfish3\\glassfish\\bin\\glassfish2993564184607483536.zip

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

GlassFish is already installed on underpass.india.sun.com under
/home/Yamini/gf.



Command install-node executed successfully.







create-node-ssh


Purpose: creates a node for a
specific host with the GlassFish software installation identified by an
installation path. Also contains the connection information for the
node.




Windows Cygwin: create-node-ssh
will fail if it cannot find a valid installation on remote host.



cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">Yamini@ducati
~/gf/glassfish3/glassfish/bin

$ asadmin create-node-ssh --nodehost underpass.india.sun.com --sshuser
Yamini --installdir c:/home/Yamini/gf/glassfish3 n1

remote failure: Warning: some parameters appear to be invalid.

SSH node not created. To force creation of the node with these
parameters rerun the command using the --force option.

Invalid install directory: could not find
c:/home/Yamini/gf/glassfish3/glassfish/modules/admin-cli.jar on
underpass.india.sun.com

Command create-node-ssh failed.






Using create-node-ssh --install to provision the remote
host. This is the same as running install-node first and
then create-node-ssh



cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">Yamini@ducati
~/gf/glassfish3/glassfish/bin

$ asadmin create-node-ssh --nodehost underpass.india.sun.com
--installdir /home/Yamini/gf/glassfish3 --install n1

Successfully installed GlassFish on underpass.india.sun.com.

Command create-node-ssh executed successfully.



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin list-nodes

localhost-domain1  CONFIG  localhost

n1  SSH  underpass.india.sun.com

Command list-nodes executed successfully.







cellspacing="2">


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-weight: bold; font-style: italic;">Note:style="font-style: italic;"> In case public key authentication is
preferred, setup-ssh should be run prior to create-node-ssh, to setup
the communication.






delete-node-ssh


Windows cygwin: Deleting the
node configuration along with uninstalling the software.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">Yamini@ducati
~/gf/glassfish3/glassfish/bin

$ asadmin delete-node-ssh --uninstall n1

Successfully un-installed GlassFish on underpass.india.sun.com.

Command delete-node-ssh executed successfully.






uninstall-node


Purpose:  To un-install
GlassFish software on remote host(s).
Either
password or key auth (with/without passphrase) can be used for
authentication.




The command first tries to use key authentication  and checks if a
key
is available. In case key authentication fails, it falls back to using
password authentication. It also checks if the
key is encrypted. It could also be possible that the SSH password or
key passphrase is stored in aliased form in the domain key store. Such
an alias is generally passed to the command in non-interactive mode
using the --passwordfile option.



Windows Cygwin:
Non-interactive and aliased key passphrase.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">Yamini@ducati
~/gf/glassfish3/glassfish/bin

$ echo AS_ADMIN_MASTERPASSWORD=changeit>>pass2.txt



Yamini@ducati ~/gf/glassfish3/glassfish/bin

$ asadmin --passwordfile pass2.txt uninstall-node --installdir
/home/Yamini/gf underpass.india.sun.com

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Command uninstall-node executed successfully.






Solaris: Interactive mode and
aliased key passphrase.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
asadmin
uninstall-node --sshuser yamini1 --installdir
/space/yamini1/gf/glassfish3 sunone172.india.sun.com

Key /home/yamini/.ssh/id_rsa is encrypted

Enter key passphrase>

Enter SSH password for yamini1@sunone172.india.sun.com>

Authenticating with password <concealed>

Command uninstall-node executed successfully.






cellspacing="2">


style="vertical-align: top; background-color: rgb(255, 204, 153);">style="font-style: italic;">Note:
setup-ssh, install-node, uninstall-node are local commands whereas
create-node-ssh is a remote command.




Troubleshooting:

  • uninstall-node fails with sftp error.

cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">Yamini@ducati
~/gf/glassfish3/glassfish/bin

$ asadmin --passwordfile pass2.txt uninstall-node --installdir
/home/Yamini/gf/glassfish3 underpass.india.sun.com

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

Successfully connected to Yamini@underpass.india.sun.com using keyfile
C:\\Documents and Settings\\Yamini\\.ssh\\id_rsa

com.trilead.ssh2.SFTPException: Failure (SSH_FX_FAILURE: An error
occurred, but no specific error code exists to describe the failure.)

Command uninstall-node failed.





Reason:
One of the directories within installdir is in use on remote
host, hence removal fails.



Solution: Make sure installdir
(and its children) are not in use before
uninstalling.


  • setup-ssh fails on Windows MKS


cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">[C:/yamini/gf/glassfish3/glassfish/bin] 
asadmin
setup-ssh
crux.india.sun.com

Key C:\\Documents and Settings\\Administrator\\.ssh\\id_rsa is encrypted

Enter key passphrase>

Enter SSH password for Yamini@crux.india.sun.com>

SSH key setup failed: SSH password authentication failed for user
Yamini on host underpass.india.sun.com

Command setup-ssh failed.





Reason:
Password authentication is disabled in MKS ssh server on remote
host.



Solution: Enable password
authentication using MKS configuration tool.
Note that enabling/disabling password authentication requires MKS SSH
server restart.


  • setup-ssh fails during connection verification


cellspacing="2">


style="vertical-align: top; background-color: rgb(153, 153, 153);">$
asadmin
setup-ssh --sshuser yamini1 sunone172.india.sun.com

Key
/home/yamini/.ssh/id_rsa is encrypted

Enter key passphrase>

Enter SSH password for yamini1@sunone172.india.sun.com>

Copied keyfile /home/yamini/.ssh/id_rsa.pub to
yamini1@sunone172.india.sun.com

Connection verification failed.

Command setup-ssh failed.





Reason:
authorized_keys file permissions are too loose on remote host.



Solution: Follow step 5 in pre
requisites section above.






Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.