This is the first time I am setting up a new 11.2.0.2 cluster with the automatic SSH setup. Until now, I ensured user equivalence by copying ssh RSA and DSA manually to all cluster nodes. For two nodes that’s not too bad, but recently someone asked a question around a 28 (!) node cluster on a mailing list I am subscribing to. So that’s when I think the whole process gets a bit too labour intensive.
So setting up user equivalence using a script may be the solution. You can also use OUI to do the same, but I like to run “cluvfy stage -post hwos” to check everything is ok before even thinking about executing ./runInstaller.
Here’s the output of a session, my 2 cluster nodes are acfsprodnode1 and acfsprodnode2 (yes, they are for 11.2 ACFS replication and encryption testing). I am using the grid user as the owner of Grid Infrastructure, and oracle to own the RDBMS binaries. Start by navigating to the location where you unzipped the Grid Infrastructure patch file. Then change into directoy “sshsetup” and run the command:
[grid@acfsprdnode1 sshsetup]$ ./sshUserSetup.sh Please specify a valid and existing cluster configuration file. Either user name or host information is missing Usage ./sshUserSetup.sh -user <user name> [ -hosts "<space separated hostlist>" | -hostfile <absolute path of cluster configuration file> ] [ -advanced ] [ -verify] [ -exverify ] [ -logfile <desired absolute path of logfile> ] [-confirm] [-shared] [-help] [-usePassphrase] [-noPromptPassphrase]
Next execute the command, I opted for option noPromptPassphrase, as I don’t use them for the key.
[grid@acfsprdnode1 sshsetup]$ ./sshUserSetup.sh -user grid -hosts "acfsprdnode1 acfsprdnode2" -noPromptPassphrase The output of this script is also logged into /tmp/sshUserSetup_2010-12-22-15-39-18.log Hosts are acfsprdnode1 acfsprdnode2 user is grid Platform:- Linux Checking if the remote hosts are reachable PING acfsprdnode1.localdomain (192.168.99.100) 56(84) bytes of data. 64 bytes from acfsprdnode1.localdomain (192.168.99.100): icmp_seq=1 ttl=64 time=0.017 ms 64 bytes from acfsprdnode1.localdomain (192.168.99.100): icmp_seq=2 ttl=64 time=0.019 ms 64 bytes from acfsprdnode1.localdomain (192.168.99.100): icmp_seq=3 ttl=64 time=0.017 ms 64 bytes from acfsprdnode1.localdomain (192.168.99.100): icmp_seq=4 ttl=64 time=0.017 ms 64 bytes from acfsprdnode1.localdomain (192.168.99.100): icmp_seq=5 ttl=64 time=0.018 ms --- acfsprdnode1.localdomain ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 3999ms rtt min/avg/max/mdev = 0.017/0.017/0.019/0.004 ms PING acfsprdnode2.localdomain (192.168.99.101) 56(84) bytes of data. 64 bytes from acfsprdnode2.localdomain (192.168.99.101): icmp_seq=1 ttl=64 time=0.331 ms 64 bytes from acfsprdnode2.localdomain (192.168.99.101): icmp_seq=2 ttl=64 time=0.109 ms 64 bytes from acfsprdnode2.localdomain (192.168.99.101): icmp_seq=3 ttl=64 time=0.324 ms 64 bytes from acfsprdnode2.localdomain (192.168.99.101): icmp_seq=4 ttl=64 time=0.256 ms 64 bytes from acfsprdnode2.localdomain (192.168.99.101): icmp_seq=5 ttl=64 time=0.257 ms --- acfsprdnode2.localdomain ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4000ms rtt min/avg/max/mdev = 0.109/0.255/0.331/0.081 ms Remote host reachability check succeeded. The following hosts are reachable: acfsprdnode1 acfsprdnode2. The following hosts are not reachable: . All hosts are reachable. Proceeding further... firsthost acfsprdnode1 numhosts 2 <span style="color: #ff0000;">The script will setup SSH connectivity from the host acfsprdnode1 to all</span> <span style="color: #ff0000;">the remote hosts</span>. After the script is executed, the user can use SSH to run commands on the remote hosts or copy files between this host acfsprdnode1 and the remote hosts without being prompted for passwords or confirmations. NOTE 1: As part of the setup procedure, this script will use ssh and scp to copy files between the local host and the remote hosts. Since the script does not store passwords, you may be prompted for the passwords during the execution of the script whenever ssh or scp is invoked. NOTE 2: AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEDGES TO THESE directories. Do you want to continue and let the script make the above mentioned changes (yes/no)? <span style="color: #ff0000;">yes</span> The user chose yes User chose to skip passphrase related questions. Creating .ssh directory on local host, if not present already Creating authorized_keys file on local host Changing permissions on authorized_keys to 644 on local host Creating known_hosts file on local host Changing permissions on known_hosts to 644 on local host Creating config file on local host If a config file exists already at /home/grid/.ssh/config, it would be backed up to /home/grid/.ssh/config.backup. Removing old private/public keys on local host Running SSH keygen on local host with empty passphrase Generating public/private rsa key pair. Your identification has been saved in /home/grid/.ssh/id_rsa. Your public key has been saved in /home/grid/.ssh/id_rsa.pub. The key fingerprint is: de:e3:66:fa:16:e8:6e:36:fd:c5:e3:77:75:07:9a:b0 grid@acfsprdnode1 Creating .ssh directory and setting permissions on remote host acfsprdnode1 THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT. The script would create ~grid/.ssh/config file on remote host acfsprdnode1. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup. The user may be prompted for a password here since the script would be running SSH on host acfsprdnode1. Warning: Permanently added 'acfsprdnode1,192.168.99.100' (RSA) to the list of known hosts. grid@acfsprdnode1's password: Done with creating .ssh directory and setting permissions on remote host acfsprdnode1. Creating .ssh directory and setting permissions on remote host acfsprdnode2 THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT. The script would create ~grid/.ssh/config file on remote host acfsprdnode2. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup. The user may be prompted for a password here since the script would be running SSH on host acfsprdnode2. Warning: Permanently added 'acfsprdnode2,192.168.99.101' (RSA) to the list of known hosts. grid@acfsprdnode2's password: Done with creating .ssh directory and setting permissions on remote host acfsprdnode2. Copying local host public key to the remote host acfsprdnode1 The user may be prompted for a password or passphrase here since the script would be using SCP for host acfsprdnode1. grid@acfsprdnode1's password: Done copying local host public key to the remote host acfsprdnode1 Copying local host public key to the remote host acfsprdnode2 The user may be prompted for a password or passphrase here since the script would be using SCP for host acfsprdnode2. grid@acfsprdnode2's password: Done copying local host public key to the remote host acfsprdnode2 cat: /home/grid/.ssh/known_hosts.tmp: No such file or directory cat: /home/grid/.ssh/authorized_keys.tmp: No such file or directory SSH setup is complete. ------------------------------------------------------------------------ Verifying SSH setup =================== The script will now run the date command on the remote nodes using ssh to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP, THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR PASSWORDS. If you see any output other than date or are prompted for the password, ssh is not setup correctly and you will need to resolve the issue and set up ssh again. The possible causes for failure could be: 1. The server settings in /etc/ssh/sshd_config file do not allow ssh for user grid. 2. The server may have disabled public key based authentication. 3. The client public key on the server may be outdated. 4. ~grid or ~grid/.ssh on the remote host may not be owned by grid. 5. User may not have passed -shared option for shared remote users or may be passing the -shared option for non-shared remote users. 6. If there is output in addition to the date, but no password is asked, it may be a security alert shown as part of company policy. Append the additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file. ------------------------------------------------------------------------ --acfsprdnode1:-- Running /usr/bin/ssh -x -l grid acfsprdnode1 date to verify SSH connectivity has been setup from local host to acfsprdnode1. IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR. Wed Dec 22 15:40:10 GMT 2010 ------------------------------------------------------------------------ --acfsprdnode2:-- Running /usr/bin/ssh -x -l grid acfsprdnode2 date to verify SSH connectivity has been setup from local host to acfsprdnode2. IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR. Wed Dec 22 15:40:10 GMT 2010 ------------------------------------------------------------------------ SSH verification complete. [grid@acfsprdnode1 sshsetup]$ ssh acfsprdnode1 hostname acfsprdnode1 [grid@acfsprdnode1 sshsetup]$ ssh acfsprdnode2 hostname acfsprodnode2 [grid@acfsprdnode1 sshsetup]$
Nice! That’s a lot of work taken away from me, and I can start runing cluvfy now to fix problems before OUI warns me about shortcomings on my system.
You should note that per the above output, the script only distributes the local ssh keys to the remote hosts. When in OUI’s cluster node addition screen (6 of 16 in the advanced installation) you still need to click on the “SSH Connectivity” button and then on “Setup” after providing username and password to establish cluster wide user equivalence.
Hi Martin,
After using this method to setup ssh between a two-node cluster, CVU was able to pass the node connectivity tests. However, OUI fails to detect ssh configuration and you still have to do the automatic setup via OU.
My guess is that OUI actually configures for ssh connectivity in both directions whereas the sshUserSetup.sh script only sets up ssh in one direction only.
It does! I initially wanted to update the post some time ago but never got around to do it. Thanks for brining it back to the radar! The section where the script tells you it’s taking the local ssh setup and distributes it across the cluster nodes is now highlighted in red, and also the fact that you still need to set the user equivalence up in OUI.
To setup in two directions add -advanced option
Pingback: Installing Oracle 12.1.0.2 RAC on Oracle Linux 7-part 2 « Martins Blog