Martins Blog

Trying to explain complex things in simple terms

Getting started with iSCSI part III-Mounting Storage

Posted by Martin Bach on October 22, 2009

After all the necessary work has been completed on the iSCSI appliance we are now ready to present the storage to the RAC nodes; I am using RHEL 5 for these. RHEL 5 and clones require a specific RPM to manage iSCSI storage, which is part of the distribution. Note that the package has a slightly different name in RHEL 4,and the steps to discover storage are slightly different to the ones presented here.

Installing the iSCSI initiator

The initiator, as the name implies, is required to initiate an iSCSI connections. You need to install the initiator if not done so already:

# rpm -ihv iscsi-initiator-utils-*.rpm

The RPM will install the software and also creates the central configuration directory, /etc/iscsi/ with all necessary configuration files. The example here assumes passwordless authentication, you’ll have to change /etc/iscsi/iscsid.conf if passwords are required for CHAP authentication. Maybe I’ll cover the security aspects in a later post, right now I won’t.

The first thing after installing the iscsi daemons is to start them:

[root@rhel5 ~]# service iscsi start
iscsid dead but pid file exists
Turning off network shutdown. Starting iSCSI daemon:       [  OK  ]
                                                           [  OK  ]
Setting up iSCSI targets: iscsiadm: No records found!
                                                           [  OK  ]
[root@rhel5 ~]#

iSCSI administration on the client

The iscsiadm tool is used for discovery, logging in, and deletion of targets. Once a target has been discovered, the iscsi startup scripts will try to mount it automatically.

But first, the target needs to be discovered for which we’ll use the iscsiadm tool. It comes in 3 modes, discovery, node and session. Use iscsiadm in discovery mode to discover the targets presented by the Openfiler appliance:

# iscsiadm --mode discovery --type sendtargets --portal openfiler

[root@rhel5 ~]# iscsiadm -m discovery -t sendtargets -p openfiler
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.asm_001
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.ocr_a
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.ocr_b
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_a
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_b
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_c

Since we set up all ACL correctly previously, the command should succeed. Note that the portal parameter accepts the name of the openfiler appliance, which is openfiler in my case. You could have used an IP address instead. If there are any issues, the debug option can be used to get to the bottom of these:

[root@rhel5 ~]# iscsiadm -m discovery -d 4 -t sendtargets -p openfiler
iscsiadm: ip openfiler, port 3260, tgpt -1
iscsiadm: Max file limits 1024 1024

iscsiadm: starting sendtargets discovery, address openfiler:3260,
iscsiadm: connecting to 192.168.220.140:3260
iscsiadm: connected local port 47796 to 192.168.220.140:3260
iscsiadm: connected to discovery address 192.168.220.140
iscsiadm: login response status 0000
iscsiadm: Could not open /var/lib/iscsi/send_targets/openfiler,3260 err 2

iscsiadm: discovery process to openfiler:3260 exiting
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.asm_001
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.ocr_a
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.ocr_b
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_a
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_b
192.168.220.140:3260,1 iqn.2009-10.com.openfiler:rac_vg.vote_c

Note that the maximum debug level is 8, but then you are really swamped with information so use the good old 2>&1 | less to view all of that. If that still doesn’t help, check that the host has permissions to query the iSCSI targets and also ensure that the netmask for a single host is 255.255.255.255. There might be firewall settings (iptables -L) in the way.

Log in to the target

Once the target is discovered, you can either manually log in to the target or let the iscsi startup scripts do it for you. The syntax for doing this manually is as follows:

# iscsiadm --mode node --portal openfiler --login --targetname <iqn>

For the ocr_a target the syntax could be as follows:

[root@rhel5 ~]# iscsiadm -m node -T iqn.2009-10.com.openfiler:rac_vg.ocr_a -l
Logging in to [iface: default, target: iqn.2009-10.com.openfiler:rac_vg.ocr_a, portal: 192.168.220.140,3260]
Login to [iface: default, target: iqn.2009-10.com.openfiler:rac_vg.ocr_a, portal: 192.168.220.140,3260]: successful

This repeatedly crashes domUs in Oracle VM Server 2.1.5! Unfortunately I don’t have a support contract so I can’t figure out why that happens but it seems that the dom0 is exclusively allowed to assign device letters which is exactly what the domU does when it discovers the iSCSI target.

The new LUN will appear as a new scsi device to Linux:

Oct 20 18:57:17 rhel5 iscsid: iSCSI logger with pid=7059 started!
Oct 20 18:57:18 rhel5 iscsid: transport class version 2.0-724. iscsid version 2.0-868
Oct 20 18:57:18 rhel5 iscsid: iSCSI daemon with pid=7060 started!
Oct 20 19:04:51 rhel5 kernel: scsi3 : iSCSI Initiator over TCP/IP
Oct 20 19:04:52 rhel5 kernel:   Vendor: OPNFILER  Model: VIRTUAL-DISK      Rev: 0
Oct 20 19:04:52 rhel5 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 04
Oct 20 19:04:52 rhel5 kernel: SCSI device sdc: 131072 512-byte hdwr sectors (67 MB)
Oct 20 19:04:52 rhel5 kernel: sdc: Write Protect is off
Oct 20 19:04:52 rhel5 kernel: SCSI device sdc: drive cache: none
Oct 20 19:04:52 rhel5 kernel: SCSI device sdc: 131072 512-byte hdwr sectors (67 MB)
Oct 20 19:04:52 rhel5 kernel: sdc: Write Protect is off
Oct 20 19:04:52 rhel5 kernel: SCSI device sdc: drive cache: none
Oct 20 19:04:52 rhel5 kernel:  sdc: unknown partition table
Oct 20 19:04:52 rhel5 kernel: sd 3:0:0:0: Attached scsi disk sdc
Oct 20 19:04:52 rhel5 kernel: sd 3:0:0:0: Attached scsi generic sg2 type 0
Oct 20 19:04:52 rhel5 iscsid: received iferror -38

Please ignore the size of the LUN-I am writing this on my laptop on the train home and needed a working example without wasting too much storage I don’t have :)

From this stage on on it’s exactly the same as any other scsi device: you use fdisk to create a partition on it, but don’t create a file system: these LUNs are for RAC and don’t require a file system. Unless of course you’d like to put OCR and voting disk on OCFS2 which I don’t want to do, I prefer ASM for these.

A final note on the automatic mode: after the devices are discovered using iscsiadm as described above, simply restart iscsi services and you’ll be logged in to all devices saving you quite a bit of typing. If you need more control over the process, set node.startup to manual in /etc/iscsi/iscsid.conf

Summary

This part of the article showed how to initiate target discovery and mounting. The iscsi daemons will take care of mounting the devices on the next client reboot, however, there is no device name stability yet which is important up until 11.2 (where OCR and vote disk go into ASM). The final article of the series will add some bits and bops about the different parts of the IQN, shows you how to unpresent a LUN and finally how to guarantee device name stability.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: