Renaming an ASM diskgroup
Posted by Martin Bach on October 14, 2009
This is part of the random new features series in 11.2 I found interesting. Today it’s about renaming disk groups in ASM, something which hasn’t been officially supported until the advent of 11.2. There has been a way to do so in 10.2 but that was an evil hack-but it worked.
Why would you want to rename a disk group?
I asked myself the same question before I had access to the SSSU command line tool. In a nutshell, the SSSU tool I am using in conjunction with an EVA 8100 allows me to clone 1 to n LUNs on the storage array. The advantage is that this way I can create a copy of the 2.5TB production database in very little time compared to a duplicate database. This has many reasons in my current environment, but the most significant is the lack of spindles for everything. These guys would run on a pair of 1TB SATA disks if that was at all possible. But I digress.
Part of the move to ASM entailed creating and using LUNs as ASM disks, and all of them were stamped with ASMLib. The problem arises when I am trying to clone a database on the same host. ASM disks use the first 4k as a header, into which quite a lot of meta information gets stored. Part of the metadata is the name of the disk and the name of the diskgroup the disk is assigned to. As you know, in ASM the name of the disk needs to be unique. Also, you can’t have 2 diskgroups “DATA” with otherwise identical disks in an ASM instance. So in short, invoking the snapclone for LUNs (“disks”) with the same source and destination host doesn’t work. There is no problem snapping the disks on host A and presenting them to host B though. The source of the snapclone is a local standby database if you wondered how the actual implementation worked.
The renamdg command
The renamedg command is new in 11g Release 2 and documented in the storage administrator’s guide. The prerequisite is to unmount the disk group on all cluster nodes, and it only works on cloned disk groups. In other words, it’s perfect for what I need. Unfortunately I don’t have a test system at hand so my venerable laptop will have to do. I am running opensuse 11.1 with xen 3.3.1. I have a domU called “rhel5” with Oracle’s Red Hat 5.2 clone (32bit). My “disks” are under /var/lib/xen/images/rhel5, so first of all I have to copy the disk for disk group FRA which happens to be disk3. A simple cp disk3 disk4 does the trick. I can add block devices to my domU on the fly (can you do this with vmware server? No you can’t), as shown in the following example:
linux-hgcu:/var/lib/xen/images/rhel5 # xm block-attach rhel5 "file:/var/lib/xen/images/rhel5/disk4" "xvdf" w linux-hgcu:/var/lib/xen/images/rhel5 # echo $? 0 The next free disk number can be found out by querying xm: linux-hgcu:~ # xm list -l rhel5 | grep xvd (dev xvda:disk) (dev xvdd:disk) (dev xvdc:disk) (dev xvde:disk) linux-hgcu:~ #
Consider this initial setup – 2 disk groups, DATA and FRA are setup as follows:
SQL> select name,path from v$asm_disk_stat; NAME PATH ------------------------------ ---------------------------------------- DATA_0000 /dev/raw/raw2 FRA_0000 /dev/raw/raw1
Now with the addition of the 3rd disk as described above, things are slightly different:
SQL> select state,path,name from v$asm_disk STATE PATH NAME -------- --------------- ------------------------------ NORMAL /dev/raw/raw1 NORMAL /dev/raw/raw3 NORMAL /dev/raw/raw2 DATA_0000
As you can see, the FRA disks are in a bit of confusion about themselves. Also, I can’t mount the diskgroup:
[oracle@rhel5 ~]$ srvctl start diskgroup -g fra PRCR-1079 : Failed to start resource ora.FRA.dg ORA-15032: not all alterations performed ORA-15017: diskgroup "FRA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "FRA" CRS-2674: Start of 'ora.FRA.dg' on 'rhel5' failed
Let’s follow the documentation now and stop the disk group before we rename it:
[oracle@rhel5 ~]$ srvctl stop diskgroup -g FRA [oracle@rhel5 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE rhel5 ora.FRA.dg ora....up.type OFFLINE OFFLINE ora....ER.lsnr ora....er.type ONLINE ONLINE rhel5 ora.asm ora.asm.type ONLINE ONLINE rhel5 ora.cssd ora.cssd.type ONLINE ONLINE rhel5 ora.diskmon ora....on.type ONLINE ONLINE rhel5 ora.orcl.db ora....se.type OFFLINE OFFLINE ora....srv.svc ora....ce.type OFFLINE OFFLINE ora.stdby.db ora....se.type OFFLINE OFFLINE
Another pleasant surprise is the fact that kfed and kfod are both created by default – in 10g you had to manually link it (see below). So here’s a part from the disk header:
$ kfed read /dev/raw/raw1 ... kfdhdb.dskname: FRA_0000 ; 0x028: length=8 kfdhdb.grpname: FRA ; 0x048: length=3 kfdhdb.fgname: FRA_0000 ; 0x068: length=8 ... kfdhdb.secsize: 512 ; 0x0b8: 0x0200 kfdhdb.blksize: 4096 ; 0x0ba: 0x1000 kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000 kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80 kfdhdb.dsksize: 3992 ; 0x0c4: 0x00000f98 ...
So there’s a lot of interesting information in there, and for once it’s almost readable for non-support technician like me.
That matches with what ASM tells us. Let’s see which parameters are available to renamedg:
[oracle@rhel5 rhel5]$ renamedg -help NOTE: No asm libraries found in the system Parsing parameters.. phase Phase to execute, (phase=ONE|TWO|BOTH), default BOTH dgname Diskgroup to be renamed newdgname New name for the diskgroup config intermediate config file check just check-do not perform actual operation, (check=TRUE/FALSE), default FALSE confirm confirm before committing changes to disks, (confirm=TRUE/FALSE), default FALSE clean ignore errors, (clean=TRUE/FALSE), default TRUE asm_diskstring ASM Diskstring (asm_diskstring='discoverystring', 'discoverystring1' ...) verbose verbose execution, (verbose=TRUE|FALSE), default FALSE keep_voting_files Voting file attribute, (keep_voting_files=TRUE|FALSE), default FALSE
The tool can work in 2 phases, or do the work in one go. When using multiple phases, it first generates a configuration file which then can be loaded into the header.
Now let’s try the phased approach – start with phase 1:
[oracle@rhel5 ~]$ renamedg phase=one dgname=FRA newdgname=FRABIS confirm=true verbose=true config=/tmp/cfg NOTE: No asm libraries found in the system Parsing parameters.. Parameters in effect: Old DG name : FRA New DG name : FRABIS Phases : Phase 1 Discovery str : (null) Confirm : TRUE Clean : TRUE Raw only : TRUE renamedg operation: phase=one dgname=FRA newdgname=FRABIS confirm=true verbose=true config=/tmp/cfg Executing phase 1 Discovering the group Performing discovery with string: ERROR: -5(Duplicate disk FRA:FRA_0000)KFNDG-00410: file not found; arguments:  Terminating kgfd context 0xb7e7a050
Ooops, that didn’t work. As it turned out I shouldn’t have bound the new disk group to the raw device. Similarly I assume you wouldn’t issue an /etc/init.d/oracleasm scandisks immediately after disk addition to your system. Let’s reverse the that:
[root@rhel5 ~]# raw -qa /dev/raw/raw1: bound to major 202, minor 49 /dev/raw/raw2: bound to major 202, minor 65 /dev/raw/raw3: bound to major 202, minor 81 [root@rhel5 ~]# raw /dev/raw/raw3 0 0 /dev/raw/raw3: bound to major 0, minor 0 [root@rhel5 ~]# raw -qa /dev/raw/raw1: bound to major 202, minor 49 /dev/raw/raw2: bound to major 202, minor 65 [root@rhel5 ~]#
And try phase 1 again:
[oracle@rhel5 ~]$ renamedg phase=one dgname=FRA newdgname=FRABIS confirm=true verbose=true NOTE: No asm libraries found in the system Parsing parameters.. Parameters in effect: Old DG name : FRA New DG name : FRABIS Phases : Phase 1 Discovery str : (null) Confirm : TRUE Clean : TRUE Raw only : TRUE renamedg operation: phase=one dgname=FRA newdgname=FRABIS confirm=true verbose=true Executing phase 1 Discovering the group Performing discovery with string: Identified disk UFS:/dev/raw/raw1 with disk number:0 and timestamp (32926132 385312768) Checking for hearbeat... Re-discovering the group Performing discovery with string: Identified disk UFS:/dev/raw/raw1 with disk number:0 and timestamp (32926132 385312768) Checking if the diskgroup is mounted Checking disk number:0 Checking if diskgroup is used by CSS Generating configuration file.. Completed phase 1 Terminating kgfd context 0xb7e61050 [oracle@rhel5 ~]$
“Terminating kgfd context” sounds like “didn’t work” in the bad way to me-but the good news is that it completed phase 1. The config file is rather unremarkable, I was expecting something else:
[oracle@rhel5 ~]$ cat renamedg_config /dev/raw/raw1 FRA FRABIS [oracle@rhel5 ~]$ renamedg phase=two dgname=fra newdgname=frabis config=/home/oracle/renamedg_config NOTE: No asm libraries found in the system Parsing parameters.. renamedg operation: phase=two dgname=fra newdgname=frabis config=/home/oracle/renamedg_config Executing phase 2 Completed phase 2
Let’s start ASM now and check if it worked:
srvctl start asm SQL> alter diskgroup frabis mount; Diskgroup altered. SQL> select name,path from v$asm_disk; NAME PATH ------------------------------ ---------------------------------------- FRA_0000 /dev/raw/raw3 DATA_0000 /dev/raw/raw2 FRA_0000 /dev/raw/raw1 SQL> select name,state,type,voting_files 2 from v$asm_diskgroup; NAME STATE TYPE V ------------------------------- ----------- ------ DATA MOUNTED EXTERN N FRA MOUNTED EXTERN N FRABIS MOUNTED EXTERN N
Voila! Problem solved. The renamedg command can also be executed in one go – making it very easy to script the whole lot.
In addition to this, I have managed to use kfed to modify an ASM disk header in 10g Release 2 thus effectively renaming the disk group. I think it’s an evil hack, it’s not supported (but there is an internal metalink note) and shouldn’t really be done. Is there any interest to publish this? Please comment on the article if yes and I’ll draft it up.