ASM在线迁移LUN遇到的问题
在2个节点上都配置好了RAW之后，确保在两边ll /dev/raw都能看到新配置好的raw，
在节点1用命令行的方式添加新LUN并删除旧的RAW，执行了一会报错：
SQL> Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5' drop disk '/dev/raw/raw1' rebalance power 4 nowait;
Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5' drop disk '/dev/raw/raw1' rebalance power 4 nowait
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15054: disk "/DEV/RAW/RAW1" does not exist in diskgroup "ZXDG"
SQL>
在两边的v$asm_disk中查看，发现节点1上已经能看到新LUN了，但节点2上却看不到新LUN。
SQL> select disk_number from v$asm_disk;
DISK_NUMBER
-----------
0
1
0
于是在节点2上重新添加了一次：
SQL>
SQL>
SQL> Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5' ;
Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15031: disk specification '/dev/raw/raw5' matches no disks
ORA-15025: could not open disk '/dev/raw/raw5'
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 42
Additional information: 255
Additional information: -750856672
ORA-15031: disk specification '/dev/raw/raw4' matches no disks
ORA-15025: could not open disk '/dev/raw/raw4'
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 42
Additional information: 255
Additional information: -750856672
上面的报错一般是raw的属主、权限等问题，也可能是节点2识别raw有问题（ll /dev/raw的结果有很大的欺骗性），
检查发现raw的属主、权限都没问题，那么是不是节点2未能正确识别lun呢，在节点2上执行partprobe之后再次添加disk：
SQL>
SQL>
SQL> Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5'
2 ;
Alter diskgroup ZXDG add disk '/dev/raw/raw4','/dev/raw/raw5'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15033: disk '/dev/raw/raw4' belongs to diskgroup "ZXDG"
ORA-15033: disk '/dev/raw/raw5' belongs to diskgroup "ZXDG"
这回报ORA-15032与ORA-15033了，显示raw4/raw5已经属于DG的一部分了，
但再次查看v$asm_disk，发现raw4,raw5的状态有异常，group_number为0，mount_status为closed，没有分配name：
group_number disk_number mount_status header_status mode_status state path name
1 0 cached member online dropping /dev/raw/raw1 ZXDG_0000
0 1 closed member online normal /dev/raw/raw4
0 2 closed member online normal /dev/raw/raw5
而此时raw1已经处于dropping状态了，因为raw4/raw5没有被正常add到dg，所以无法对数据做rebalance，此时不允许将raw1从dg中drop掉，所以状态一直为dropping。
以防万一，先undrop：
SQL> Alter diskgroup ZXDG undrop disks ;
Diskgroup altered.
由于add raw的时候asm并有为raw4/raw5分配name,也无法正常将之从dg中删除：
SQL> Alter diskgroup ZXDG drop disk '/dev/raw/raw4';
Alter diskgroup ZXDG drop disk '/dev/raw/raw4'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15054: disk "/DEV/RAW/RAW4" does not exist in diskgroup "ZXDG"
读取3个raw的metadata，庆幸的是发现metadata并没有出现混乱的情况，如下：
[oracle@zxdb01 ~]$ kfed read /dev/raw/raw1 > /tmp/raw1
[oracle@zxdb01 ~]$ kfed read /dev/raw/raw4 > /tmp/raw4
[oracle@zxdb01 ~]$ kfed read /dev/raw/raw5 > /tmp/raw5
[oracle@zxdb01 ~]$ diff /tmp/raw4 /tmp/raw5
6c6
< kfbh.block.obj: 2147483649 ; 0x008: TYPE=0x8 NUMB=0x1
---
> kfbh.block.obj: 2147483650 ; 0x008: TYPE=0x8 NUMB=0x2
20c20
< kfdhdb.dsknum: 1 ; 0x024: 0x0001
---
> kfdhdb.dsknum: 2 ; 0x024: 0x0002
23c23
< kfdhdb.dskname: ZXDG_0001 ; 0x028: length=10
---
> kfdhdb.dskname: ZXDG_0002 ; 0x028: length=10
25c25
< kfdhdb.fgname: ZXDG_0001 ; 0x068: length=10
---
> kfdhdb.fgname: ZXDG_0002 ; 0x068: length=10
[oracle@zxdb01 ~]$ diff /tmp/raw1 /tmp/raw4
6,7c6,7
< kfbh.block.obj: 2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
< kfbh.check: 203544188 ; 0x00c: 0x0c21d67c
---
> kfbh.block.obj: 2147483649 ; 0x008: TYPE=0x8 NUMB=0x1
> kfbh.check: 3389207210 ; 0x00c: 0xca0332aa
20c20
< kfdhdb.dsknum: 0 ; 0x024: 0x0000
-- |