Idrac监控报错:登陆ideac卡后如下如所示:硬盘状态是红叉,状态是外来

wKiom1g1a1zD5MxEAADsDNwti64997.png命令行安装MegaCli

rpm -ivh MegaCli-8.07.08-1.noarch.rpm 

查看包的安装路径

rpm -ql MegaCli-8.07.08-1.noarch

/opt/MegaRAID/MegaCli/MegaCli

/opt/MegaRAID/MegaCli/MegaCli64

/opt/MegaRAID/MegaCli/libstorelibir-2.so.14.07-0

执行状态检测命令:

/opt/MegaRAID/MegaCli/MegaCli64 -pdlist -aall |grep 'Firmware state'

Firmware state: Unconfigured(good), Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

此时的状态如下:

############

Enclosure Device ID: 32

Slot Number: 4

###Drive's position: DiskGroup: 0, Span: 2, Arm: 1   ###应该有这个信息的,但是这个没有

Enclosure position: 1

Device Id: 4

WWN: 5000C5005AAD3E28

Sequence Number: 1

Media Error Count: 0

Other Error Count: 0

Predictive Failure Count: 102

Last Predictive Failure Event Seq Number: 41501

PD Type: SAS


Raw Size: 558.911 GB [0x45dd2fb0 Sectors]

Non Coerced Size: 558.411 GB [0x45cd2fb0 Sectors]

Coerced Size: 558.375 GB [0x45cc0000 Sectors]

Sector Size:  0

Firmware state: Unconfigured(good), Spun Up

Device Firmware Level: ES66

Shield Counter: 0

Successful diagnostics completion on :  N/A

SAS Address(0): 0x5000c5005aad3e29

SAS Address(1): 0x0

Connected Port Number: 0(path0) 

Inquiry Data: SEAGATE ST3600057SS     ES666SL5A74S            

FDE Capable: Not Capable

FDE Enable: Disable

Secured: Unsecured

Locked: Unlocked

Needs EKM Attention: No

Foreign State: Foreign 

Foreign Secure: Drive is not secured by a foreign lock key

Device Speed: 6.0Gb/s 

Link Speed: 6.0Gb/s 

Media Type: Hard Disk Device

Drive Temperature :45C (113.00 F)

PI Eligibility:  No 

Drive is formatted for PI information:  No

PI: No PI

Port-0 :

Port status: Active

Port's Linkspeed: 6.0Gb/s 

Port-1 :

Port status: Active

Port's Linkspeed: Unknown 

Drive has flagged a S.M.A.R.T alert : Yes

############




执行导入命令

/opt/MegaRAID/MegaCli/MegaCli64  -CfgForeign -Import -aall        

Foreign configuration is imported on controller 0.

Exit Code: 0x00

再次执行状态检测命令:

/opt/MegaRAID/MegaCli/MegaCli64 -pdlist -aall |grep 'Firmware state'

Firmware state: Rebuild

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

Firmware state: Online, Spun Up

查询 Rebuild 进度:

/opt/MegaRAID/MegaCli/MegaCli64 -pdrbld -showprog  -physdrv[32:0] -a0         

显示如下:                          

Rebuild Progress on Device at Enclosure 32, Slot 0 Completed 38% in 54 Minutes.

Exit Code: 0x00

或者

/opt/MegaRAID/MegaCli/MegaCli64 -pdrbld -ProgDsply  -physdrv[32:0] -a0

显示如下:

      Rebuild progress of physical drives...


  Enclosure:Slot               Percent Complete                       Time Elps

       032 :00     ####################***40 %*********************** 00:56:40


    Press <ESC> key to quit...


备注:

Enclosure Device ID: 32

Slot Number: 0

以上两条信息通过/opt/MegaRAID/MegaCli/MegaCli64 -pdlist -aall |less  来查看




扫描外来配置的个数:

# /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -scan -a0

清除外来配置:

# /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -clear -a0

再次扫描外来配置的个数:

# /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -scan -a0 


参考:http://erikimh.com/raid-rebuilding-foreign-disk-by-hand/


Megacli是一款管理维护硬件RAID的工具,有LSI公司提供,LSI公司的raid卡,使用的比较广泛。我们可以通过megacli了解当前raid卡的所有信息,包括raid卡的型号,raid的阵列类型,raid上的磁盘状态,也可以通过它来直接创建阵列,在线添加磁盘等。

一,Megacli工具安装

可以在LSI公司的官网直接下载工具:

http://www.lsi.com/downloads/Public/Nytro/downloads/Nytro%20XD/MegaCli_Linux.zip

 下载完成之后,是一个zip包,然后解压,安装:

# unzip MegaCli_Linux.zip

# cd MegaCli_Linux

# ls
megacli_8.07.08-1_all.deb  MegaCli-8.07.08-1.noarch.rpm  MegaSAS.log

# rpm -ivh MegaCli-8.07.08-1.noarch.rpm

安装成功之后,命令的默认安装路径为:

# /opt/MegaRAID/MegaCli/MegaCli64

二,查看磁盘的状态

作用:显示Raid卡型号,Raid设置,整列类型,Disk相关信息

# /opt/MegaRAID/MegaCli/MegaCli64 -cfgdsply -aALL|less

1,查看raid整列类型和大小

spacer.gifwKioL1POQKnA2giGAAG8e41-998140.jpg

如上图所示:

(1)通过RAID Level字段得知,disk group 0做的是一个raid5;

(2)disk group 0的大小为1.6TB;

2,查看raid的cache策略

wKioL1POQLvCUGw6AAJSbzEs7lc391.jpg

如上图所示,raid的默认以及当前生效的cache策略为writeback(还有一种cache策略为WriteThrough)


策略说明:

(1). 第一段: WriteBack, WriteThrough

* WriteBack:进行写操作时,将数据写入RAID卡缓存,并直接返回,RAID卡控制器将在系统负载低或者Cache满了的情况下把数据写入硬盘。该设置会大大提升RAID卡写性能,绝大多数的情况下会降低系统IO负载。 数据的可靠性由RAID卡的BBU(Battery Backup Unit)进行保证。大多数情况下,我们都使用这种策略。

* WriteThrough: 数据写操作不使用缓存,数据直接写入磁盘。RAID卡写性能明显下降,在大多数情况下该设置会造成系统IO负载上升。特别对于io负载很大的服务,表现特别明显。

(2). 第二段: ReadAheadNone, ReadAdaptive, ReadAhead.

* ReadAheadNone: 不开启预读。这是默认的设置

* ReadAhead: 在读操作时,预先把后面顺序的数据加载入Cache,在顺序读取时,能提高性能,相反会降低随机读的性能。

* ReadAdaptive: 自适应预读,当Cache memory和IO空闲时,采取顺序预读,平衡了连续读性能及随机读的性能,需要消耗一定的计算能力。

(3). 第三段: Direct, Cached.

* Direct: Direct IO模式,读操作不缓存到cache memory中,数据将同时传输到cache中和应用,如果接下来要读取相同的数据块,则直接从Cache memory中获取. 这是默认的设置

* Cached: Cached IO模式,所有读操作都会缓存到cache memory中。

(4). 第四段: Write Cache OK if Bad BBU, No Write Cache if Bad BBU

* Write Cache OK if Bad BBU: 在BBU有问题时(如电池失效), 依旧使用Write Cache, 有一定的数据丢失风险.

* No Write Cache if Bad BBU: 在BBU有问题时, 不使用Write Cache

策略自动切换的问题由于MegaSAS RAID卡默认采用No Write Cache if Bad BBU的设置,将可能发生Write Cache策略变更的情况(由WriteBack变成WriteThrough),导致写性能下降,如果该自动变更发生在业务高峰且系统Io负载高的时候,可能会引发不可预测的问题,如卡机。以下原因将造成Write Cache策略的变更.

(1). RAID卡进入BBU Learn Cycle: 详细介绍见下面

(2). 检测到某些电池故障,如电池容量过低等,一般是电池老化带来的影响,IBM建议一年更换一次RAID卡电池

(3). 没有安装电池, 部分服务器购买时不带电池,导致被自动设置为WriteThrough

3,判定磁盘是否损坏

spacer.gifwKioL1POQM2ySMGxAANV_4sR2wo177.jpg

如上图所示,我们一般通过如上5个值,来判断磁盘是否应该报修:

1,Media Error

磁盘存在错误,可能是磁盘有坏道。值越大,越危险。根据磁盘状况,一般大于100报修更换。

2,Other Error

磁盘存在未知的错误,可能是磁盘松动,需要重新再插入。根据磁盘状况,一般大于100报修更换。

3,Predictive Failure Count

磁盘的预警数。一般大于0,就报修更换。

4,Last Predictive Failure Event Seq Number

最后一条预警的时间序列号。这个值不为0,肯定Predictive Failure Count也不为0

5,Firmware state

磁盘目前的状态。一般有9种,即

(1)Unconfigured Good – A drive accessible to the RAID controller but not configured as a part of

a virtual drive or as a hot spare.

(2)Online – A drive that can be accessed by the RAID controller and will be part of the virtual

drive.

(3)Rebuild – A drive to which data is being written to restore full redundancy for a virtual drive.

(4)Failed – A drive that was originally configured as Online or Hot Spare, but on which the

firmware detects an unrecoverable error.

(5)Unconfigured Bad – A drive on which the firmware detects an unrecoverable error; the drive

was Unconfigured Good or the drive could not be initialized.

(6)Missing – A drive that was Online, but which has been removed from its location.

(7)Offline – A drive that is part of a virtual drive but which has invalid data as far as the RAID

configuration is concerned.

(8)Hot Spare – A drive that is configured as a hot spare.

(9)None – A drive with an unsupported flag set. An Unconfigured Good or Offline drive that has

completed the prepare for removal operation.

(10)还有一种特殊的状态copyback:

从磁盘组中把数据复制到非磁盘组的磁盘中,然后等failed的盘更换之后,再从这个非磁盘组的磁盘中把数据给copyback回来。

做hot spare的盘,会出现这种情况:即原来的hot spare盘只是临时存放了数据,等failed的盘更换之后,把数据从hotspare的盘中复制回来,正常使用的还是新更换的盘,hot spare的盘永久做hot spare。










1.显示Rebuid进度

/opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ShowProg -physdrv[20:2] -aALL

2.查看E S

/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aAll -NoLog | grep -Ei "(enclosure|slot)"

3.查看所有硬盘的状态

/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aAll -NoLog

4.查看所有Virtual Disk的状态

/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aAll -NoLog

RAID Level对应关系:

RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0

RAID 1

RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0

RAID 0

RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3

RAID 5

RAID Level : Primary-1, Secondary-3, RAID Level Qualifier-0

RAID 10

5.在线做Raid

/opt/MegaRAID/MegaCli/MegaCli64 -CfgLdAdd -r0[0:11] WB NORA Direct CachedBadBBU -strpsz64 -a0 -NoLog
/opt/MegaRAID/MegaCli/MegaCli64 -CfgLdAdd -r5 [12:2,12:3,12:4,12:5,12:6,12:7] WB Direct -a0

6.点亮指定硬盘(定位)

/opt/MegaRAID/MegaCli/MegaCli64 -PdLocate -start -physdrv[252:2] -a0

7.清除Foreign状态

/opt/MegaRAID/MegaCli/MegaCli64 -CfgForeign -Clear -a0

8.查看RAID阵列中掉线的盘

/opt/MegaRAID/MegaCli/MegaCli64 -pdgetmissing -a0

9.替换坏掉的模块

/opt/MegaRAID/MegaCli/MegaCli64 -pdreplacemissing -physdrv[12:10] -Array5 -row0 -a0

10.手动开启rebuid

/opt/MegaRAID/MegaCli/MegaCli64 -pdrbld -start -physdrv[12:10] -a0

11.查看Megacli的log

/opt/MegaRAID/MegaCli/MegaCli64 -FwTermLog dsply -a0 > adp2.log

12.设置HotSpare

/opt/MegaRAID/MegaCli/MegaCli64-pdhsp -set[-Dedicated[-Array2]][-EnclAffinity][-nonRevertible]-PhysDrv[4:11]-a0
/opt/MegaRAID/MegaCli/MegaCli64-pdhsp -set[-EnclAffinity][-nonRevertible]-PhysDrv[321}]-a0

 

MegaCli -PDHSP   -Set   -Dedicated  -Array0  -physdrv[E:S] -a0   添加局部热备盘,其中array0表示第0个raid 

MegaCli  -pdhsp  -set   -physdrv[E:S]  -a0                                       添加全局热备盘

MegaCli  -pdhsp  -rmv  -physdrv[E:S]  -a0                                      移除全局和热备局部热备

 

13.关闭Rebuild

/opt/MegaRAID/MegaCli/MegaCli64 -AdpAutoRbld -Dsbl -a0

14.设置rebuild的速率

/opt/MegaRAID/MegaCli/MegaCli64 -AdpSetProp RebuildRate -30 -a0

 

 

附其他详细命令:

1. 常用命令:

#/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aALL  查raid级别

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpAllInfo -aALL  查raid卡信息

#/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL  查看硬盘信息

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -aAll  查看电池信息

#/opt/MegaRAID/MegaCli/MegaCli64 -FwTermLog -Dsply -aALL  查看raid卡日志

#/opt/MegaRAID/MegaCli/MegaCli64 -adpCount  【显示适配器个数】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpGetTime –aALL  【显示适配器时间】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpAllInfo -aAll  【显示所有适配器信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -LALL -aAll  【显示所有逻辑磁盘组信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aAll  【显示所有的物理信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuStatus -aALL |grep ‘Charger Status’ 【查看充电状态】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuStatus -aALL【显示BBU状态信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuCapacityInfo -aALL【显示BBU容量信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuDesignInfo -aALL 【显示BBU设计参数】

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuProperties -aALL 【显示当前BBU属性】

#/opt/MegaRAID/MegaCli/MegaCli64 -cfgdsply -aALL 【显示Raid卡型号,Raid设置,Disk相关信息】

#/opt/MegaRAID/MegaCli/MegaCli64 -adpallinfo -aall|grep -i temp  查看温度

 

2.  磁带状态的变化,从拔盘,到插盘的过程中

Device |Normal|Damage|Rebuild|Normal

Virtual Drive |Optimal|Degraded|Degraded|Optimal

Physical Drive |Online|Failed –> Unconfigured|Rebuild|Online

 

3. 查看磁盘缓存策略

#/opt/MegaCli -LDGetProp -Cache -L0 -a0

or

#/opt/MegaCli -LDGetProp -Cache -L1 -a0

or

#/opt/MegaCli -LDGetProp -Cache -LALL -a0

or

#/opt/MegaCli -LDGetProp -Cache -LALL -aALL

or

#/opt/MegaCli -LDGetProp -DskCache -LALL -aALL

 

4.设置磁盘缓存策略

缓存策略解释:

WT (Write through

WB (Write back)

NORA (No read ahead)

RA (Read ahead)

ADRA (Adaptive read ahead)

Cached

Direct

例子:

#/opt/MegaCli -LDSetProp WT|WB|NORA|RA|ADRA -L0 -a0

or

#/opt/MegaCli -LDSetProp -Cached|-Direct -L0 -a0

or

enable / disable disk cache

#/opt/MegaCli -LDSetProp -EnDskCache|-DisDskCache -L0 -a0

 

/opt/MegaRAID/MegaCli/MegaCli64 -DiscardPreservedCache -Lall -a0 -NoLOG    【清空缓存】

 

5. 创建一个 raid5 阵列,由物理盘 2,3,4 构成,该阵列的热备盘是物理盘 5

#/opt/MegaCli -CfgLdAdd -r5 [1:2,1:3,1:4] WB Direct -Hsp[1:5] -a0

6. 创建阵列,不指定热备

#/opt/MegaCli -CfgLdAdd -r5 [1:2,1:3,1:4] WB Direct -a0

7. 删除阵列

#/opt/MegaCli -CfgLdDel -L1 -a0

8. 在线添加磁盘

#/opt/MegaCli -LDRecon -Start -r5 -Add -PhysDrv[1:4] -L1 -a0

9. 阵列创建完后,会有一个初始化同步块的过程,可以看看其进度。

#/opt/MegaCli -LDInit -ShowProg -LALL -aALL

或者以动态可视化文字界面显示

#/opt/MegaCli -LDInit -ProgDsply -LALL -aALL

10. 查看阵列后台初始化进度

#/opt/MegaCli -LDBI -ShowProg -LALL -aALL

或者以动态可视化文字界面显示

#/opt/MegaCli -LDBI -ProgDsply -LALL -aALL

11. 指定第 5 块盘作为全局热备

#/opt/MegaCli -PDHSP -Set [-EnclAffinity] [-nonRevertible] -PhysDrv[1:5] -a0

12. 指定为某个阵列的专用热备

#/opt/MegaCli -PDHSP -Set [-Dedicated [-Array1]] [-EnclAffinity] [-nonRevertible] -PhysDrv[1:5] -a0

13. 删除全局热备

#/opt/MegaCli -PDHSP -Rmv -PhysDrv[1:5] -a0

14. 将某块物理盘下线/上线

#/opt/MegaCli -PDOffline -PhysDrv [1:4] -a0

 

15. 查看物理磁盘重建进度

#/opt/MegaCli -PDRbld -ShowProg -PhysDrv [1:5] -a0

或者以动态可视化文字界面显示

#/opt/MegaCli -PDRbld -ProgDsply -PhysDrv [1:5] -a0